Hi,

I'm playing with some matlab code porting to Eigen. In one case however, I am getting a hugely different result from a matrix multiplication.
Consider these input matrices:

a: Dim 15000x128. Value range 10^6 to 10^7. Numerical difference to octave 10^-13 %
b: Dim 128x128. Value range 10^-9 to 10^-13. No difference down to machine epsilon.
c: Dim 128x600. Value range 10^2 to 10^4. Numerical difference to octave 10^-13 %.

First operation:
m = a * b:

Resulting dim is 15000x128, value range 10^-2 to 10^-3, the numerical difference to octave's result is +/- 0.5%, would be acceptable.

Second operation:

n = m * c:

Resulting dim is 14700x600, value range is 10^0 to 10^1. With an error of up to +/- 200%!!

All computation is done with "double" as data type.

Question: Is octave correct, or Eigen? What is Octave doing different? Or is it the input error of 10^-13%? What can I do to get a correct result?

Octave relies on a blas implementation for the matrix products. So actually, you are comparing Eigen to another BLAS lib, the question is which one are you using ? The default fortran one ? ATLAS ? GOTO blas ? MKL ?

What is your architecture ? 32 or 64 bits ? If 32 bits, did you enabled SSE ?

If you have a 64 bit system, you can try to disable SSE vectorization in Eigen by defining EIGEN_DONT_VECTORIZE, if you still get the same difference, you can try to disable the use of SSE instr. at the GCC level : -m32 ....

I also observe huge error between computational results of matrix multiplication between Eigen and Matlab (and also CUBLAS). It may take place when the value of matrix elements are very large (around 10^3-10^4).

actually I did not looked close enough at your relative errors in the inputs, but the very large error you get at the end is easily explained by them. So just try to start from the same inputs !

The input are generated with C++ program who directly call Eigen and CUBLAS to compute the results and save it into a HDF5 file. Then matlab read that HDF5 file and do the same computation.

Each of three results from Eigen, CUBLAS and Matlab is different with the other two.

This generally does not happen when the elements of matrix are small, such as in [0,1], but happens when they are large.

Indeed. I made an experiment with all eight combinations of a,b,c either produced by Eigen, or produced by octave, and really, the tiny differences of 10^(-13)% in a and c seem to explain the result.
So both Eigen and Octave are right, and the computation is sensitive to these small differences in input.

Thanks!