Hi,

I've been using Eigen for some time now, and I've often noticed strange changes in the numerical results, as a result of seemingly unrelated changes in the code. In the most recent example, I have a large program that involves thousands of matrix operations in each iteration. If I make the following change in the code, in just one very small part of it

A.noalias() = -B *C.transpose() * D;

versus

A = -B *C.transpose() * D;

the results change. In each iteration the change is negligble, but after a few thousands of iterations the difference becomes obvious, and could amount to 1-5% change in the results. Due to the nature of the program one cannot say that either of the two results is "wrong", they are just different.

In all the above, A,B,C,D are 3 x 3 matrices (A is a 3x3 block), allocated statically, and it is absolutely certain that no aliasing happens here (besides, if aliasing was taking place, the results would be way off, not just a tiny amount). This is just one example where I have observed changes it the results. One of the most weird things I've seen is that including a cout command at some place in the code changes the numerical results. Also, in some cases compiling with or without -DNDEBUG gives different numerical results.

Obviously, the most likely explanation would be that there is a bug somewhere in the code, that causes wrong memory accesses, or something like that. However, I have not been able to find any bug (this is a very stable piece of software developed and tested over many months), and I get no failing assertions, or any other issues. Any possible test I could come up with indicates that the code is OK.

So far I had mostly ignored this, assuming that it is just due to the compiler using different optimizations, which cause operations to occur in different order, leading to slight numerical differences. However, it remains very strange. It also makes comparing different versions of the code difficult, since it is impossible to say what causes differences in the results of different versions.

I would like to see if anyone else has observed similar behavior, and if this is something normal or something to be worried about.

Thank you in advance for any insights/suggestions!

You could try to run your code with valgrind to detect memory issues. I also guess you are compiling in 32 bits, isn't it? This could explain the difference w/o .alias() in which case the extra temporary could enforce some intermediate values to be copied from the 80bits floating point registers to memory thus losing the extra bits of precisions. Also, do you compile with aggressive optimization flags?

Yes, I am compiling with 32 bits, and using -O3, -ffast-math, -msse3.

So, as I understand from your response, you believe that slight changes in accuracy are to be expected, correct?

Thanks!

yes the most reasonable guess is that's due to rounding steps between the 80 and 64 bits of precisions which can occurs at "random" places in the code. You can enforce SSE for scalar ops with: -mfpmath=sse and the randomness should go away.