I'm using the C++ Eigen header library, and I've got millions of largish VectorXd objects (10K elements) in a cache (an array). Then I'm trying to retrieve these vectors one at a time to be used in some computation.

Now, moving these large VectorXds in and out of the cache is expensive because it seems to induce copies, profiling using linux perf shows the program is spending a lot of time in:

Code: Select all

      :        /* Store two DPFP values.  The address must be 16-byte aligned.  */
       :        extern __inline void __attribute__((__gnu_inline__, __always_inline__, __artificial__))
       :        _mm_store_pd (double *__P, __m128d __A)
       :        {
       :          *(__m128d *)__P = __A;
 14.70 :          40cceb:       movapd %xmm2,(%rcx,%r9,8)
  0.67 :          40ccf1:       mov    (%rbx),%rcx

So my intention is to just store the data in the cache once and later move around pointers instead of objects.

However, when the final computation using the VectorXd is done, it requires an object not a pointer, and as I understand it dereferencing to a local object it will cause an copy (expensive again).

Would it make sense to move around double pointers and only use Map to instantiate VectorXd objects just before I need the computation done, is Map an expensive operation?

Any suggestions welcome.

Are you sure you benchmarked with optimization ON? (e.g., with -O2) It's weird that the _mm_store_pd intrinsic appears in the profiler.

Constructing a Map object is like a no-op.

I'm compiling Eigen 3.2.0 using g++ 4.6.3 on 64-bit Ubuntu 12.04 for Intel Xeon:

Code: Select all

g++ -Iboost -IEigen -march=native -O3 -DNDEBUG -fopenmp -lgomp -lm -fno-omit-frame-pointer -funroll-loops -ffast-math -ggdb3

I will try the Map approach.