UPDATE: This issue was related to overloaded new operator. If I include only it works.

Is there a way to compile Eigen so that it doesn't overload operator new?

---------------------------------------

Hi,

Including #include does not compile on Visual Studio 2008. I've tried both the 2.0 stable from SVN and the 2.0.1 release package.

Below is the error message copy-pasted:

corematrix.h(139) : error C2059: syntax error : 'constant'
corematrix.h(518 ) : see reference to class template instantiation 'Eigen::Matrix' being compiled
corematrix.h(139) : error C2091: function returns function
corematrix.h(139) : error C2802: static member 'operator new' has no formal parameters
corematrix.h(139) : error C2333: 'Eigen::Matrix::operator new' : error in function declaration; skipping function body
corematrix.h(139) : error C2059: syntax error : 'constant'
corematrix.h(139) : error C2090: function returns array
corematrix.h(139) : error C2059: syntax error : 'return'
corematrix.h(139) : error C2238: unexpected token(s) preceding ';'
corematrix.h(139) : error C2143: syntax error : missing ';' before 'void'
coremap.h(109) : error C2244: 'Eigen::MatrixBase::MatrixBase' : unable to match function definition to an existing declaration
1> d:usersjanidocumentsvisual studio 2008projectseigeneigensrccorematrixbase.h(140) : see declaration of 'Eigen::MatrixBase::MatrixBase'


Thanks,
Jani

I just updated trunk/ so that you can now prevent Eigen from overloading new by just #defining EIGEN_DONT_ALIGN.

If you want to stay with 2.0, you can prevent overloading of operator new by editing the file Eigen/src/Core/util/Memory.h and setting the value of EIGEN_ARCH_WANTS_ALIGNMENT to 0 instead of 1.

Disabling alignment also disables vectorization and makes your code binary-incompatible with other eigen-based code.

That said, we really mean to support alignment and vectorization on MSVC 2008.

Can you help us debug that? First, at Matrix.h line 139, replace the macro EIGEN_MAKE_ALIGNED_OPERATOR_NEW_IF(NeedsToAlign) by its value, so that we get meaningful line numbers in the error messages. Thanks!

please, could you give us more information so that we can reproduce the issue, because here all unit tests pass with MSVC 2008...

Hi,

I took the trunk version and it compiled better.

(code removed)

bjacob wrote:I just updated trunk/ so that you can now prevent Eigen from overloading new by just #defining EIGEN_DONT_ALIGN.

This is how you can reproduce it:

// MSVC debug memory allocation routines for finding leaks
#if defined(_DEBUG) && defined(_MSC_VER) && !defined(new)
#undef new
#define _CRTDBG_MAP_ALLOC
#include
#include
#define new new(_NORMAL_BLOCK,__FILE__,__LINE__)
#endif

// #define EIGEN_DONT_ALIGN // compiles ok now if EIGEN_DONT_ALIGN
#include

int main( int argc, char* argv[] )
{
_CrtSetDbgFlag( _CrtSetDbgFlag(_CRTDBG_REPORT_FLAG) | _CRTDBG_LEAK_CHECK_DF );
new int(123); // memory leak!

MatrixXf a;
a = MatrixXf::Zero( 123, 123 );
return 0;
}

Debug output memory leak report after execution:

d:usersjanidocumentsvisual studio 2008projectslsls.cpp(117) : {66} normal block at 0x00AD4FA8, 4 bytes long.

ggael wrote:please, could you give us more information so that we can reproduce the issue, because here all unit tests pass with MSVC 2008...

Note: You can work around the issue by placing #define new stuff after #include , but it would be nice to use the source code line based memory leak debugging also for template based libs like Eigen.

kajala wrote:template inline T* ei_aligned_new(size_t size)
{
void *void_result = ei_aligned_malloc(sizeof(T)*size);
#ifdef EIGEN_DONT_ALIGN
return ::new T[size];
#else
return ::new(void_result) T[size];
#endif
}

no, this is not a solution because here:

Code: Select all

return ::new(void_result) T[size];

we are using the "placement-new" operator which calls the constructor of T on each element of the allocated data.
With your change, you now have a big memory leak since the requested memory is allocated twice (in case the ei_aligned_new function is called that is unlikely to happen if EIGEN_DONT_ALIGN if defined)

kajala wrote:// MSVC debug memory allocation routines for finding leaks
#if defined(_DEBUG) && defined(_MSC_VER) && !defined(new)
#undef new
#define _CRTDBG_MAP_ALLOC
#include
#include
#define new new(_NORMAL_BLOCK,__FILE__,__LINE__)
#endif

ok, the problem is that you are refining new in a way which is not C++ compatible. I mean, this redefinition is not compatible with custom operator new, nor with the placement new operator.

To workaround that, we could detect if new is defined as a macro, push its definition, undef it, and pop its previous definition.

Very true, sorry missed that when just trying to get the thing to compile.

ggael wrote:we are using the "placement-new" operator which calls the constructor of T on each element of the allocated data.

But that would kind invalidate the purpose of hooking new operator in the first place, since the allocations still wouldn't be tracked (unless the push/pop thing is done just around the replacement new). So I guess the best solution is just leave it as it is, and place the new macro trick as the last include. Not ideal but works, and not so big deal anyway.

ggael wrote:To workaround that, we could detect if new is defined as a macro, push its definition, undef it, and pop its previous definition.

I think that your approach for debugging memory allocations, based on redefining 'new' as a macro, is not the right one, because other solutions exist which give you the same advantages without changing your source code at all.

On Unix platforms there is:
http://valgrind.org/

I'm sure there is something like this on windows too. These tools can find any memory leak, double free, bad memory access, even "conditional jump based on uninitialized values"... and give you everytime a full backtrace into your program with line numbers, without having to modify or even recompile your program.

Gael: not sure about the idea of testing whether new is defined. i'd say a c++ program may safely assume that c++ keywords aren't redefined.

kajala wrote:But that would kind invalidate the purpose of hooking new operator in the first place, since the allocations still wouldn't be tracked (unless the push/pop thing is done just around the replacement new). So I guess the best solution is just leave it as it is, and place the new macro trick as the last include. Not ideal but works, and not so big deal anyway.

ggael wrote:To workaround that, we could detect if new is defined as a macro, push its definition, undef it, and pop its previous definition.

Sure there are alternatives, and generally speaking better avoid structures that can cause memory leaks in the first place, i.e. prefer RAII, smart pointers, etc., but sometimes that Visual Studio built-in memory debugging support is just handy, especially if you want to avoid dependencies to external libraries.

Tools like valgrind don't add any dependency on any library!
It's just a program that runs your program and analyzes it. Check if something like this exists for windows...

kajala wrote:Sure there are alternatives, and generally speaking better avoid structures that can cause memory leaks in the first place, i.e. prefer RAII, smart pointers, etc., but sometimes that Visual Studio built-in memory debugging support is just handy, especially if you want to avoid dependencies to external libraries.

Aa, ok. Might work well, I have to check. On games and real-time interactive 3D-graphics which I usually work on, though, you really want something with very low overhead that you can keep running all the time, and possibly control within the application as well. For example, you might want to take a snapshot just before loading a level within a game, and then compare before loading the next level that all memory was released from the last one.

Indeed, Valgrind has a very big overhead. Your program runs like 20x slower than normal. Still it's worth it when you see the amount of information that you get.

In a 3D application where a large part of the time is spent outside of the application process (I mean e.g. if you're doing opengl rendering), i have some hope that the overhead may be smaller.

Also notice that a tool like valgrind can see much more than you can ever hope to with a trick redefining new. valgrind catches all memory allocations that occur in your process, even those that come not directly from your code but from a library function that you call.