Hey all,
I've written some Eigen to NumPy converters using boost::python. They are not particularly efficient as they copy going in and out of Python but they are easy to use. I probably haven't done everything right but I'd be interested in feedback.

Any interest in this code?

What would be a good venue to start this as a project?

Thanks!

Paul

Why don't you just upload them and announce on the mailing list, you'll reach a lot of people there (as well as on this forum).

Extremely interested! I only transfer quite small arrays around, so copy isn't the end of the world. In my view this dual language approach is the way forward for scientific computing.

I am currently using this:

http://code.google.com/p/ndarray/

which also has Numpy->Eigen converters, but I'm not particularly interested in the ndarray part so it would be good to get rid of the cruft. How do you rate the approach taken in this library compared to yours?

Perhaps a good way forward would be to put your code in a mercurial repository on bitbucket (this is where Eigen is as well).

I'm interested as well.
I posted my own converter snippet for that before:
viewtopic.php?f=74&t=86057

but I suppose it's not working on 64bits, 1D array etc.

I would be interested in a better solution.

I'm definitely interested. Having to dynamically allocate a copy of each array and copy would be a showstopper for me, though. I'm trying to write Boost::Python wrappers manually around my functions, i.e. something like:

Code: Select all

#include <iostream>
#include <Eigen/Core>
#include <Eigen/Array>
using namespace Eigen;

class FooClass
{
public:
   int foo(const MatrixBase<Derived> &bar);
...
private:
   int m;
...
}

#if WRAP_PYTHON
int FooClass::foo_python( PyObject* bar){
   Map<MatrixBase<Derived>> _bar((double *) PyArray_DATA(bar),m);
        // I already know the array size from the Foo's constructor.
   return foo(_bar);
}
using namespace boost::python;
BOOST_PYTHON_MODULE(_FooClass)
{
    class_<FooClass>("FooClass", init<int>(args("m")))
        .def("foo", &FooClass::foo_python)
    ;
}
#endif


At the moment, I'm getting a compile error:

Code: Select all

error: ‘Derived’ was not declared in this scope


Anyone have any idea what else I need to include?

When I told you to pass a MatrixBase<Derived>, I meant Derived to be a 'typename' template parameter. Like this:

Code: Select all

template<typename Derived>
void foo(const MatrixBase<Derived>& m)
{
 ...
}


In Eigen code, 'Derived' only appears as a template parameter like that.

I got an example using Boost::Python working, and posted it up on:

http://eigen.tuxfamily.org/index.php?ti ... teropBoost

The method is error prone, ugly, and not very general, but it works.

Cheers,
Drew

Thanks very much for doing this!

drewm1980 wrote:I got an example using Boost::Python working, and posted it up on:

http://eigen.tuxfamily.org/index.php?ti ... teropBoost

The method is error prone, ugly, and not very general, but it works.

Cheers,
Drew

My code is failing if I try to do anything more interesting to the passed entries (like set them to zero).

In the following version:

http://paste.ubuntu.com/425545/

I get the compile error:

wrapper_example.cpp:50: error: no matching function for call to ‘Eigen::MatrixBase<Eigen::Map<Eigen::Matrix<double, 10000, 1, 2, 10000, 1>, 1> >::setZero(int&)’
/usr/local/include/eigen2/Eigen/src/Core/CwiseNullaryOp.h:400: note: candidates are: Derived& Eigen::MatrixBase<Derived>::setZero() [with Derived = Eigen::Map<Eigen::Matrix<double, 10000, 1, 2, 10000, 1>, 1>]

One suggestion I got was to switch from MatrixBase to DenseBase. If I do that, I get:
/home/awagner/optimization/other/python_wrapped_eigen/FooClass/wrapper_example.cpp:25: error: expected ‘,’ or ‘...’ before ‘<’ token
/home/awagner/optimization/other/python_wrapped_eigen/FooClass/wrapper_example.cpp:28: error: ISO C++ forbids declaration of ‘DenseBase’ with no type
/

Any ideas?

Stepping back...
I learned some Eigen basics and ported and tested a numerical optimization routine, all in a couple of days (yay!), but so far I have been utterly incapable of getting it to work with external code/data. I was able to get one version of the above example working, but I required expert help (thanks Benoit!) practically every step of the way, and it broke as soon as soon as I tried to do something non-trivial.

I need to have some idea if:

A: I am one or two cryptic bugs away from having working glue code and using Eigen happily ever after without understanding all of the template magic it is build on ~or~
B: A deep understanding of C++ templates and the Eigen internals is needed to be able to integrate Eigen code with other code.

Thanks!

You are passing a reference-to-const argument to your function:

Code: Select all

const MatrixBase<Derived>&


So, by definition of 'const' you're not allowing yourself to call non-const methods on that object. Like setZero(). You're basically promising not to modify this object. If you want to do that, remove the 'const'. See any good C++ tutorial on constness.

It's nothing deep about templates. It's just understanding what 'const' means.

bjacob wrote:You are passing a reference-to-const argument to your function:

Code: Select all

const MatrixBase<Derived>&


So, by definition of 'const' you're not allowing yourself to call non-const methods on that object. Like setZero(). You're basically promising not to modify this object. If you want to do that, remove the 'const'. See any good C++ tutorial on constness.

It's nothing deep about templates. It's just understanding what 'const' means.

I looked closely at that too, but the function parameter I'm operating on is not one of the ones that is declared const. (unless I'm ~really missing something about the semantics of const...)

Ok so:

Code: Select all

   barOut.setZero(m);  //  THIS IS WHAT IS FAILING


here's the documentation for MatrixBase:

http://eigen.tuxfamily.org/dox/classEig ... xBase.html

there is no setZero(int) method in class MatrixBase. The only setZero method in MatrixBase is taking void, i.e. it's not resizing.

All the resizing methods are in class Matrix, not in MatrixBase. This is because you can only resize a plain matrix, not an abstract expression like "sum of two matrices". In particular you can't resize a Map expression (Map just wraps an existing array, it doesn't allocate/resize/free it).

bjacob wrote:there is no setZero(int) method in class MatrixBase. The only setZero method in MatrixBase is taking void, i.e. it's not resizing.

Ok, makes sense now; thanks! I was following the tutorial and didn't know of this form of setZero. I made the bad assumption that you just always had to pass the dimensions, and it was Eigen's job to decide when re-allocation happened.

bjacob wrote:All the resizing methods are in class Matrix, not in MatrixBase. This is because you can only resize a plain matrix, not an abstract expression like "sum of two matrices".

This may not be obvious to people coming from libraries that support slicing and views into matrices(i.e. numpy). In numpy slicing out a chunk of an expression before assigning it to a variable is pretty natural. In an eigen context where you have delayed evaluation you might even be able to compute only the needed values, i.e. so a user can do something like:
y = (A*x+z+e+q)[0:10]
instead of
y = A[0:10]*x+z[0:10]+e[0:10]+q[0:10]
... or does .corner() already do something like this?

bjacob wrote: In particular you can't resize a Map expression (Map just wraps an existing array, it doesn't allocate/resize/free it).

Does setZero(int[,int]) ~always trigger dynamic memory allocation?

I actually have several questions about Memory allocation and Eigen; I'll collect them in another thread.

Thanks!

drewm1980 wrote:

bjacob wrote:there is no setZero(int) method in class MatrixBase. The only setZero method in MatrixBase is taking void, i.e. it's not resizing.

Ok, makes sense now; thanks! I was following the tutorial and didn't know of this form of setZero. I made the bad assumption that you just always had to pass the dimensions, and it was Eigen's job to decide when re-allocation happened.

Yeah so no, on general expressions you call the methods that dont take size parameters, they adapt to the existing size.

bjacob wrote:All the resizing methods are in class Matrix, not in MatrixBase. This is because you can only resize a plain matrix, not an abstract expression like "sum of two matrices".

This may not be obvious to people coming from libraries that support slicing and views into matrices(i.e. numpy). In numpy slicing out a chunk of an expression before assigning it to a variable is pretty natural. In an eigen context where you have delayed evaluation you might even be able to compute only the needed values, i.e. so a user can do something like:
y = (A*x+z+e+q)[0:10]
instead of
y = A[0:10]*x+z[0:10]+e[0:10]+q[0:10]
... or does .corner() already do something like this?

All the block() / corner() / segment() etc... methods allow to do that. Yes, thanks to lazy evaluation, this means that only the required parts of the matrices at hand, are accessed.

bjacob wrote: In particular you can't resize a Map expression (Map just wraps an existing array, it doesn't allocate/resize/free it).

Does setZero(int[,int]) ~always trigger dynamic memory allocation?

Only if the actual size changed. For example if matrix has size 3x4 and you resize to size 6x2, no memory allocation is performed (12=12).