Hi all,

is there a performant way in Eigen to perform the following Matlab operation:

Code: Select all

a = magic(8); % Just creating a matrix
a(a>40) = median(a(a>40)); % The interesting operation

What the command does is: Looking for matrix entries that satisfy a certain condition (>40 in this case) and replacing all those entries by their median value leaving the other matrix entries untouched.

There will be a chain of operations (e.g., >10, >20, >30, ...) performed. Unlike this example, the sets that fulfil the conditions are usually disjoint and every matrix element belongs to one of the sets.

How are manipulations like this done best in Eigen?

Best regards
vernal

Didn't I make myself clear?
Is it possible to do this line

Code: Select all

a(a>40) = median(a(a>40)); % The interesting operation

efficiently with Eigen?

In Eigen we don't have any median operator, and can only do things like:

Array a;
a = (a>40).select(value,a);

to mimic a(a>40) = ...;

Regarding your specific use case, I would recommend you to sort all your entries into a vector of pair of value,index (e.g., std::pair<scalar,int>), where the index is the index of the respective coefficient in the original vector a. Then your chain of operation is a O(n) process!

Thank you! I didn't look here for a while. I have to try the select feature of Eigen. Sounds like what I need here.

That's an interesting idea. If I understand correctly you mean sorting with bucketsort in O(n). And then perform an operation (e. g., Median) on the buckets. Is that correct?

It's only now that I come to test the suggestion.

The expression in your example works but if I'm doing it a bit differently I can't assign the result of (myEigenArray < somevalue) to another array to save that "binary mask" for later re-use. I read in the docs that the return type of the expression is a CwiseBinaryOp but I know too little of the inner Eigen workings to figure out how I can save that into an array.

Cheers
vernal

you can store it in a boolean matrix:

Matrix<bool,Dynamic,Dynamic> foo = a>40;

or convert it to int

MatrixXi foo = (a>40).cast<int>()

You could also store it as an expression which would simply be an alias for (a>40), but then writing the type is quite painful unless you are ok to use a C++1x feature:

auto foo = a>40;

The auto keyword is supported by recent versions of GCC and MSVC.

ggael wrote:

vernal wrote:if I'm doing it a bit differently I can't assign the result of (myEigenArray < somevalue) to another array to save that "binary mask" for later re-use. I read in the docs that the return type of the expression is a CwiseBinaryOp but I know too little of the inner Eigen workings to figure out how I can save that into an array.

you can store it in a boolean matrix:
Matrix<bool,Dynamic,Dynamic> foo = a>40;

or convert it to int

MatrixXi foo = (a>40).cast<int>()

How would you use that matrix to index the original matrix afterward? Can you do something like this?

Code: Select all

a(foo) = ...

How would you use that matrix to index the original matrix afterward? Can you do something like this?

Code: Select all

a(foo) = ...

Hi, are there any changes w.r.t. this feature? Right now the above seems to be failing (using Eigen 3.0.0), but perhaps I'm doing it wrong.

I'm currently trying to "clean" some data, where by "cleaning" I mean keeping selected rows based on a defined criterion (storing the columns of a big "dirty" ArrayXXd into a bunch of clean VectorXd's).

Here's how I currently do it (just a workaround):

Code: Select all

Eigen::VectorXd A, B, c;
//...
Eigen::ArrayXXd Data(ROWS, COLS);
// ... read into Data

// data cleaning
auto ratio = Data.col(0)/Data.col(1);
auto keep_row = (Data.col(2) > 0.5)
              * (ratio > 0.01) // * on bool is simulating AND
              * (ratio < 0.99); // filename.cpp(123), relevant below
auto keep_row_count = keep_row.count();
A.resize(keep_row_count);
B.resize(keep_row_count);
C.resize(keep_row_count);
size_t row = 0;
for (size_t i = 0; i < keep_row.rows(); ++i)
{
  if ( keep_row(i) )
  {
    A(row) = Data.col(0)(i);
    B(row) = Data.col(1)(i);
    C(row) = Data.col(2)(i);
    ++row;
  }
}


I would love to get this shorter and more optimized, if possible. I'd rather do it this way (much more elegant and concise than manual resizing with a hand-crafted for-loop):

Code: Select all

A = Data.col(0)(keep_row);
B = Data.col(1)(keep_row);
C = Data.col(2)(keep_row);


However, then compilation fails:


In addition, when applying the above-mentioned workaround, compiler treats me with the following performance warning (pointing exactly at the line where I "multiply" the boolean subexpressions in order to simulate the AND operator -- that is the location marked below as "***filename.cpp(123)***"):


So, even if the current workaround is the way to do this, I'm worried about the long warning (and performance).

Any advice/suggestions/comments/help?

Hi, nope this feature has not been implemented yet. Regarding, the use of operator* on bool, we should implement proper boolean operators && and ||. This later is very easy to do and patch welcome.

If you don't want us to forget about these features, you can fill bug reports in our bugzilla: http://eigen.tuxfamily.org/bz/

OK, just need a classification clarification:
1. &&, || bool operators -- does this belong in core-expression templates or core-general?
2. Element selection based on bool expressions -- core-expression templates?

Don't worry, that's not very important.I saw you already filled an entry for, 1, and for 2 yes core-expression sounds fine.

Hi!

I just stumbled upon this thread.

Is the use of bool for indexing like described in the above already possible?
Like mattd is saying, it would give much more elegant lines for the code that I am writing!

Cheers