Hello,

is there an efficient way to calculate Variance of the data composed in single Matrix?
in Matlab there is quire convenient method for that:

Code: Select all

// Compute the variance of A along the first dimension.
var(A,0,1)
// Compute the variance of A along the second dimension.
var(A,0,2)


i wonder what would be the fastest way doing it in Eigen
Thanx

What about (not tested):

(A.rowwise()-A.colwise().mean()).colwise().squaredNorm()/A.cols()

swap row/col to compute it per row.

Thanx for the tip.
Yeah it looks like a straight forward approach but not very efficient because you can actually compute the result in a single loop per row/col:

Code: Select all

sum = 0, sum2 = 0;
for(v in row)
{
    sum += v;
    sum2 += v*v;
}
mean = sum / num_values;
variance = sum2 / num_values - mean * mean;
stdv = std::sqrt(variance);


Yes I known, but be extremely careful with such an approach because it is subject to severe loss of accuracy unless your data is already nearly centred. In some cases you can significantly reduce the accuracy loss by accumulating (v_i-v_0)^2 and removing n*v_0^2-2*v_0*sum after the accumulation. This implies that v_0 is not an outlier!

I see, great thanx for another tip!

FYI:
There is a method of computing both mean and variance in one loop which is less sensitive to numerical rounding:
http://www.johndcook.com/blog/standard_deviation/
For reference, it quotes Donald Knuth’s Art of Computer Programming, Vol 2, page 232, 3rd edition.
But implementing that outside of Eigen might easily be slower than the two iterations suggested above.