I am using Eigen to solve various least squares problems and it is working quite well. The overdetermined linear systems are relatively simple with the form:

Ax= b

These can be easily solved using LLT (which is blazing fast) by setting the equation up as A'Ax= A'b. The solution x will minimize the L2 norm for the system,

This is perhaps a more mathematics question rather than strictly Eigen but was hoping someone can lead me to the right direction. Is there an analytic solution if I wanted to minimize the L4 or L6 norm. If so how would my normal equations look for L4 and L6 norm minimization respectively. I asked this question on Mathematics@StackExchange as well. Thanks for your help.

http://math.stackexchange.com/questions/300644/minimizing-l4-l6-l2n-norm-for-linear-regression

There is no direct solution for L_p minimizations. If p<1 then the Iterative Reweighted Least Square method is pretty simple to implement. For p a multiple of 2, you might re-cast your problem to a non linear LS one:

argmin_x = sum_i (f_i(x))^p

becomes:

argmin_x = sum_i (f_i(x)^(p/2))^2

which can be solved with our Levenberg-Marquart solver available in unsupported/.

Thanks. I was thinking that for multiples of 2 there could be analytic solution, but I'll use the LM solver. Best Regards.