I just configured all I had to configure to make Krita work for math, and I'd like to save every preset, every brush, and every configuration, because I might lost it or update to a newer version that does not have it. How do I save everything?

Basically, backup the Krita folder in %appdata% or .kde.

You can find the resource folder easily via the resource manager as well, but the config file is one level higher.