I would like to be able to use "Outline Selection Tool" by holding down a modifier and drawing while in brush mode, and return to brush when releasing that modifier, just like I pan/zoom/rotate with modifiers + stylus.. I use that selection a lot and it's annoying having to switch back and forth between that and the brush all the time.

I was looking in the canvas input methods settings but it doesn't seem doable there.

Is there a way to do this?

No.

I see. Then I would like to suggest this feature. I believe it would be useful, but I have no idea how hard would it be to implement.

Not hard, as such, but a fair bit of work. The concept of temporarily switching to another tool on pressing a modifier is present -- it's how panning works -- but it would mean having different tools to switch to depending on different modifiers.