each time I restart kate it re-identifies (incorrectly) that I want Matlab highlighting. For each file I have to switch it back to Octave. I tried deleting the Matlab highlighter, but it lives.

How can I do this, or is there a better way?

Looks like octave and matlab use *.m and *.M patterns, octave also uses *.octave.
Foo.octave will likely use the octave highlighter, while foo.m the matlab one (for alphabetic precedence)?