I usually browse around in my folders, switching between songs. But when I open a song, it just gets added to the playlist rather than starts playing.
Is there any way to change this behaviour?

Also, is there any way to browse the folder structure, within Amarok? Can't find such an option.

Thanks

the Files browser, lowest tab on the left, will allow you to browse directory structure. Have a look through the rest of the forum for options for immediate playback.