I want the menu bar to be hidden in all apps and to be able to access it by pressing "alt" like it is in VSCode by default.

Is it possible?

I'm not sure if there is a global setting for this, but in each app you can hit Ctrl+M to hide the Menu Bar

You can then go to System Settings -> Application Style -> Window Decoration -> Titlebar Buttons and drag the 'Application Menu' on to the Titlebar, then hit Apply.
While you can't just hit Alt to get the menu bar back, you can use standard shortcuts like Alt+F to bring up the File menu or Alt+V to bring up View. And from there use the mouse or keyboard arrows to navigate the menues.

EDIT: Actually, if you first drag the application menu from Window Decoration -> Titlebar Buttons, then the menu bar seems to be hidden by default. So no need to hit Ctrl+M in every app