Hi VDG,

I'm currently implementing the pointer constraints protocol for Wayland and I think we need some UI for it.

To explain: pointer constraints allow to constrain the mouse pointer. E.g. a window can say "please keep the cursor in this area". If the compositor honors this the cursor cannot leave the area. Use case are things like virtual machines and games.

This renders a risk: the user might not know how to free the cursor again. Thus I think it needs user interaction.

• Well known way to free cursor (e.g. right ctrl key)
• User needs to be informed about the well known way
• User needs to be asked whether the pointer may be constraints

Please think about the situation and how we can show it to the user and how we can make the interaction.

One thing that should be considered is the common ways to deal with it as used by games. I mean with borth Virtual Machines and most games we know there is a keyboard present. So in gaming "tabbing out" is not an uncommon phrase. Simply using the shortcut to swap windows (Alt+Tab), or displaying the desktop, and thereby free the cursor from constraints (like in CSGO for those who play it.

The risk with somehow enforcing a popup window that informs the user of this is that it may break the application he/she is trying to use. For example, a popup saying "You can use Alt-tab to get out of the game and get your cursor back" may very well become locked on the screen (unless the user alt-tabs to that window, close it and then alt-tab back). Although... on the other hand with a small check box that says "never display this again" that issue would be quickly resolved and then since we KNOW that the user will at that time have alt-tabbed away from the window we also know he/she masters the method.

So that is my suggestion:
Use Alt-Tab to free or Lock in the mouse and have the mouse locked in when the application is focused and dictate it (many games for example allow for the mouse to be freed when they are in menu's (like Minecraft)).
Display a "First Run" popup saying "The Cursor is now controlled by this application and will not leave the application window. To get out of that please Alt-Tab out out of the application to regain cursor control" and then a check box with the text "Never Show this Warning again".

mgraesslin wrote:Well known way to free cursor (e.g. right ctrl key)

I would suggest to use the same key combination(s) as windows, too. Since most games are optimized for windows, that keyboard short cut shouldn't collide with in game short cuts.

As far as I recall:
There are several shortcuts in windows that free the cursor, eg meta key, ctrl + esc, alt + tab. At least meta can be overridden/blocked by games, maybe others too?

jensreuterberg wrote:One thing that should be considered is the common ways to deal with it as used by games. I mean with borth Virtual Machines and most games we know there is a keyboard present. So in gaming "tabbing out" is not an uncommon phrase. Simply using the shortcut to swap windows (Alt+Tab), or displaying the desktop, and thereby free the cursor from constraints (like in CSGO for those who play it.

No, global shortcuts may not necessarily break the pointer grab. If I'm only using one application at a given time Alt+Tab is not functional. We cannot depend on such functionality.

jensreuterberg wrote:The risk with somehow enforcing a popup window that informs the user of this is that it may break the application he/she is trying to use. For example, a popup saying "You can use Alt-tab to get out of the game and get your cursor back" may very well become locked on the screen (unless the user alt-tabs to that window, close it and then alt-tab back).

Luckily there is no risk here. The protocol in question is well enough designed and doesn't guarantee the application that it can lock the pointer.

jensreuterberg wrote:So that is my suggestion:
Use Alt-Tab to free or Lock in the mouse and have the mouse locked in when the application is focused and dictate it (many games for example allow for the mouse to be freed when they are in menu's (like Minecraft)).
Display a "First Run" popup saying "The Cursor is now controlled by this application and will not leave the application window. To get out of that please Alt-Tab out out of the application to regain cursor control" and then a check box with the text "Never Show this Warning again".

What I don't like about that and especially about the "Never show this warning again" is that it doesn't tell the user when a window locked the pointer. It results in the game "let's try to press random shortcuts" in order to get out of the application (please don't think in games here, the first application to use that will be KWin itself).

We also need to consider that the application also grabbed keyboard. E.g. with a virtual machine both pointer and keyboard might be grabbed and alt+tab should go to the nested Windows instead of breaking the pointer grab.

fabianr wrote:

mgraesslin wrote:Well known way to free cursor (e.g. right ctrl key)

I would suggest to use the same key combination(s) as windows, too. Since most games are optimized for windows, that keyboard short cut shouldn't collide with in game short cuts.

As far as I recall:
There are several shortcuts in windows that free the cursor, eg meta key, ctrl + esc, alt + tab. At least meta can be overridden/blocked by games, maybe others too?

I do not know which keys are used by windows. But see my comment above: we cannot use things like Alt+Tab or Ctrl+Esc or even meta as those need to go to e.g. an virtual machine. The fact that meta currently always opens the launcher and is not ignored in case of a VM is a common complaint against 5.8. So no, all that cannot work.