What if?
A study of what-if scenarios which posit what KDE would look like if it took a different approach to certian aspects of design.

This thread is dedicated to KDE designs and mockups which answer what-if questions such as;
"How would a Windows 10 style start menu look/work in plasma?"
"What would Elementary/Android/Gnome/iOS components look like as KDE components? How would they work?"
"What would a bare-minimum default layout look like?"
"How would KDE client-side-decorations look? How might they work?"
"What would KDE look like if the user only had 5 reliable buttons? (like a remote); What can we learn from Smart TVs?"
"What would a voice-assistant look like? How would it integrate? How would it affect accessibility?"
"What if projects like KDE connect were taken to the extreme?"

These questions are designed to look at how others' have designed their user interfaces, how we might apply their philosophies to KDE, and what it might look like. Even if a question doesn't directly impact KDE, I still feel we have massive potential to learn from others', or at least cross-pollinate good ideas. This is especially important as KDEs' converged interfaces increasingly begin to operate in exotic non-desktop enviornments; Televisions, tablets, and phones are all known targets while technologies like vertical wall displays, in-car infotainment systems, computerized tables, and even digital 'paper' displays are inching closer to consumer reality. This thread will hopefully offer a basis for future design decisions, and offer what did or didn't work.

I'll be posting designs over the next week or two in this thread; for each question I'll create a design or two, each design will be followed by an explanation, technical challenges, major pros, cons, comparisons, and final notes with particular interest in the mantra of "simple by default, powerful when needed". I'm hoping this thread will operate more like a study, where not only the designs are analysed, but where the ramafications, issues, and alternatives are discussed and expanded on. The important thing isn't minor details, but broad concepts. Another important factor is realistic thinking; designs in this thread should be grounded (somewhat) in practical reality.

I don't know if I'll be getting to all of the questions above, and even still I might do entirely different questions. I'd also like more questions to be posted, especially if they would require radical designs - or are simply unanswered. I also highly encourage others to create their own designs, answer questions, and perform UI studies in this thread. My own mockups will be designed to look like screenshots - but this is not the requirement, I simply have some extra time at the moment to do so. If you do post designs, please label them as design mockups accordingly.

(quote this for formatting on any what-if answers)
[size=150]What if... [color=#f67400][i]KDE ______ ___ ______?[/i][/color][/size]
[color=#777]Forward[/color]
...
*any designs*
[color=#777]Technical Challenges[/color]
...
[color=#777]Pros[/color]
[list]
[*]...[/list]
[color=#777]Cons[/color]
[list]
[*]...[/list]
[color=#777]Notes[/color]
...

What if... KDE Started using Client-Side Decorations?
Forward
"Client-Side Decorations" (or "CSDs" for short) are a method of decorating windows where the application is responsible for the window frame. Commonly, Linux and Linux desktop enviornments have favored "Server Side" decorations (SSDs for short), where it was up to the window manager to draw the window frames. In this "What if" scenario we'll explore two hypothetical applications; a media player and a PDF viewer, programs which might benefit from CSD technology. Before the CSD-enabled mockup, here's what the media player application might look like using KDEs' current SSD method:

Click to Enlarge

And here's what it might look like using CSD:

Click to Enlarge

In the CSD design, the various window buttons are moved down into the control bar of the media player. This saves vertical space, meaning an un-maximised video will always be slightly larger. In addition, the buttons can follow a separate theme from the main window; in this case a less distracting button is used. Additionally, the application could partially fade buttons in the bottom bar, or, if the bar was laid over the video the entire bar could be faded out (when the mouse leaves) for maximum display - though this is not in the mockup.


Click to Enlarge

In this PDF reader, the CSD is on the left, and uses the thumbnail column as the decoration. As PDFs generally tend to be a series of portrait sheets, vertical space is at a premium.

Technical Challenges
The main techncial challenges faced by KDE developers is ensuring application consistency will continue to work under various form-factors. When Kwin (the KDE window manager) controls window borders, it can quickly and gracefully adapt to multiple form-factors. For example, in Plasma active space is at such a premium KDE can hide window decorations and embed them into the workspace itself.

The other technical challenge is protocol and cross-enviornment consistency. It's known that CSD-enabled applications can look extremly awkward when window borders are wrapped around an application not designed to use them. In addition, protocols for drawing CSDs on Linux are a mish-mash at best, and CSD code tends to be far less portable to other desktop enviornments. Compounding that, KDE has additional features (such as window tabbing) which are inherently incompatible with the feature.

If KDE were to use CSDs, it would likely be preferable for all applications to support CSDs as a secondary 'upgrade' and default to using SSDs until the window manager explicitly enables the CSD; meaning KDE apps with CSDs would need alternate layout code. KDE might also need to reserve the ability to revoke CSDs from an application or flag them to keep CSDs disabled; for example, it might only allow Wayland-composited enviornments to use CSDs, as Wayland has protocols for CSD-specific problems (such as application hangups). It might revoke the CSD layout if a user disabled it, or if an application is moved into a window-tab.

Application crashes are another significant issue; If applications are made responsible for their own close buttons you enter a situation where the close button on a crashing program will fail to function. There are several technical aspects to this which can defer the impact, but in a worst-case scenario the GUI of an application will become an nonresponsive picture. Wayland has solutions to detect this, while Xorg is a hot mess; if Wayland/KDE used CSDs, a window manager overlay might need to be created in the event of a crash to retake the minimal window-level control of applications.


Click to Enlarge

Pros

• Applications would waste less space on screen; for small laptops this is a significant advantage.
• The decoration can be placed on any side. Applications like a movie player could hide decorations completly
• Applications with a full-screen mode may just use the same chrome in full-screen, maintaining placement of close/minimise/maximise buttons.
• A strong CSD protocol/library could ensure reliability

Cons

• Outside of the additional code overhead, applications could more easily break in the future as technology changes.
• Depending on the implementation, applications could be larger
• Kwin would need additional code to recover, mark, or kill crashed CSD applications.
• Consistency is lost; Windows might place buttons on the top, bottom, left, or right. Users would need to adjust to applications individually.
• Poorly chosen widgets can interfere with window dragging, space meant for dragging windows may be lost
• Depending on implementation, polling mechanisms might falsely label applications as nonresponsive

Other Points

• Applications can crash many ways, causing whole-system breaks. SSDs can't police the whole system; Bad applications shouldn't be the example.
• Using an "SSD until otherwise permitted" model can still allow users who dislike CSDs to enforce SSDs.
• KDE applications are beginning to ship with multiple layouts for convergence purposes; CSD layouts might piggyback on this.

Notes
On a personal level, I love applications which properly use CSDs. They're more screen-efficent and applications can look much more professional. I also beleive that CSDs will be inevetable, and that a strong KDE CSD library which enforces smart use of CSDs would be better now than applications inventing their own solutions later; especially when other enviornments are starting to make good use of it. After looking at Gnomes solution, I do think CSDs as-an-upgrade would be ideal, as applications can behave "normally" until they get the go-ahead to be shiny. This will help KDE apps fit into enviornments like Windows, OSX, or Gnome without issues. Also, users might want to forcefully disable CSDs desktop-wide for consistency purposes, and having CSDs as-an-upgrade would assist in this.

I also think that CSDs should also be required to adhere to Kwin specifications will first and foremost; while it's nice applications might 'brand' themselves using colours or unique styling, I beleive allowing Kwin to dictate the style would be a usability win. In addition, KDE 5 is incredibly friendly to dark themes (or is incresingly getting better at it), and allowing CSDs to colour themselves would take away from user options. On the other hand, since windows would not necessarily display titles, colouring becomes an important identifier.

I also like the idea that a full-screen application could just continue recycling the same piece of chrome; often applicaitons use entirely different UIs in fullscreen mode, which reduces usability - especially when hotkeys are required to exit fullscreen.

Technology-wise, I've read impressive articles on Wayland in regards to how they've thought through many CSD-specific problems - such as application crashes rendering windows umovable. No matter how its sliced though, CSDs should be approached carefully and thoughtfully if they were to be implemented as a part of KDE in the future.

In terms of the "simple by default, powerful when needed" mantra, CSD decorations follow the pros and cons listed above; mainly by making more efficent use of screen-space and giving applications a more simple, less intimidating demeanor. On the other hand, it decreses the intuitivity of applications by potentially moving window controls, or causing issues in event of a hangup. I think CSD driven designs tend to be more simple and reserved (at least as Gnome has done it). Disabling the CSD desktop-wide for features like window tabbing is where "powerful when needed" kicks in; especially since its an uncommon and difficult to discover feature.


Chime in!
What are your thoughts on CSD; what did I miss, any addendums? Let us know!
The next What-If I'll be addressing is: "What would 'Start' Menus and Launchers from other enviornments look like in KDE?"

Great initiative, Kver!
Your approach to think the pros and cons of in this case CSDs through and try to address the issues raised before is definitely commendable!
I like the idea of "CSD as an upgrade", as it prevents user interface designers and developers from neglecting how their application looks and works with SSD. The added work of having to design two versions of an application is of course a downside, but given KDE's cross-device-strategy, applications should be designed and implemented with different UI variants for different device classes anyway, so the overhead of having a CSD and SSD variant should not be too big.
Of course using CSDs would still be completely optional.

From the usability perspective, the decreased consistency is of course the biggest concern. If we embraced this approach, we would have to make sure that the Human Interface Guidelines are applied in all UIs using CSDs, so that users always have a chance to find the basic window controls in all of them.

From the technical part, woud it be possible to have the up, down, close icon in an icon bar together with all other icons. so that you have the same functionality as from the window decoration icons in the icon bar. whan this would work, you can shrink the window decoration to a nice shadow and have the icons in the application. And the benefit would also be that the user can arange the icons as he like AND can remove icons.

Kver wrote:What are your thoughts on CSD; what did I miss, any addendums?

Pro: Flexibility, Appearance
Con: Relaying on the icon (and color) only might be problematic. Color shouldn't be used as primary indicator because it doesn't pop out in case of colorful screens (like the media player) or attracts too much attention in case of plain screens. Consider color impaired people as well. The red cross icon is very similar to actions:list-remove. Furthermore, I don't know where to grab the window to move/maximize it (left click on media player pauses, meta key is far away). Placing destructive functions at the lower right border makes the application behave like a secondary dialog. The application style (overlay vs. form) is only conveyed by the shadow. Whitespace is not your enemy.

Too many cons. If I think about the pros I'd certainly find as much. But you did that already.

Edit: +1 Con: Without the headline/titlebar I don't for sure what application I'm using and where I am.
Edit: +1 Con: The integration of non-KF5 programs will be broken probably.

One issue that i had with the Gnome CSDs was with their file manager.

The window title bar that you'd normally drag to move the window, was replaced with clickable elements with their new CSDs. One of these was the 'Breadcrumb' bar to show where you'd navigated to, it often stretched right across the top of the window - exactly where you wanted to click and drag to move the window around. You'd either end up accidentally clicking on the breadcrumb when you wanted to move the window or you'd have to be really accurate about where you were clicking in the titlebar so that you didn't hit the breadcrumbs by mistake.

A suggestion - in Firefox, if you go into fullscreen mode then the whole top bar (window decorations, tab bar, toolbar) autohides and the contents of the browser is displayed absolutely fullscreen. Some of the elements of the top bars are revealed by moving the mouse to the top of the screen including all the normal window controls BUT they don't look like our normal window controls so I assume that they're drawn by Firefox itself.

Could we not just improve fullscreen mode in those applications where making the best use of screen space is useful & make all the fullscreen modes of these applications really consistent? Oh and a convenient way of cycling between maximised windows would be a good idea too (think 'alt-tab' but where the switch was made instantly by clicking an on screen button - no OSD & no keyboard shortcuts).

That way for those that have got a big monitor things would stay as they are, if you were happy without fullscreen mode then you wouldn't bother and your windows would be just like before - for those using small screens they can just go all fullscreen and they'll have a really efficient use of their screen-space. One possible advantage is that our applications may continue to be usable in other DE's etc and we wouldn't end up in the situation where someone using desktop X can't use our applications because they'll only ever display properly in KDE.

I have no idea at all what the technical implications of this are but it seems like a fair compromise to me & fits in with the idea that KDE fits in with what you want - you only have things running fullscreen if you want to.

A Form Factor modul in the system settings would be nice. there you can choose if you work normally in fullscreen mode, you want to show only icons without text, ...

but that's the result first we should dream how apps should look like with different form factors.

First of all, the mockups are great and you made a great effort sharing your ideas, kudos!
I am however not so fond of CSD, even if they really look nice in your mockups and I can understand the arguments for CSD.
However Martin Gräßlin, our great kwin maintainer has repeatedly pointed out why CSD's are not really the best solution from a technical point of view (see for example http://blog.martin-graesslin.com/blog/2 ... corations/ and http://blog.martin-graesslin.com/blog/2 ... corations/).
My argument against CSD is simply the argument of consistency. I hate inconsistent behavior. Without client side decorations all applications are consistent in the regard of window managing, not matter if it is some proprietary software or simply software written using an uncommon toolkit.
Kwin allows for advanced window managing, you can add a lot of buttons to your window decoration etc, but you can also hide the window decoration if it gets in your way (right click -> more actions -> no border). Currently kwin and its window managing is a really stable, technically advanced as well as a consistent experience, no matter how much effort you make, you will never get that back with CSD.

http://blog.martin-graesslin.com/blog/2 ... corations/
http://blog.martin-graesslin.com/blog/2 ... corations/
http://blog.martin-graesslin.com/blog/2 ... corations/
http://blog.martin-graesslin.com/blog/2 ... d-wayland/
http://blog.martin-graesslin.com/blog/2 ... corations/

https://bugzilla.gnome.org/show_bug.cgi?id=729721
resulting in:
https://bugzilla.gnome.org/show_bug.cgi?id=729767
https://bugzilla.gnome.org/show_bug.cgi?id=729769
https://bugzilla.gnome.org/show_bug.cgi?id=729772
https://bugzilla.gnome.org/show_bug.cgi?id=729773
https://bugzilla.gnome.org/show_bug.cgi?id=729777
https://bugzilla.gnome.org/show_bug.cgi?id=729780
https://bugzilla.gnome.org/show_bug.cgi?id=729781
https://bugzilla.gnome.org/show_bug.cgi?id=729782
https://bugzilla.gnome.org/show_bug.cgi?id=729783
https://bugzilla.gnome.org/show_bug.cgi?id=729784
https://bugzilla.gnome.org/show_bug.cgi?id=729786
https://bugzilla.gnome.org/show_bug.cgi?id=729788
https://bugzilla.gnome.org/show_bug.cgi?id=729792
https://bugzilla.gnome.org/show_bug.cgi?id=729793


Long story short:
-----------------------
leaving aside the inconsistency threat ("where and what is the close button?") and the rather hackish gtk+ approach on the topic, client side decorations cause major technical issues because a frozen client can no longer be close, minimized or even moved (resized).
This especially threatens "the unskilled" (long time KDE users can deactivate all decorations and control KWin "blindly" by shortcuts and the mod key - and kick a really nasty client from VT1)

Personally, I would completely NOT mind alternative approaches to the titlebar (eg. 80% of the time the titlestring is useless), but the GUI to control the window position/size/visibility MUST remain under control of the WM process.

Esp. in a composited environment there's hardly anything we could not do - clients could hint to prefer vertical titlebars (or the WM/deco determine that bases upon the client geometry) and it would also be possible to hide tiny title"bars" until the mod key is pressed (hard to detect, though) or the window is hovered (like plasma extenders - maybe a little nasty)

luebking wrote:Long story short:
-----------------------
leaving aside the inconsistency threat ("where and what is the close button?") and the rather hackish gtk+ approach on the topic, client side decorations cause major technical issues because a frozen client can no longer be close, minimized or even moved (resized).
This especially threatens "the unskilled" (long time KDE users can deactivate all decorations and control KWin "blindly" by shortcuts and the mod key - and kick a really nasty client from VT1)

That's why KVer suggested to not use CSD by default under X11, but only under Wayland where according to his research these cases are taken care of better than under X11. Is that true, or do the problems with frozen clients persist under KWin-Wayland?

colomar wrote:under Wayland where according to his research these cases are taken care of better than under X11

Not to my knowledge - the challenges remain the same

a) a stalled/hunk/locked client can simply not induce movement/minimization/closing etc. - it's locked in some function and does not process events.
b) detect that another process is stalled and offer to kill it

(a) doesn't change depending on the display server - a locked process simply cannot perform any event processing. Nowhere. Never.

For (b) the NETWM protocol (on X11) offers the "_NET_WM_PING" message - WMs ping the client and if the client doesn't respond "in time", the client is likely locked and we offer to kill it.
KWin pings the client if you attempt to close it - if it doesn't respond, it's likely locked and we offer to close it "the hard way™"
This does naturally not work w/ CSD, since KWin never knew that you attempted to close it (using the close button)
Compiz (and afaiu Mutter) additionally ping the client when it's activated (so locks of an already active client would go unnoticed).

The gtk+ suggestion was to simply ping it over and over again, what can of course easily lead to false positives (when the client is in some expensive function - which oc. was better done in an extra thread, but this isn't a perfect world) and causes a lot of communication overhead.

Wayland of course supports pinging as well, but that doesn't change the basic issues - I'd certainly like to see sources on "better" CSD handling on Wayland.

The only way it could somehow work was if the client provided detailed information on where the WM should handle what inpt event, ie. geometries for close/minimize/move/resize areas.
The WM (being the natural input redirector on Wayland, but also can intercept events on X11) could then perform the actual event processing while the client would only do some visual feedback ("pressing" the button) - latter would be lost on stalled clients, but they could be handles as with SSD.

Notice that even solving this issue would only get us into the inconsistency "trap", where clients would not only decide how a button looks and where you can find it (you may know this from windows... and you can usually forget about custom titlebutton layout then), but also esp. WHAT buttons are provided itfp, ie. simply denying minimization, maximization or closing (and while no minimizing/maximizing are "just" nasty: if they - like GNOME dialogs - don't allow to "close" them because you shall do that by pressing either "ok" or "cancel", you are immediately back in the "can't close hung client" situation)

luebking's explanation really sounds reasonable and pinging the application all the time is stupid imho. Apart from that I do not see why an application should kind of emulate a window manager itself, I find that idea pretty strange. An application should focus on what it is supposed to do and not managing itself in the environment it is run.
At some point applications have to assume something underneath them and giving full control to applications does simply not sound reasonable, neither from a technical perspective nor from a security perspective.

kdeuserk wrote:luebking's explanation really sounds reasonable and pinging the application all the time is stupid imho. Apart from that I do not see why an application should kind of emulate a window manager itself, I find that idea pretty strange. An application should focus on what it is supposed to do and not managing itself in the environment it is run.
At some point applications have to assume something underneath them and giving full control to applications does simply not sound reasonable, neither from a technical perspective nor from a security perspective.

From what I understand about Wayland and CSD pinging, Wayland will check to see if a program is responsive every time you attempt to interact with it, and then offer a failure fallback if the application fails to respond. By the sounds of it, (and this is where my brain slipped out of my ear while reading documents) every interaction with a wayland surface will illicit a response back to the server. Either way, the WM needs to monitor responsivness anyway. Even if a program does crash, its only the application-level signals which are lost: maximise and minimise are still capable of functioning.

So, for example, if the media application fails to respond, the window manager may take control and "layer over" a nonresponse screen complete with whole-surface dragging and a close button. This might even be a good behaviour if we don't use CSDs', because people may still attempt to use menus other than the window managers' close button - and having a whole window dim with a big red close button and a "this application is not responding" might be preferable to altert people that something is off.

EDIT: The "crashed window" image has been moved to the main what-if post.

In my opinion though, designing around failure cases is the worst possible design method possible; When applications crash all sorts of things can go wrong; CPUs can be run up, RAM can be eaten alive, memory leaked - nothing is really safe. Xorg itself has a runaway memory issue on one of my computers, I need to restart it daily or face a desktop which can't even move the mouse properly. Designing around failure creates an artificial limit on things and declaires one component "sacred". While I think its good to account for failure, the designer in me seeths when the UI of a system is dictated by academic problems; but the dev in me also really likes the firm stance Martin took to ensure things are less prone to failure.

Having applications manage themselves is odd; but I think that falls back to building a well-designed library, and ensuring there are very specific use-cases and conditions to minimise the impact. One thing I had written (and removed) from the origional post was the idea of making CSDs exclusive to QML-based applications, as QML is capable of abstracting a lot of issues, meaning a single cohesive library could step in to provide all the functionality. And in 5 years, when Wayland is the beating heart of most Linux desktops, I think the majority of these issues will be solved.

Either way, some of the points in this thread are going to be circled back into the main what-if post - very very good discussion. Later this afternoon I'll be posting the next round of designs; I've got a nice Windows 10 launcher, a nice Elementary launcher... Just a few more to go...

I hate to meta-comment but, thanks for sharing these ideas Kver! No matter what happens in the end for each of these What Ifs, I personally think it's freaking fantastic that you and everyone else here are willing to put some ideas out there and talk about and attempt to navigate potential solutions to any issues. We may not solve world hunger but, so long as we stay constructive and continue to assume the best of everyone's participaion, the absolute worst that could happen is that we acquire a much better understanding of why we can or cannot do something. I know I'm learning a LOT! Of course the best that could happen is that we uncover something totally fresh and exciting. Talk about a WIN-WIN!