colomar wrote:Either there is a misunderstanding or you lost me here

He was quoting the wrong passage of your reply. He was replying to your idea of visually grouping streams under devices.

We had agreed earlier already that we can drop labels use the speaker/microphone icon instead. I did not take the time though to publish the new version of the mockup done using the mockup toolkit, where I planned to put in all the feedback.

There are a plenty of possibilities to groups stuff (after devices, input/output, individual streams, among others). While the grouping hierarchy after devices makes sense logically for simple uses I see a lot of problems with that approach for more complicated use cases: Just think of a stream being assigned to different devices and all the duplicated entries (After all in most of the cases you have more streams than devices, so listing devices when pressing an individual streams seems to be far better from an orgaizational pov.). The user may think of turning down the volume for a specific stream, he may not want to search different devices for the stream. Apart from that, most users do not have 3 or 4 devices/sinks, so much space would be blank, despite we have plenty of information to present.
About the drag and drop part: This could be really messy and inconvenient, especially given the space. Apart form that I do not know another applet handling it like that.

To avoid confusion I might present my idea again cleanly (using an svg mockup instead of the **** approach I used for the initial mockups) and considering all the input from others.

EDIT: Rethought some parts of my comment

What do you think about tabs "Applications" and "Devices"?

Quick proof of concept for grouping. Right now indention is completely arbitrary and the background highlight as seen in the network manager applet is missing. Also output labels are still there, although we might ditch them.

Default view everything collapsed, only showing devices:


Clicking on first device to expand it shows all streams running into that device:

kdeuserk wrote:There are a plenty of possibilities to groups stuff (after devices, input/output, individual streams, among others). While the grouping hierarchy after devices makes sense logically for simple uses I see a lot of problems with that approach for more complicated use cases: Just think of a stream being assigned to different devices and all the duplicated entries (After all in most of the cases you have more streams than devices, so listing devices when pressing an individual streams seems to be far better from an orgaizational pov.).

If a stream can be assigned to multiple devices then of course this organization does not make sense. I was not aware of that possibility, though (I'm only aware of the "move to device" function, not "play on these devices"). I thought each stream could be only assigned to one device, in which case that grouping would make sense, and listing devices for each stream would make no sense as there'd be only one for each.
If there is an m:n relationship between streams and devices, we indeed have to come up with a different representation.

The user may think of turning down the volume for a specific stream, he may not want to search different devices for the stream. Apart from that, most users do not have 3 or 4 devices/sinks, so much space would be blank, despite we have plenty of information to present.

I would not favor the expander paradigm. I'd always show all streams, just grouped by device (with the per-device volume shown above the streams assigned to that device).

About the drag and drop part: This could be really messy and inconvenient, especially given the space. Apart form that I do not know another applet handling it like that.

One would have to see it in action to decide that.

What do you think about tabs "Applications" and "Devices"?

It all depends on whether each stream is always assigned to exactly one device, really. If that is the case, then grouping the streams under devices would clearly be the best representation, as changing the volume of a device affects all streams assigned to it as well. Tabs would not make that clear.

kdeuserk wrote:Just think of a stream being assigned to different devices and all the duplicated entries (After all in most of the cases you have more streams than devices, so listing devices when pressing an individual streams seems to be far better from an orgaizational pov.).

One SinkInput (audio stream) in pulseaudio has exactly one Sink (device). So the only scenario where you could have the same application show up under different devices is if that application has two distinct SinkInputs (which in turn warrant individual control one would argue).

kdeuserk wrote:The user may think of turning down the volume for a specific stream, he may not want to search different devices for the stream.

I think one usually hears which device it is. But yeah, this is a bit tricky. I think in the end we can't have the popup applet address every possible scenario perfectly which is why I was looking into a more advanced gui as a standalone application (a la pavucontrol). Space is just too limited.

kdeuserk wrote:Apart from that, most users do not have 3 or 4 devices/sinks, so much space would be blank, despite we have plenty of information to present.

If there is only one input and one output device we could expand both (or at least output) by default, in fact I would find that very reasonable.

kdeuserk wrote:What do you think about tabs "Applications" and "Devices"?

I can not imagine that there are sufficiently common use cases that would warrant tabs to be honest. A config option might be more suitable perhaps.
Maybe it is just how I do volume control though. 99% of the time I will only change the device volume and maybe once a week have to fiddle with an application specific volume.

apachelogger wrote:

kdeuserk wrote:Just think of a stream being assigned to different devices and all the duplicated entries (After all in most of the cases you have more streams than devices, so listing devices when pressing an individual streams seems to be far better from an orgaizational pov.).

One SinkInput (audio stream) in pulseaudio has exactly one Sink (device). So the only scenario where you could have the same application show up under different devices is if that application has two distinct SinkInputs (which in turn warrant individual control one would argue).

Oh, amendment to give some more technical insight in case someone cares:

• One Client (i.e. application) can have multiple SinkInputs (output streams), as well as multiple SourceOutputs (input streams). A Client has no volume of its own. It is a purely organizational unit.
• One SinkInput has exactly one Sink (output device).
• One SourceOutput has exactly one Source (input device).

So, iff we were to represent volume control on a per-Client basis (bundles of inputs/output streams) rather than individual SinkInputs/SourceOutputs then one Client can in fact have multiple different devices associated with it. At that point the client also has multiple streams. All of which is why I would actually dismiss this form of representation right away as we'd end up with a three level nesting to get to a device volume slider . Still, technically a lot of things are possible.

apachelogger wrote:Quick proof of concept for grouping. Right now indention is completely arbitrary and the background highlight as seen in the network manager applet is missing. Also output labels are still there, although we might ditch them.
Default view everything collapsed, only showing devices:
Clicking on first device to expand it shows all streams running into that device:

I like it, but it feels a bit off. In terms of users, they probably wouldn't recognise a sound chip as a "thing". It's also not immediately obvious you can click a device to open it. Since the device is serving as the master, (and I don't know how possible it is) but I'm wondering if we should draw markers to illustrate what the master control is doing, and make the percentages based on overall volume (as opposed to individual max volumes). Mockup below.



For the many-to-many relationship of sources/outputs, I'd probably think controlling one output overall is more important than an individual source (and the outputs its going to); It limits functionality a bit, but in all reality we should consider the "Simple by Default, Powerful when Needed" mantra and try to have the tray controls be the simple default, and have opening an advanced dialogue be the more powerful option; even if that means we need to lose some flexibility.

What about having a dropdown with the following options in the settings:
"Control devices and applications separately"
"Group applications by device"
"Group devices by application"
"Only show devices"
"Only show applications"

... Hrm.

Kver wrote:I like it, but it feels a bit off. In terms of users, they probably wouldn't recognise a sound chip as a "thing". It's also not immediately obvious you can click a device to open it. Since the device is serving as the master, (and I don't know how possible it is) but I'm wondering if we should draw markers to illustrate what the master control is doing, and make the percentages based on overall volume (as opposed to individual max volumes). Mockup below.

It is an awesome idea, unfortunately... weird things ...

Volume relative to master will only work in certain Pulseaudio setups as pulseaudio supports two ways the volume might be controlled:

• flat volume: 100% of application volume = 100% of hardware volume. As application volume gets increased it will also increase the master volume such that the application with the highest volume in fact also is the application that dictates what volume level the hardware is at. 'Master' at 55% volume means there is at least one application at 55% and an application at 50% volume will be 50% less loud than the overall capacity of the hardware (i.e. actually 50% of maximum hardware volume).
• master volume: 100% of application volume = 100% of whatever % the hardware volume is. Application volume does not influence master volume. Hardware at 55% volume means an application at 50% volume will be 50% less loud than 55% the overall capacity of the hardware (i.e. 50% of 55% -> effectively the application is at 27.5% hardware volume).

When pulesaudio operates in flat volume mode I think it would be very reasonable to visual represent the relative maximum
For master volumes it would probably lessen the experience. If your master is at 25% you only have 25% of the width to slide in when changing the volume.

Random note: implementing the visual cue suggested requires changes to the plasma slider item (and possibly also the plasma widgets svgz)

I revised my working copy a bit to align with your suggestion (still in rough shape though )

Kver wrote:For the many-to-many relationship of sources/outputs, I'd probably think controlling one output overall is more important than an individual source (and the outputs its going to); It limits functionality a bit, but in all reality we should consider the "Simple by Default, Powerful when Needed" mantra and try to have the tray controls be the simple default, and have opening an advanced dialogue be the more powerful option; even if that means we need to lose some flexibility.

What about having a dropdown with the following options in the settings:

I absolutely agree.

This last mockup looks very good. I would move applications still a little more to the right, so that the difference is absolutely clear.

The mockups look great!

Would the device grouping be exposed by default? Suppose Susan is browsing a website in Chrome and when she opens a particular site it automatically (and annoyingly) starts playing a video or music over the music she was enjoying in Amarok. For this scenario, it would seem like the sound devices groupings are additional information she has to wade through just to find the volume for Chrome so she can turn it down or mute it. There are probably other similar scenarios that others might be able to think of.

I'm definitely not suggesting we shouldn't have device-specific volume control at all. I'm just thinking "simple by default, powerful when needed" out loud. If this was already thought about then never mind my comment.

In any event, it's exciting to see the progress with this.

alake wrote:The mockups look great!

Would the device grouping be exposed by default? Suppose Susan is browsing a website in Chrome and when she opens a particular site it automatically (and annoyingly) starts playing a video or music over the music she was enjoying in Amarok. For this scenario, it would seem like the sound devices groupings are additional information she has to wade through just to find the volume for Chrome so she can turn it down or mute it. There are probably other similar scenarios that others might be able to think of.

I'm definitely not suggesting we shouldn't have device-specific volume control at all. I'm just thinking "simple by default, powerful when needed" out loud. If this was already thought about then never mind my comment.

In any event, it's exciting to see the progress with this.

Hrm, that's a good point, and an excellent use-case. Especially since, in this scenario, the user *must* potentially turn down several individual sources, or mute the entire machine first. What if it was an unexpectedly loud telephony call and you didn't want to mute the person for 20 seconds while you adjusted knobs?

Maybe it is best to do away with grouping entirely. Have the widget list applications, then have it list outputs, then list inputs. Similar to the 4th mockup on the first page, but with clutter reduced; then, again, go to the dialog for more advanced controls.



Some notes about this one;
- The apps 'listening' are grouped under the microphone, one-click mute, with the dropdown to individuall toggle volume (not shown)
- Lots'-o-quick-mutes everywhere.
- Maybe speakers should displayed above the inputs. My bad. They would look like the app versions though. I would still avoid using the 'chip' as the graphic though; if possible preferring the best guess as to what it is, be it internal or external speakers.
- Can we re-color the sliders to disabled colour, and retain the ability to adjust the values? This way we could visually illustrate that the slider represents a muted device, but still allow the adjustments if the user wants to turn down the volume before un-muting.

And again, this doesn't show app <-> speaker relations, just the global controls for speakers, and the global controls for apps.

apachelogger wrote:flat volume: 100% of application volume = 100% of hardware volume. As application volume gets increased it will also increase the master volume such that the application with the highest volume in fact also is the application that dictates what volume level the hardware is at. 'Master' at 55% volume means there is at least one application at 55% and an application at 50% volume will be 50% less loud than the overall capacity of the hardware (i.e. actually 50% of maximum hardware volume).

What if I directly set the master volume to e.g. 40% then? Does it reduce any application volumes that are > 40% to 40% and leave those <= 40% where they are?

I revised my working copy a bit to align with your suggestion (still in rough shape though )


I like it, though I agree with pedrorodriguez that increasing the indentation a bit would make the grouping a bit clearer.

alake wrote:The mockups look great!

Would the device grouping be exposed by default? Suppose Susan is browsing a website in Chrome and when she opens a particular site it automatically (and annoyingly) starts playing a video or music over the music she was enjoying in Amarok. For this scenario, it would seem like the sound devices groupings are additional information she has to wade through just to find the volume for Chrome so she can turn it down or mute it. There are probably other similar scenarios that others might be able to think of.

I'm definitely not suggesting we shouldn't have device-specific volume control at all. I'm just thinking "simple by default, powerful when needed" out loud. If this was already thought about then never mind my comment.

Admittedly, I don't get how the grouping would complicate things. Users with simple setups (all streams playing in the same device) would not have to wade through much additional information to get to the application they're looking for. In fact, the only difference would be that the master volume would be above the applications instead of below. Then again, for me that would make more sense anyway, because I rarely see "slaves" listed above "masters" anyway.
This assumes that the grouping is always expanded, which I'd strongly prefer. In that case, Susan would not have to know which devices the application whose volume she wants to change plays on, as it would be visible right away anyway.

Kver wrote:Hrm, that's a good point, and an excellent use-case. Especially since, in this scenario, the user *must* potentially turn down several individual sources, or mute the entire machine first. What if it was an unexpectedly loud telephony call and you didn't want to mute the person for 20 seconds while you adjusted knobs?

Maybe it is best to do away with grouping entirely. Have the widget list applications, then have it list outputs, then list inputs. Similar to the 4th mockup on the first page, but with clutter reduced; then, again, go to the dialog for more advanced controls.



And again, this doesn't show app <-> speaker relations, just the global controls for speakers, and the global controls for apps.

And that is the problem I see with this: It makes it looks like devices and application streams are completely unrelated, which they are not. We're failing to visualize important relationships. User story:
Susan asks herself why she cannot hear the music from Amarok even though its stream is not muted.
The stream is set to play on her headphones, which
a) Are turned off hardware-side
b) Are muted
Both reasons can easily be detected in a grouped view, but not in a flat view

Yes, for users who usually use only one device and have all streams play there, there isn't much benefit of the grouping. Then again, there isn't much disadvantage, because the only thing the grouping changes is that there is an additional line for the master volume of the device above the list of applications. Is one additional line really much of a problem?

This example clearly shows the problem of not having any user/usage data to work with at all. Without actual data, we rely on assumptions, which in turn are usually based on our own usecases. I change master volumes far more often than application volumes, you and Ken apparently change application volumes more often.
Who is closer to "the mysterious average user"? We simply cannot tell.

Fair points all colomar. Perhaps the grouping as described would be just fine after all. I just wanted to share a couple thoughts I had. It seems the design works fine for the scenario I mentioned.

Ok, never mind me. Carry on.

alake wrote:Fair points all colomar. Perhaps the grouping as described would be just fine after all. I just wanted to share a couple thoughts I had. It seems the design works well enough for teh scenario I mentioned.

And I thank you for doing this! We can only produce great design if we challenge our suggestions, so this was definitely helpful.

Perhaps device strings like "Speaker", "USB Headphones", "Microphone" would be helpful as well (and still allow exposing the technical device string as well).

Definitely! I don't know if there is any way we can find that out, but it would give an incredible boost to the usability of the whole thing.
I currently have the devices "Built-In Audio Analog Stereo" and "Turks/Whistler HDMI Audio [Radeon HD 6000 Series] Digital Stereo (HDMI)". How is a non-technical user ever supposed to make sense of these names?

colomar wrote:

apachelogger wrote:flat volume: 100% of application volume = 100% of hardware volume. As application volume gets increased it will also increase the master volume such that the application with the highest volume in fact also is the application that dictates what volume level the hardware is at. 'Master' at 55% volume means there is at least one application at 55% and an application at 50% volume will be 50% less loud than the overall capacity of the hardware (i.e. actually 50% of maximum hardware volume).

What if I directly set the master volume to e.g. 40% then? Does it reduce any application volumes that are > 40% to 40% and leave those <= 40% where they are?

Everything is reduced relative to 'master'. If master was 55% and the loudest application was 50% then it will be 35% after you changed master to 40%. Another app that was at 40% would now be at 25% (actually it wouldn't because of non-linear loudness scaling, but for the sake of argument I'll pretend it's linear ).

The primary difference to a setup without flat volumes is that the percentage is always the actual percentage of hardware capability and thus increasing the application volume beyond the present device volume will increase the device volume (thereby also relatively increasing all other applications), so one application can make the entire system louder. Which is why some people consider the flat-volume setup bad as a misbehaving application can blow out your ears by bumping volume to 100%. Ubuntu for example doesn't use flat volumes by default.

colomar wrote:

Perhaps device strings like "Speaker", "USB Headphones", "Microphone" would be helpful as well (and still allow exposing the technical device string as well).

Definitely! I don't know if there is any way we can find that out, but it would give an incredible boost to the usability of the whole thing.
I currently have the devices "Built-In Audio Analog Stereo" and "Turks/Whistler HDMI Audio [Radeon HD 6000 Series] Digital Stereo (HDMI)". How is a non-technical user ever supposed to make sense of these names?

Theoretically there are optional properties in pulseaudio that would allow us to build more understandable strings. Having tried it on my system however, it doesn't seem to give me the necessary metadata for any device other than builtin speakers.

So, unfortunately I fear in practise this won't do us much good. We probably could try to come up with some sort of algorithm that breaks down all the meta information we have available to deduce whether something is a headphone or a speaker or a tv... I fear at its best that still would only be an educated guess though. I'll have to talk to our resident PA guy, maybe he has a suggestion. If not what we have right now is probably the best we'll get in the foreseeable future.

Hey everyone. Having used the nested structure with devices on the first level and apps on the second level for the better part of this week, I think it feels really good. In fact, it appears that everything is coming together so nicely that the applet is almost release quality already \o/.

There are still a couple of semi-relevant features missing though as they mostly have not been considered in the design so far. So I went ahead and made a list off the top of my head.

Not addressed in the design yet:

• How to set the default/fallback device for all streams at once (i.e. reroute all output to a different device or all input to a different device)?
• How to set the port of a card? Ports are for the most part actual physical ports on the hardware, for example the Headphones and the Speaker port. Changing the port offers necessary control over which physical ports will actually output the sound.
• How to set the profile of a card? Profiles are essentially controlling which ports will be available. For example one card might offer Surround 5.1 which would result in limited port selections (those that can do 5.1) while a Stereo profile would offer both Headphones and Stereo as available ports as both can produce stereo sound.
• How to set the port and profile of a card?
• What happens when one scrolls on the applet icon in the system tray? Suggestion: relatively increase/decrease the volume of all devices.
• Notification sounds from my understanding get a dedicated volume control disregarding the volume of the device, so a separate volume control for those needs to be put somewhere.
• Should there be audible feedback (i.e. a notification sound) when one changes the volume? And if so, when is it played.
• Volume overdrive. Pulseaudio allows the volume to be increased beyond 100%. This is an often requested feature and should be supported in some form. It likely needs visual cues as to where 100% is and where the overdrive limit is (usually applications will allow up to 130% although technically much more is possible).

Features possibly not suitable for applet (KCM?):

• Terminate a stream. This is pretty much a kill feature killing the entire stream. Since this is roughly equal to simply muting a stream I find it a questionable feature at best and with unaware applications this can actually mess up the application such that it requires a restart to output audio again.
• Multi-channel volume control (i.e. volume balance). This actually has some inherent design problems from PulseAudio which makes me not want to spend to much brain power on it. For now I consider this very advanced feature.
• Display of virtual devices (e.g. monitors allowing the use an output device's output as input) and virtual streams (I don't think there is anything implementing a virtual stream at this point).
• Rename a device.
• Change device latency (technically this is a port-dependent latency).

There also is the possibility of having a volume meter (i.e. a bar that actually displays the volume of whatever is output/input right now). It is really useful for input configuration so we might want to put it somewhere, alas, I would actually argue that it is so useful it might actually be good to find a way to put it in the applet.

Glossary (because we have not really made clear what we are talking about I noticed )

• card: A physical device. Within pulseaudio this is a mostly organizational unit above input output devices that allows control over what abilities can be offered.
• port: A physical port of a device. For example 'Headphones' and 'Speakers' port.
• profile: A configurational profile of a card, regulating which ports can be used interchangeably. For example Headphones and Speakers are both stereo so one profile will be able to offer both ports as options. Surround 5.1 however can have streams with up to 6 channels which makes it impossible to directly switch to Headphones.
• device: An input or output device. Each card has usually 0-1 inputs and 0-1 outputs.
• stream: A virtual output entity created by an application. A stream will either be an input or output stream and will have exactly one device it outputs to or gets input from.
• application: A piece of software using one or more streams to generate output or obtain input.
• volume: Is the loudness of either a device or a stream.

PS: I am also trying to get you a way to easily test what I have, more on this next week I guess.