Okay, since I have no time at the moment for that I want to make it easier for others to start where I left, here is the link to the modified mockup toolkit (be aware the spacing etc. was mostly measured by eye, when I have time I will correct that), but it should allow you to mockup things quickly.

https://drive.google.com/file/d/0B-ihXi ... sp=sharing

BTW: alake, do you have plans for enhancing the vdg toolkit? some elements in the version from may are outdated now. want to hangout on irc?

When I have time for that again I will look into enhancing and improving that.

kdeuserk wrote:BTW: alake, do you have plans for enhancing the vdg toolkit? some elements in the version from may are outdated now.

The latest version of the mockup toolkit was updated a under 3 weeks ago and contain the up-to-date visuals for all UI controls and examples for the layout patterns described in the HIG. Could you share what ideas you have in mind for enhancing the toolkit? Feel free to start a new thread since it might be off-topic for this thread.

I do like the concept. I am not too fond of the advanced settings though :S

There's a presentational problem with the balance and devices: there maybe 0 of them and there may be a 100 of them. The fact that the amount of both is pretty much dependent on someone's setup while the tray popup always has a given size, makes it more than likely that the plasmoid will look way too crowded on way too many systems.

This is what my pavucontrol for VLC looks like, and that's only with 5.1 output:


Now imagine this squeezed into the tray popup

For devices (aka sinks) this is less of a problem as we can expect the majority of people having <=3 devices at a time (builtin analog, hdmi, external headset - latter is already unlikely but should still be common enough). What makes devices a bit of an annoying thing to do deal with is that technically speaking there should not be a common need to move one application to a device different from the default as a well-behaved application will request a suitable device from pulseaudio to begin with. But assuming it is not very well behaved it ought to still be moved to the same device as previously configured by the user. So, at the end of the day you should only need to set the device once, ever, after that pulseaudio should correctly force the application to use the same device in subsequent sessions (assuming the device is plugged in etc. etc.). Considering per-application-sink-switching ought to be an unlikely thing one does daily I do not think device selection should be as prominent as originally proposed. (mind you, if things are not behaving like that it is a bug or a lack of policy enforcement deeper in the pulseaudio stack).

I would not offer balance or per-app device control, but instead offer global device control (i.e. route all sound to a different device) and either throw all the advanced stuff into a KCM (in fact I have already started to ) or into a context menu. Also, I am not yet convinced the distinction between output and input is clear enough, if we only differantiate between them via the in/out string and the icon then I feel the application icons might want an overlay in the bottom right corner reflecting whether the stream is an input our output.

Sorry if I started rambling, I am not quite in possession of all my brain power due to a cold.

apachelogger wrote:For devices (aka sinks) this is less of a problem as we can expect the majority of people having <=3 devices at a time (builtin analog, hdmi, external headset - latter is already unlikely but should still be common enough). What makes devices a bit of an annoying thing to do deal with is that technically speaking there should not be a common need to move one application to a device different from the default as a well-behaved application will request a suitable device from pulseaudio to begin with. But assuming it is not very well behaved it ought to still be moved to the same device as previously configured by the user. So, at the end of the day you should only need to set the device once, ever, after that pulseaudio should correctly force the application to use the same device in subsequent sessions (assuming the device is plugged in etc. etc.). Considering per-application-sink-switching ought to be an unlikely thing one does daily I do not think device selection should be as prominent as originally proposed. (mind you, if things are not behaving like that it is a bug or a lack of policy enforcement deeper in the pulseaudio stack).

I must admit that I haven't really understood that paragraph, so I'll just present my typical usecase for moving an application stream to an output device:
My PC has its own speakers connected and is also connected to my TV (with attached home theater system). Usually, the TV is off and I use the PC's speakers. However, when I for example I want to watch an online video on my TV, I switch the TV on and move the browser's stream to the TV. I might not want to have system sounds on the TV, because they're only relevant to me when I'm at the PC.

colomar wrote:My PC has its own speakers connected and is also connected to my TV (with attached home theater system). Usually, the TV is off and I use the PC's speakers. However, when I for example I want to watch an online video on my TV, I switch the TV on and move the browser's stream to the TV. I might not want to have system sounds on the TV, because they're only relevant to me when I'm at the PC.

Yep, so this makes a lot of sense. I didn't think of this at all.

There are some technical problems (from PulseAudio) that make this somewhat undesirable however. Most notably moving an output to a device manually actually breaks the category system seen in the multimedia section of systemsettings. The manual override always takes preference over the category settings, which is incredibly confusing for people. There have been numerous bug reports on this already.

That is not to say that there is much of a way around this to address the pc-with-tv-attached use case. In general however I would argue that in a perfect world you would not want to route an individual application to the TV but a whole category (e.g. video). Whether or not that would fly I'll leave for discussion ^^

We do however have a conceptual problem I noticed. Half the system talks about categories (e.g. Phonon, Pulseaudio, sensible integrated applications) while what we considered for the sound applet until now was on a device level. I believe that we should unify this by either dropping the public concept of categories or somehow fiddle categories in the applet to such an extent that one would not route applications to a specific device but categories. All very confusing really


A bit of background on categories: the main point of categories actually is for pulseaudio to (potentially) control the volume of all applications in a specific category all at once. So pulseaudio would mute everything but voip applications upon an incoming call in skype for example.
At the same time with routing in mind one wouldn't route one application to a specific devices (or very seldomly anyway) but entire categories. Video goes to the TV unless the TV is not active, if so it goes to pc speakers. Audio goes to USB headphones unless not plugged in, if so it goes to bluetooth headphones unless not paired, if so it goes to pc speakers. And so forth.
That is also why I'd argue that with a TV you'd not route one application but an entire category. Assume you want to watch something on the TV using VLC and then you want to watch something else using the browser (perhaps on netflix) you have to route two applications manually to the TV and then route them back because someone else wants to use the TV, with with a category-based approach you'd route all video content (and hope that the browser actually manages to set the suitable category )

To make up for my wall of text, here's what the current thing looks like


I think we might need to rethink the 'Input' 'Output' labels in the label row, judging from my system most of the time they'll need to cut the actual device/application names short (for apps we have a bit of control over how much we want to show, for devices the less information we show the less useful the applet gets with >1 device). Also in different languages those two terms might be substantially longer words, making matters worse. So, I am not too convinced that will work out.

I totally love the direction we are going in though

(random note: the version in the screenshot is using default spacing as used everywhere else in plasma, which is slightly less spatially 'dense' than what the original mockup suggested, hence less things fit in the popup).

I'm not sure about the idea of categories. The problem I have is that I sometimes watch programs with VLC on my TV, which is my second monitor, but keep my headphones plugged into my high quality sound card on my TV.

I think sticking with devices is the best bet. I have four sound cards, which is probably more than most, but you know what's plugged into what. There's no confusion.

So I think the applet should present a list of the devices, which could then be expanded to show the applications that are playing in or out of that device. Making a nice drag and drop interface for the apps would be nice to save right clicking on everything if you want to move them to another device.

On the subject of misbehaving apps; Amarok has a very nasty habit of resetting the output device every time you change song (well it does for me anyway when I am trying to play music through HDMI).

Also, as a very cheeky pretty please feature request. Can we add some kind of amplification (e.g increasing the volume to more than 100%) in kmix? It's especially needed for the micophone stream as depending on what kind mic you own it can be far too quiet.

davidwright wrote:I'm not sure about the idea of categories. The problem I have is that I sometimes watch programs with VLC on my TV, which is my second monitor, but keep my headphones plugged into my high quality sound card on my TV.

I think sticking with devices is the best bet. I have four sound cards, which is probably more than most, but you know what's plugged into what. There's no confusion.

I am not sure that necessarily contradicts categories. i.e. you still map categories to devices and categories internally have a preference order for the devices and as devices are available or not it picks the best available device per category.

davidwright wrote:So I think the applet should present a list of the devices, which could then be expanded to show the applications that are playing in or out of that device. Making a nice drag and drop interface for the apps would be nice to save right clicking on everything if you want to move them to another device.

I like the idea. It might fit better into a possible systemsettings module though. The space constraints of the plasmoid popup could make this potentially awkward as 4-5 entries are pretty much all that is going to fit in it. That being said, perhaps we could simply offer more features when using the plasmoid outside the system tray, at that point we don't have the notification area space constraints anymore.

davidwright wrote:On the subject of misbehaving apps; Amarok has a very nasty habit of resetting the output device every time you change song (well it does for me anyway when I am trying to play music through HDMI).

That is actually fallout of categories conflicting with manual override. Unless your HDMI driver is flaky and switches active states on track change, which would be odd ^^

davidwright wrote:Also, as a very cheeky pretty please feature request. Can we add some kind of amplification (e.g increasing the volume to more than 100%) in kmix? It's especially needed for the micophone stream as depending on what kind mic you own it can be far too quiet.

This feature has been there for a year or so, you just need to enable it.


On that note, We currently also miss a good solution for volume overdrive. That is when one increases the volume >100%. In the more advanced pulseaudio GUIs you can overdrive to 130% by default (i.e. slider actually goes to 130%) and have some visual cue as to where 100% is. Technically speaking though there is no real limit to the overdrive, could as well be 1000% if we want it to.

apachelogger wrote:That is also why I'd argue that with a TV you'd not route one application but an entire category. Assume you want to watch something on the TV using VLC and then you want to watch something else using the browser (perhaps on netflix) you have to route two applications manually to the TV and then route them back because someone else wants to use the TV, with with a category-based approach you'd route all video content (and hope that the browser actually manages to set the suitable category )

Yes, I can imagine using categories instead of applications, if the catgeory system works reliably. There are all kinds of different Flash video implementations, for example. Would they all be reliably associated with the video category?
Are all VoIP applications put in VoIP? What about Games? Same goes for all categories, I guess.

So I'd say: If at least the common applications and media streams in the browser (regardless of Flash or HTML5) are associated with the right category, I'm fine with dropping applications in favor of categories. If not, however, I'd vote for dropping categories. A category system which puts half of the streams in the wrong category is worse than no category system.

colomar wrote:Yes, I can imagine using categories instead of applications, if the catgeory system works reliably. There are all kinds of different Flash video implementations, for example. Would they all be reliably associated with the video category?
Are all VoIP applications put in VoIP? What about Games? Same goes for all categories, I guess.

So I'd say: If at least the common applications and media streams in the browser (regardless of Flash or HTML5) are associated with the right category, I'm fine with dropping applications in favor of categories. If not, however, I'd vote for dropping categories. A category system which puts half of the streams in the wrong category is worse than no category system.

I very much doubt an optional feature of pulseaudio gets or will ever get the necessary backing to make it worthwhile. So I guess we should do away with categories

now also with type-label.

apachelogger wrote:

davidwright wrote:So I think the applet should present a list of the devices, which could then be expanded to show the applications that are playing in or out of that device. Making a nice drag and drop interface for the apps would be nice to save right clicking on everything if you want to move them to another device.

I like the idea. It might fit better into a possible systemsettings module though. The space constraints of the plasmoid popup could make this potentially awkward as 4-5 entries are pretty much all that is going to fit in it. That being said, perhaps we could simply offer more features when using the plasmoid outside the system tray, at that point we don't have the notification area space constraints anymore.

I agree that drag & drop would become cumbersome within the confinements of the systray popup (I'm still hoping to see that decision reversed at some point, but for now we have to deal with it) for users with many devices and/or applications running, but that doesn't make the display of applications grouped by the device they're playing on any less useful, if only for information purposes. And I'd still enable drag & drop even in the Plasmoid, with an ability to launch a bigger window for situations where drag & drop in the small space of the popup won't work well.

So we should really explore the option to display application streams grouped by devices. Currently, it looks like devices and applications are on the same level, which they clearly aren't. If I mute the device an application is playing on, the application is effectively muted as well, regardless of how loud it is set.

apachelogger wrote:

now also with type-label.

About the input/output labels: Wouldn't it make more sense to just use a microphone instead of a speaker as the volume icon for input streams? I mean, a speaker for input levels doesn't make sense anyway, and with different icons we could clearly indicate which is which without the need for a textual label.

colomar wrote:About the input/output labels: Wouldn't it make more sense to just use a microphone instead of a speaker as the volume icon for input streams? I mean, a speaker for input levels doesn't make sense anyway, and with different icons we could clearly indicate which is which without the need for a textual label.

Certainly a possibility if we were to visually group streams under devices. With the flat layout right now we'd have to utilize an overlay icon for application streams (e.g. skype would have an input stream and an output stream) and that probably doesn't play too well with audible accessibility (e.g. using orca).

apachelogger wrote:Certainly a possibility if we were to visually group streams under devices. With the flat layout right now we'd have to utilize an overlay icon for application streams (e.g. skype would have an input stream and an output stream) and that probably doesn't play too well with audible accessibility (e.g. using orca).

Either there is a misunderstanding or you lost me here
What I meant is the little speaker icon on the right, next to the volume level. That could be changed to a mic for the input ones, right? So if an application had both input and output streams, I'd show two lines for it, one with a speaker on the right and one with a mic.

colomar wrote:So we should really explore the option to display application streams grouped by devices. Currently, it looks like devices and applications are on the same level, which they clearly aren't. If I mute the device an application is playing on, the application is effectively muted as well, regardless of how loud it is set.

Couldn't we do it the same as the network manger? So you have the device and when you click on the device it would expand with a list of programs playing though it underneath? Even if it's just a basic list with the application icon it would be better than nothing.

For reference windows 7 just give you the device sound level when you click on the system tray icon. You have to go into the mixer to get the application levels.

colomar wrote:About the input/output labels: Wouldn't it make more sense to just use a microphone instead of a speaker as the volume icon for input streams? I mean, a speaker for input levels doesn't make sense anyway, and with different icons we could clearly indicate which is which without the need for a textual label.

+1