I keep hitting a roadblock when shading my images. What is the best way to select one side of a line, and not the other? Assuming of course that the magnetic select doesn't work (line too complicated) and the magic wand doesn't work (one side of the line isn't directly separated from the other), what is the best way to select one side of a line?

Let's say I have this weird oblong shape. What is the best way to add a gradient to the left side, and not the right? So that the final product looks like this.

obviously I would first have to mask off the left side with a selection tool. In this example I used the polygon tool, and it took me a bit of time to go in and out of each little crevice of the 'pointy' bits. There has to be a better way. Let's look at a couple improvements which could save hours in the long run:


1. Limit to current selection

Limit to current selection is a way to define limits of the magic wand tool's search. A user first selects an area to search, then uses the magic wand tool inside of said selection to only match contiguous colors within the threshold inside of the selection.
For example:





2. Fuzzier or 'smarter' magnetic lasso
The magnetic lasso is great by itself, but it's obvious it was designed for blurrier pics and more general selection, such as cropping someone out of a background. I feel it could receive a couple optional enhancements to make it an absolute beast when it comes to linework. For example:




I feel that one or both of these would improve the lives of myself and many of my friends so greatly that I'm willing to put money toward helping the production of the tool if need be. I'm just an animator, so it's likely not much in terms of programmer funding, but I figured I'd try.

If you can imagine a boundary around a zone, you can draw required additional boundary lines on a separate layer and then use the Fill tool to fill the zone with colour. Then use the Similar Colour Selection tool to select that filled zone.
Using a combination of selection tools with the intersect, add, subtract variant options and manually defined/filled zones, you can create any arbitrarily shaped selection you like, if you want to get really complicated selections.

Here is your original artwork divided into three zones by using two additional 'separation' lines:



The end result with additional line and fill layers turned off is here: https://i.imgur.com/sTazV2G.png

While this would work in the example, in actual practice it's not that easy.
In order to allow separation lines to affect the magic wand's selection, you have to have 'Limit to current layer' turned off. Which means all other layers in the document affect that selection. And when you have 50-700 layers as I usually do when trying to do selections, that becomes a big problem very fast.

As a more practical example, let's say I wanted to add shading to the underside of this Zebra, and a white gradient on the leg to make the stripes fade out. Let's also say that every single stripe is on a different layer. Yes, in practice you wouldn't have them on separate layers but we're merely emulating scene complexity here for the sake of me not having to spend more than 3 minutes drawing up something more complicated as an example.



In the end, I want it to look like this:



There's no way to accomplish that without:
- Manually tracing out the outline of the selection, which if the linework is complex enough, wastes significant time that could easily be automated.
- Unchecking all layers besides the linework layer manually. (Becomes impossibly time consuming after about 40 layers or so)
- Using shift-click on the layer to temporarily isolate it for the magic wand. (I did this once on a pic with 600 layers, I accidentally clicked the eye on the wrong layer when I went to un-isolate it, and then spent the next 45 minutes figuring out which of the 600 layers were originally visible and which were hidden. Needless to say, I am never, ever, touching the isolate tool again until it has protection from that.)

For the zebra example, I'd have stripes in different groups for body, leg and tail and then I'd group those groups so that all stripes could be turned off/on with a single click.
Whatever fancy selection tool you can think of will come up against this problem of 'what do I pay attention to?'.

For the zebra, make a copy of the body lineart layer, add separation lines to the copy and then use the single layer colour fill and select method on that modified copy to create the selection. Then make the modified copy layer invisible and make the gradient layer using the selection.

The first selection tool I proposed doesn't even have to ask 'what do I pay attention to?' In fact, in the first tool example, you explicitly define what to pay attention to with the bounding selection. Any code that asks 'what do I pay attention to?' is already there, in the magic wand, you're just adding a way to set a limit to it's search.
The second tool would be a stretch, yeah. And would require a whole bunch of logic and deliberation. I included it primarily as an example and don't expect it any time soon. An artist can dream.

As for the 'separate line layer for selections', there's two things wrong with that. First off, selections on transparent and black lineart layers are broken right now. Second, even with decent layer grouping on large projects, you have to scroll ALL the way up or down to your 'fake lineart layer' to make a selection, then all the way back to your working layer just to gradient in one part, then repeat 50-100 times for all the other parts. Either that, or keep dragging the 'selection layer' around with you as you navigate in and out of folder. It's a bit barbaric and also why I highly support selection reference layers.

Now I'm not at all familiar with how the magic wand function actually calculates the selected area, but if it's a 'branching out' method from the target pixel, I don't see how defining a selection area or even a bounding rect would be too difficult, from an idealist standpoint.

Like I said, the current selection tools can be so insanely annoying on professional projects to us as artists that several others and I have got together wanting to help fund something like this. But none of us are coders and none of us have ever funded anything like this, so we have no idea if that's how it even works, if someone would want to take it on, or if our budget is even considerable.

Ah, I see the problem, I think.
The magic wand is just a repurposed fill tool that only selects, and Even the fill tool fills before restricting to selection.
That kinda sucks. But at least if it gets fixed, that also is a plus for the fill tool and any Contiguous Selection functions.

Looking further into it, this seems to be either a polarized issue or just a common oversight. I have 5 years of Clip Studio Paint experience and 4 years of Photoshop, so that might be why it's such an issue for me.

Chiming in with RalekArts regarding the behavior of the magic wand tool. Having a more robust and granular behavior to the magic wand tool would go a long way for ease of use, comfort, and familiarity for us artists.

The people who are doing funded development don't get rich from it, so funding development doesn't take commercial rates. The problem is that we're already trying to do so much work in so many areas that we're quite overloaded already. And even though we're all working from home anyway,the pandemic is also taking up quite a bit of brain power through the anxiety it causes.

That said, if we the requirements are clear enough -- and they aren't currently -- and we can make a plan for it, I might be able to find someone to enhance the selection tools. Most projects take longer than you'd think, though. Count on ~2500 euros for four weeks of work.

Hm, the post with the differences between gimp/paintstorm/krita and the others is interesting. It shouldn't be too difficult to add a switch for that to the fill tool.

Alright, I'll try my best to be as comprehensive as possible and list out some of the major things I'd love to see changed about the selection tools.
I don't know how long they would take, and thus I can't really put an estimate on the production cost. I have about 1200USD collected right now, but depending on whatever sort of contract or funding is decided, that can be flexible and I can definitely up it as needed by supplementing my own income. Especially considering Krita is used in my main source of income and investing in it is in a sense investing in my own abilities and future.

Because I'm asked to be concise and clear, I'm going to refrain from suggesting any other alternative ways to accomplish the goal as to not muddy the water and obfuscate the requirements. Obviously there will be more than one ways to change the UI or accomplish the same task, I'm not going to elaborate.
Any examples provided are extremely overly simplified for the sake of being examples of how the proposed idea would work functionally, I'm not looking for alternative solutions to them, and I'm not going to keep spending time producing harder examples to demonstrate that the current methods do not scale well. I wouldn't be here putting my limited starving artist income toward this if I didn't feel it would improve my life as said artist substantially.


1. Ability to restrict magic wand tool to current selection.
There needs to be a way to restrict the magic wand's search to a specific area, without directly storing said area's information in the form of a layer. As far as I'm aware, this isn't a function I've ever seen on another art software. And as a digital artist of almost 10 years, I'm not quite sure why it wasn't thought of before. Arguably, this primarily benefits artists who work with hard linework, and graphic designers. So this wouldn't help painters as much, but let's be honest how often does a painter use a magic wand anyway? To be honest I wouldn't know, I'm not a painter, but I doubt it's as often as me.

Why this is important to me:
Very often I've wanted to select one body part of a drawing only to realize it's connected to an adjacent body part such as an arm, with no defining line between them. The only current method to accomplish this is to manually select the body part, or turn off the magic wand's 'limit to current layer' checkbox. The later of which is, in actual practice, nearly entirely impossible. And the former of which is what I've been doing, and is proving extremely time consuming, tedious, and frustrating enough for me to suggest this improvement.

Requirements:

• When a selection is already made through some other means, you have the ability to limit your magic wand to only inside that selection.
• The Magic wand must not process anything outside of the pre-selected area to avoid the selection leaking where it should not. See: Example 1, Example 2.
• Like the additive, subtractive and intersection shortcuts for all selection tools, this new method of 'limiting to current selection' can be evoked with a modifier shortcut when wielding the magic wand tool.
• Must work with any selection shape, including oblong or semi-transparent selections, taking into account the transparency of the selected pixels as a bool where any amount of transparency in a selection mask is counted as a 'searchable' pixel. See: Example 3.

Example 1
Blue is the area I would like to have selected in the end. If the above is implemented correctly, clicking inside the blue area with 'limit to selection' evoked or checked would return a selection of just the blue area.
This is because it travels out from the clicked pixel, checking for the fuzzy threshold, until it hits the black bar, and stops as the black bar is above the fuzzy threshold. It also hits the top, right, and bottom sides of the current selection, and stops.
It is important to limit the search to only the selection, because if the search is done before being applied, the pink box above will also be selected, and this is unwanted behavior. The magic wand would select the entire white area around the box, then act the same as an intersect if not otherwise restricted to the pre-selection.

Example 2
Blue is the area I'd like to have selected. By clicking inside the blue area, the magic wand searches the white pixels within the selection until it either hits the edge of the selection, or the black box which is outside of the threshold. None of the red area gets selected, as none of the selection area there is contiguous, despite there being no defining border between the two selections in terms of color difference.

Example 3
On the left is what the user sees, on the right is a visible mask of the 'current selection'. The current selection is a gradient, with the pixels at the top being 'less selected' than the ones at the bottom.
The blue area is the area I would like to select. Notice that it is less selected at the top, and more at the bottom. The 'current selection's mask opacity is disregarded, pixels which are 0 alpha are ignored and considered a hard stop for the magic wand's search. Anything above a 0 is considered searchable, wholly, and considers said pixel's color in regards to the fuzziness. This it to prevent accidental 'doubling up' on the opacity of your selection. (Imagine clicking the same 80% opacity selection with the magic wand over and over, the selection would eventually reach 0%)
After the magic wand has done its search and returned the matching pixels, the transparency of the original selection is re-applied to the new magic wand selection, preserving the gradient inside the box.



2. Reference layer
The ability to choose the entire canvas or just a single layer is great, but it can be too restrictive in situations where a single layer is not possible, and where the alternative of the entire canvas is significantly too cluttered and complicated. I propose that the 'limit to current layer' is switched to a drop-down, with these selections:

• Sample all layers
• Sample current layer
• Sample reference layers

When 'Sample reference layers' is selected, a color select dropdown shows up below it, or becomes active. This dropdown mimics the layer color dropdown.
There are several good reasons for this.

• It gives the layer color attribute a bit more depth and usefulness as a general flag rather than purely visible
• It allows multiple different 'reference groups' to be organized, putting this 'use reference layer' tool above all other programs I've used with similar tools in flexibility.
• It means there's no need to clutter up the layer panel with more flags alongside the transparency lock, alpha lock, lock, etc.

When you select, say, red, in this dropdown, Krita will act as though 'Sample all layers' is checked, but that only the red layers are visible. This prevents users from having to constantly swap between currently selected layers to make selections, set their lineart layer to isolation mode, or any other weird workaround that 'works' on a small scale, but suddenly becomes a monumental task when you get to any sort of actually complicated scene.

Why this is important to me:
As I said above, I'm fine with workarounds. When they actually work, at least. And most of the ones suggested or I've found do work for sketches, small projects, basic coloring, etc. But the second I get into realistic renderings, backgrounds, scenery, multiple characters and 10+ shading layers per section, those workarounds suddenly become extremely, grossly under powered and cumbersome. I feel Krita should be about letting you create and less about trying to find a way to accomplish something in the least painful way.

Requirements:

• A way to define one or more layers as a 'reference layer'. Allowing for multiple reference 'groups' optional, though a plus.
• A way for tools that reference pixel data to make a selection or fill an area to be used with not only 'all layers' and 'selected layer' but also 'reference layers'. This includes tools such as magic wand, similar color selection, magnetized color selection, and the fill tool. Colorize mask tool optional and probably not necessary, since I haven't found a need for it to reference anything outside of a group. 'Overlay mode' for the smudge brush engine also optional. Might help out painters, though.

Example 1
In this example, we'd normally be forced to use one of the few workarounds, such as hiding all of the splotch layers, manually selecting the area we want, or any of the others suggested here that don't usually work in practice or take eons to accomplish. But if we already defined our outline layers and 'gate' layer as what we want to reference when making selections, all we have to do is limit our fill tool or magic wand tool to the red layers to get the selection we want, no hassle.


That's about all I can think of in terms of selection improvement for now. besides various other small bugs I'd found in my time with Krita. I want to help out best I can, since Krita is the first program I work with that I actually feel I could make a difference on. Unfortunately I'm just an artist and not a programmer, so all I can really do is provide feedback that would make my life easier and report what I think are bugs. The former of which I can help with funding, even if I may not be a big help there either. Any sort of guidance for what to do next or how best I can help is appreciated.

I don't think that it's possible or wise to try to make Colorize Mask (Tool) work on several unconnected layers because it was meant to be a mask... it might confuse the UX quite a lot.

For the original idea of "fuzzy magnetic selection", I wonder if one couldn't reuse the Colorize Mask algorithms for that... but that would be a longer, bigger project.

The rest of it is pretty clear and I started working on it. (About money you guys should probably talk privately anyway, btw.)

I believe this bug is related to Feature 1: https://bugs.kde.org/show_bug.cgi?id=415439

Hi RalekArts, I implemented the second feature (reference layers in Fill Tool and magic wand, other selection tools might come later), you can test it here (Windows build made by Dmitry): https://yadi.sk/d/e8fbkiC6z4rqnA - it would be good to get your feedback.

This is absolutely exactly what I was thinking of, and the implementation is perfect! This will save an absolute boatload of time selecting sections of lineart in more complicated scenes.
I did however run into 2 errors when trying to break it (they always happen at the same time, one after the other), and even more weird, this only seems to happen on my Cintiq when using my pen. I haven't been able to get the error while using my mouse, either on my main screen or my tablet screen.
The first error that pops us is

The second is

these, as a pair, pop up between 1-6 times, occasionally around 30 times, and rarely (about 1 out of every 20 times) they will pop up so many times at once that the program crashes entirely. Ignoring them continues with the selection, and nothing seems to be wrong.
Here's the test file I've been able to reproduce the issue with consistently

@RalekArts, does it happen with both tools (Fill Tool and Contiguous Selection Tool/magic wand), or just one? If one, which one?