Hello again,

I´ve gained some experience with Greenscreen Effects in Kdenlive. The Quality is pretty good.
But there is a little nasty side effect. There are some small greencolourmixed edges on my masked layer, and I don´t want to cut them.

It would be nice if someone could give me a tip, on how to colourcorrect only the edges of an alpha masked layer.

---- FYI ----
If someone is working the first time with Green/Bluescreen here are some Tips
I try to give my best in English :)

Add multiple Keying Effects for more Precision.
Use a neutral Background for Tests to see even a single Pixel changing.
Start with the darkest colour value and invert the Subtraction to control your Result.
End with the brightest an biggest continues canvas.

This might help you to have always the outline of keyed Objects in Focus.

With virtual Regards,
yt

Can you post an example image of your problem? (a still would do)
For example, the selection on a gray background or similar.

Hello Marko,

thank you offering your help.
Here is the latest result of my rendered Image.

http://yours-truly.de/files/Kdenlive/vlcsnap-2012-03-02-15h53m37s170.png

You can see a green edge on the sleeve of the shirt. That is the problem :)
Both are nearly clean keyed. It is a still Image in the back.


Thank you.

yt

Looks like spill surpression is required.

I've downloaded the pic, and pixel-peeked it in Gimp. Must say that the key is quite clean - at least on a still, probably looks worse on a moving image?

By the shirt sleeve, you mean on the left woman, her right side?
The right side woman seems to be keyed quite cleanly.

I see worse keyspill on the chair's arm rests in the middle?

Apart from that, i guess a very slight key softening (alpha blur) would be nice, see for example the chin of the left woman. As far as I know, there's no suitable tool currently in Kdelive.
I'll add that (small blur) to the alpha operations plugin, when I'll find some time (don't hold your breath ;-)

As for the keyspill, I'll think about what could be done. For example Cinderela has an option to reduce the saturation close to the edges.
But that wouldn't help much on the arm rests (tried it out in Gimp).
So maybe "growing" the color and luma outwards near the edges?
Any suggestions or hints?

I also have/had plans to work on a new or improve one of the current greenscreen filters in terms of edge behavior but unfortunately I currently don't have much time.
Marko, you might want to have a look at this paper: http://www.tzi.de/fileadmin/resources/publikationen/tzi_berichte/TZI-Bericht-Nr._40.pdf
Since your select0r already uses the 3D method also employed by the "TZI keyer" from the paper it would be extremely easy to add a softness area. The auto spill removal presented in the paper also shouldn't be a big deal.

Hmm, I have some Ideas how to do an effective spill suppression with Kdenlive. But this will get a bit complicated not at last because of Kdenlive's missing ability to automatically do alpha operations between tracks without transitions.
I will check that out and write a tutorial as soon as I manage to find a good technique ;)

Blender has greenscreen plugin and nodes, could look at that code, RGB & YCC options. There's also a seperate spill supression node, if I get time I can dig out the code for info.

Till,
thanks for the link, I'll see what I can do in that direction.

OK, I've read the TZI paper (and some references from it).

For the "spill mop up", I'll probably write a separate plugin, so it will be usable with any color keyer.

Later, I'll add the softness control to select0r, and maybe the 3D cube and hue histogram scopes.

Also, it would be nice to have some way to work around the DV chroma subsampling. The TZI uses simple blurring, but I would like to do something a bit smarter, like matched to the particular subsampling (420, 411...).
Or even smarter, like cross filters? I think this should also go into a separate plugin, as a polished-up chroma is desirable for other purposes besides keying.

But hey, I am promising too much....

---------------------------------

Checked with pr0file, and Kdenlive/MLT already seems to do some chroma smoothing/interpolation, so I can at least postpone that :-)

I have written a plugin based on the spill suppression method described in the TZI paper above (Till's post), namely de-saturation of the key color.

I haven't pushed it in the GIT, because I am not sure about it's performance. If anybody is interested, I have put the source code and a Kdenlive effect description file here:

http://lea.hamradio.si/~s57uuu/video/keyspillm0pup.c
http://lea.hamradio.si/~s57uuu/video/frei0r_keyspillm0pup.xml

if somebody wants to test it. Compilation instructions are in the comments, then just copy the .so file to /usr/lib/frei0r-1 or wherever you have your plugins. The .xml file goes to ...share/apps/kdenlive/effects or some similar path.

I am testing it on the TZI test videos (http://www.tzi.de/tzikeyer/), and I suggest you use the same, so that we can talk about the same thing. I have a small screen, so I use "pr0be" as a magnifier, to check the edges, and to pick the color.

What I found out is, that while it works quite well on spills that fall on a dark or neutral object, like black hair (clip_1.2.avi, "girl with glass"), it is more or less useless, when the spill falls on a colored or bright object (clip_1.1.avi, "water flow"), because the key color gets mixed with the object color, creating a gradient, which is hard to cover without hurting other things.
Also, when the edge of a bright colorful object is desaturated, you get a gray rim instead of a key spill.

I'll try to find other algorithms, suggestions welcome.

------------------------------

BTW, these spills, if not too wide, can be suppressed with the existing "alpha operations" plugin, operation "shrink soft".

Also, with selector in RGB mode with blue screen (the above test videos) increasing the B tolerance helps on the hair.

Here is a (rather long) way with onboard Kdenlive effects. I know that this may not be the best way, but it is the first idea I had withour taking a look at the blender despill's source code ;)

What we want to do is to extract (in our HSV system) the hue and value into one channel - I call it the color channel from now - and the value channel into an other channel.
Then we copy the color channel (so we have 2 of them), extract the greens in color channel 1, invert the hsv value and then do a subtractive color mixing with color channel 2. Now let's call the result color channel 3. By mixing Color Channel 1 and two we got a color channel where the greens are compensated out and so turned into greys. We prefer this method to simply desaturating the greens because by desaturating the greens we would simply take all pixels that have a defined amount of green and make them grey, but if it is for example a yellowish pixel that represents a red object that is affected by green spill and so it looks yellow, instead of subtracting the green from the yellow to get nearer to the object's original color (that's what we try to do) we would simply make it ... well grey.
Can't subtract all the green components from our picture. We will have to define some kind of a filter for that.

Unfortunately I do not know a way neither to split the HSV channels nor to separate the greens then. So what I want to do is a quick and dirty workaround.

1. Duplicate the Clip in the Timeline three times and place them one below the other on the tracks Video 1 (now 1), Video 2 (2) and Video 3 (3).

2. 2 gets a Greyscale effect. 3. gets in this order:

Bezier Curves
Bezier Curves
Hue Shift
Levels
Saturation.

3. In the first Bezier Curves you select the red channel and pull it completely down. Same with the Blue on Bezier Curves 2.
Now we have done some kind of green-extraction. Now we set the Hue Shift to 180 Degree. Looks sweet now, eh?
Now we have basically the layer whichs color want to subtract from Video 1.
With the Levels and Saturation effect (maybe you want to use an other Curves effect instead. Feel free to do so if it gives you the best results) we can now control the strenght in relation to the intensity of the green (but unfortunately we can not get a control of the Saturation-Gradiation because we are seeing an image that displays a combination of Saturation and Value. But it's okay to work with this for our purpose. Let the levels and Saturation effects as they are for a moment.

4. Now, we have the problem that there is no way to only mix the Color channel of Video 1 and 3. The best way there is (at least I have seen. I'm sure there are more possibilities in Kdenlive) is to mix Video 1 and Video 3 together with an alpha value of 50%. But because Video 3 does only contain the green channel, the overall image gets darker. Because this we have created Video 2 which has a greyscale (value) version of Video 1. Place a transition between Video 2 and 3.
Then place a composite transition between 1 and 2, change "with track" to Video 3 and change the Opacity to 50.
Now change the transition type of the one between 2 and 3 to screen.
Now you should see that the Green turned to grey. The screen transition is not perfect for this purpose. All we want to do is to light up 3 a bit.

5.
Play with the Saturation and Levels of Video 3 and with the Opacity of the Composite transition to get the best results. Maybe add a curves effect to Video 2 to control the brightness. Note: The result will be undersaturated because of the way we are working with our color channels ;) But I think you can get rid of the spill with this method.

The key color is not just one of the primaries, and keyspillm0pup does not work by simply reducing the amount of that primary.
It has its own small color keyer, and is well capable of desaturating greens without affecting yellows.

The problems I encounter are basically different:

1. The key reflection on a bright object usually creates a long color gradient from key to object color, so if you specify a broad enough color subspace, other things will get caught in.

2. The edges are often dark. The color space gets very tight in the dark corner, so again, it is hard to do a good selection.

So what I'll try to do next in keyspillm0pup, is to add an edge detector in the alpha channel, and gate the selection with that.


I don't think the acrobatics described above would help very much. There is only one color space, and RGB, HSV, etc. are just different coordinate systems that we put onto this single color space.
see: http://en.wikipedia.org/wiki/HSL_and_HSV
NOTE: with some of the Hxx systems there will be some metric distortions, because of the simplified formulae (for fast computation), but these amount to a few percent and can be neglected in our case.

It is hard to keep the key colored background evenly illuminated. This causes its brightnees to variate, but (mostly) not the hue. That is the motivation to use Hxx spaces to specify keys, and make the tolerance along the luminance axis wider, while keeping the chroma tolerance tight.
Select0r can do that.
Actually the same could be done by allowing rotation of an RGB subspace, but I guess that would be rather unintuitive for the user.

Some keyers allow complicated selection subspaces, like 128 sided polyhedrons, to precisely adapt to the key variability.
I think select0r is even more versatile than that, with its ability to combine simple shapes. What it currently lacks is a "softness" parameter, to define the width of a transition zone.
I plan to add that in the future, either a single softness parameter, or a separate one for each axis.

However, even the most maleable color selection subspace can not solve the problems in the transition areas, where we have a continous gradient between the key and the foreground object colors. This happens where the key color reflects from the sides of the object, on the edge pixels that are partially exposed to object and partly to background, on transparent and fuzzy objects, and with motion blur.

So in principle, any sophistication in the generation of the alpha channel itself, is not enough for perfect compositing.
In the transition areas, one MUST correct the foreground object color too.

Suggestions and links welcome!


@yellow: if you can dig out that Blender code, I'll be very thankfull!

Here's the old outdated YCbCr greenscreen plugin code for Blenders VSE, out of date now but worked well from memory.

http://sites.google.com/site/paprmh/greenscreen1.3.c?attredirects=0

http://sites.google.com/site/paprmh/greenscreen


For Blender 2.5x RGB color spill node, tarball source link: http://download.blender.org/source/blender-2.62.tar.gz

Then navigate through the rabbit holes to blender-2.62 -> source -> blender -> nodes -> composite -> nodes

Thanks, I'll have a look at it.
I am already studying some other papers I found on the net.