Inapickle, thank you very much for your explanation! I wondered myself whether lossless would be possible at all with H.264.

On thing that still nags me: after decoding the source frame each pixel in a frame is stored by MLT in YUVA8888 format. That is, each pixel has its own YUV and alpha quadruple. But since the source frame pixels were subsampled as 4:2:0 there has been some (implementation-dependent?) upsampling to fill in the U and V components for those pixels left out during encoding.

Now, during "lossless" encoding there is subsampling again. Even for 4:2:0 subsampling I don't see how downsampling won't be lossy in the sense of introducing differences in comparison to the original source frame?

Inapickle wrote:

kalimerox wrote:...good to know that h264 lossless is really lossless ...

Actually, that's not quite true. Metric quality analyses reveal that when converting YV12 sources to h264 'lossless' , it is lossless with respect to the luma (Y component), but the chroma takes a slight "quality" hit.

Actually, I was incorrect in making that statement. I'll explain why:

If I take a lossless YV12 (UTVideo) transcode of say a 1080/25p HD AVC.mp4 clip and encode it with ffmpeg to x264 using the exact same profile as the KDenLive "lossless" h264 render preset, yes the conversion is lossless, as confirmed by comparative metric analysis of the luma and chroma (U, V) components.

If however I take that same UTVideo YV12 video, load it onto the KdenLive timeline (as a 1080/25p project) and render out with the "lossless" h264 preset (without applying any effects) the resulting encode is lossless with respect to the luma component, but there is a very slight difference in the chroma metrics. Thus, by definition, the "pass-through" conversion is not completely lossless.

But, if I instead take that UTVideo YV12 transcode and convert it to lossless Matroska.mkv (i.e. HuffYuv in YUY2 colorspace) and, in the same manner, load it on the KDenLive timeline and render out to Matroska.mkv again, the "pass-through" is completely lossless. Metric analyses detect absolutely no difference between the input and output Matroska.mkv files.

Now why is that ? Well, if as you say " after decoding the source frame each pixel in a frame is stored by MLT in YUVA8888 format", it's reasonable to suggest that chroma sampling errors, albeit minor, arise at some point in the conversions between YV12 and YUVA-8888. And if that is the case, I guess it becomes a question of perspective as to whether one considers "Lossless" H.264 lossless or "lossy" in the context of processing in KDenLive.

That said, we are talking about very minor arithmetic chroma degradation here, and I think Kalimerox can rest assured that the inclusion of "Lossless H.264" rendered material in his project is unlikely to had a significant negative impact on the quality of the final product.

What would be interesting though would be to see if this chroma degradation also occurs when other "inherently" lossless YV12 formats are used - as further proof that these errors are being introduced by the YV12 <> YUVA-8888 inter-conversion. And therein lies my frustration with the current rendering format options in KDenLive - "Lossless" H264 is the only YV12 format available. I had rather hoped that by now steps would have been taken to incorporate UTVideo in MLT as a render option, not just for YV12, but YUY2 and RGB. A far better choice than old "lossless" juggernauts like FFV1 and HuffYUV (Matroska.mkv) in my opinion. And for RGB output there is nothing other than PNG as Image Sequences - hardly practical as an edit intermediate.

On the "visually lossless" 4:2:2 edit intermediate front of course we've got ProRes (in two flavours no less) and DNxHD (at least as a custom render option), which is fine if you're of that persuasion and don't mind taking a double quality hit on conversion of 8bit YV12 sources to YUY2 and then ultimately back to YV12 for rendering to the usual target formats. But I do think there should be better choices for lossless rendering to YV12 and "good old but still useful" RGB/RGBA.

I guess that kind of voices my opinion in response to Kalimerox's pertinent question:

kalimerox wrote:.... what format would you render lossless to, to not lose quality ? or do you always lose "something"?

What resolution/frame rate were your source clips by the way ? The Sony a7s HAVC.mp4 clip that you sent me (to examine the luma scaling) was 720/100p. Were you editing mixed 1080p and 720p sources?

yes as you said the sources were actually mixed in that project, most of the footage being 1080p with 25fps but also some slowmotion shots with 100fps and 720p...

I guess now that i messed up the project a little bit while upgrading to 15.13 but creating the project in 15.12....

and yes, "automatic" is an option in the latest version i installed.. maybe it jsut detects some file and changes the resolution.. honestly I had such a messy project at the end with imported kdenlive sessions as clips, lossless mp4 clips and source footage that i can hardly trace back where the quality loss comes from .. I ll have to check again more in detail...

kalimerox wrote:I guess now that i messed up the project a little bit while upgrading to 15.13 but creating the project in 15.12....

Seems like you need to get back to a 1080p project somehow. Could you not go back to 15.12 and try and pick up from there, and maybe render out those sessions that you imported as clips, if that's still possible?

There again, you can't expect native 1080p quality from up-scaled 720p sources, and if you find you are stuck in a 720p rut, it might be best just to go with it. I've never put any of my videos up on a projector screen, but 720/30p still looks pretty look on an HDTV. And I can't see that there would be anything to be gained by going lower than CRF 15 anyway. What exactly is it about the quality that disappoints you - loss of definition, artifacts, motion characteristics?

kalimerox wrote:and yes, "automatic" is an option in the latest version i installed.. maybe it jsut detects some file and changes the resolution..

Interesting. Does it maybe make a choice on the basis of the first clip on the timeline ? I've used NLE's that have a similar feature, largely for convenience, but it didn't prevent you from going back into the project settings and making changes.

Anyhow, good luck sorting it out.

If I remember correctly, the only change in the file format between 15.12 and within 15.13 became necessary for the new animated keyframe format. Otherwise, there was no difference between 15.12 and 15.13 in the file format.

Inapickle wrote:What would be interesting though would be to see if this chroma degradation also occurs when other "inherently" lossless YV12 formats are used - as further proof that these errors are being introduced by the YV12 <> YUVA-8888 inter-conversion

Just discovered that FFV1 can be configured as a custom command to encode in YV12 colorspace by changing the pixel format to yuv420p:
https://github.com/mltframework/mlt/blo ... sless/FFV1

Repeating the above tests with "FFV1-YV12" as the render format, the results of the comparative metric tests were exactly (sic) the same as those with "lossless" H264 - that is:

Inapickle wrote:If I take that same UTVideo YV12 video, load it onto the KdenLive timeline (as a 1080/25p project) and render out with the "lossless" h264 preset (without applying any effects) the resulting encode is lossless with respect to the luma component, but there is a very slight difference in the chroma metrics. Thus, by definition, the "pass-through" conversion is not completely lossless.

That, to me, is conclusive proof that these chroma sampling errors are being introduced by the YV12 <> YUVA-8888 inter-conversions.

That said, I wouldn't consider FFV1 a practical option as an edit intermediate in YV12 or YUY2 colorspace.

Good research, thank you very much for checking our suspicions! Inquiring minds want to know...

For me, even lossy encoding is most often suitable for me, so lossy lossless will be more than good for me too. I have one case where I even would be more satisfied if I could mimic color bleeding even more, as it's a pun on an 80's TV series with a terrible intro.

Oh I'm not knocking DNxHD and ProRes at all. I did quite an intensive survey of "edit intermediate" formats a while back and they are top notch. It's just that for my present way of working I need a lossless YV12 intra-frame format that is well supported in the ffmpeg and vfw environments and UTVideo well fits that bill. HuffYUV and FFV1 are just too sluggish for my purposes; UTVideo is fast (at least for 8-bit processing) and has excellent hyper-threading support. I just find it a bit frustrating that I am able to load UTVideo files into KDenLive without any problem, but cannot render out the same, and what formats available for lossless YV12 output ("Lossless" H.264, "custom" FFV1-YV12) are just not practical options, for me. Same goes for lossless RGB/RGB32 formats.

The finding that YV12 does suffer a teeny bit of chroma degradation on "pass-through" doesn't really bother me. Of greater concern is an issue that came to light in the course of the thread studying the behavior of different input and output formats with respect to luma scaling:

viewtopic.php?f=272&t=130641&start=45#p352057

Specifically, that when native HD-AVC clips from cameras recording with full range (0-255) luma are processed in KDenlIve, they are compressed to 16-235 range and there is no option at rendering to "recover" the original 0-255 scaling. Now that to me constitutes considerable "loss". And the only way around it is to apply the various techniques, that I went on to describe, to the rendered outputs for scaling back to full range, which for the casual user might be, to say the least, daunting, and assuming they are even aware that is what's happened to their video and why. To my mind, that's something that needs to be addressed.

aw and here it comes the next problem: i did as you suggested and downgraded to 15.12, when i then try to open a kdenlive session I get an error message saying: this project type is unsupported (0.9.3) so i think i m stuck with the newest version ;( so it looks like a lot more things have changed to 15.13.and it falsely detects a 15.12 session as an unsupported 0.9.8 session ;(

Kdenlive doesn't support project downgrade. As you had installed a development version, 15.13 in this case, there are new features that are not available yet in the stable version, which is 15.12. This new feature in development is the new animated keyframe stuff that required a change in the project storage format. Even if you tinker around with the project version in the XML you'll loose the keyframes.

You should have still the project file backup that Kdenlive automatically creates when it needs to upgrade a project file. In addition it has informed you that your project had been upgraded which you had to acknowledge in order to work with your project.

Edit: I see TheDiveO already replied before me.

Yikes, can't help you with that one. But you can still open the project in 15.13, right? Assuming yes, I tell what I'd do before anything else and that's to render out to one of the lossless/visually lossless format's so that you've got a high quality master/archive there as a backup, just in case.

I don't know how fast your PC is, but "Lossless" h264 would likely take light years for an hour of video, and with Matroska.mkv (HuffYuv), the file size would be gigantic. So, as a best compromise, maybe either:

ProRes (prores+pcm_s16le) - the default preset is 'Normal' quality, but you could create a custom preset for High Quality (HQ):
https://github.com/mltframework/mlt/blo ... ess/ProRes
Change the vprofile=2 to vprofile=3 for HQ

or else DNxHD, but again you'd need to set-up a custom preset.
https://www.luhrmann.nl/blogs/techniek/ ... r-profiles
So for a 720/25p.mov profile, the custom profile parameters would be:

f=mov
mlt_profile=atsc_720p_25
vb=90M
vcodec=dnxhd
acodec=pcm_s16le
ar=48k
ac=2

Not sure about ProRes, but with DNxHD (90Mbps) you'd be looking at a file size around 34-35GB.
Just a suggestion.
Edit: ProRes HQ should be around the same.

thanks guys , true!! kdenlive made the backupfiles in the old session format! i ll check that when i m back on my studio computer..!

As soon as i have a nice rendering I ll let you guys know, i m travelling at the moment so I m a bit off the grid with internet and away from my desktop computer..

That's good to know. I guess the heat's off then.

Inapickle wrote:And therein lies my frustration with the current rendering format options in KDenLive - "Lossless" H264 is the only YV12 format available. I had rather hoped that by now steps would have been taken to incorporate UTVideo in MLT as a render option, not just for YV12, but YUY2 and RGB. A far better choice than old "lossless" juggernauts like FFV1 and HuffYUV (Matroska.mkv) in my opinion. And for RGB output there is nothing other than PNG as Image Sequences - hardly practical as an edit intermediate.

Whoopee !! After some tinkering I've discovered it is possible to render out to lossless UTVideo after all. Here are some custom profiles. I prefer PCM audio to FLAC for exchange compatibility between the ffmpeg and vfw environments:

YV12 (4:2:0) 8-bit, BT.709

Code: Select all

f=matroska
vcodec=utvideo
pix_fmt=yuv420p
colorspace=709
acodec=pcm_s16le ar=48k ac=2

YUY2 (4:2:2) 8-bit, BT.709

Code: Select all

f=matroska
vcodec=utvideo
pix_fmt=yuv422p
colorspace=709
acodec=pcm_s16le ar=48k ac=2

RGB24:

Code: Select all

f=matroska
vcodec=utvideo
pix_fmt=rgb24
acodec=pcm_s16le ar=48k ac=2

RGBA:

Code: Select all

f=matroska
vcodec=utvideo
pix_fmt=rgba
acodec=pcm_s16le ar=48k ac=2

Another parameter that can be specified is the prediction mode used for compression:

Code: Select all

pred=left

Which is the default, if not specified. Bias to faster compression (encode) rate.

Code: Select all

pred=med 

Bias to higher compression ratio = smaller files.

The encodes load and edit perfectly well in KDenLive. They won't play in Dragon or VLC media player, but will play in MPV player.

Suits me Sir !

Quick test comparing rendering times and file sizes: