Hey, I am using a quad core system and when I render with kdenlive I get no more than 220% cpu usage, ever. I'm wondering if there is a way to increase the amount of cpu's that the renderer uses..

??

Set the environment variable MLT_AVFORMAT_THREADS=2 or make a copy of a render profile and append to "threads=3" to the options string. This only works for certain codecs like MPEG-4, H.264, MPEG-2, and DV.

I could not get this to work.

I tried exporting MLT_AVFORMAT_THREADS=4 (and tried 3 and 2) on the command line, then launching kdenlive, and render something using a default MPEG-4 12000k single pass profile, and it still used ~2.5 cores.

I tried editing the MPEG-4 render profile and adding 'threads=4' at the end (and tried 3 and 2) and again, ~2.5 cores seem to be used (no change in render time).

I tried both of the above together, and it didn't seem to make any difference.

What am I doing wrong? I am using a core i7 930 with 6GB RAM and an SSD, so I am just trying to get the most out of my shiny new hardware :)

I also tried changing the 'Decoding threads' count in a clip's advanced properties, but this didn't seem to make any difference. Should I see a difference in the seek time, etc (and stuttering) of a 1080p50 AVCHD file by changing this setting? It seems to take forever to seek, is this an ffmpeg problem? I realise I should be using an intermediate format, and I will, but I am just testing at the moment, see what kind of grunt I have.

I am running the latest Ubuntu 10.04 (installed today) with the stock kdenlive (0.7.7.1) but with today's MLT (0.5.5) compiled/installed about 2 hours ago.

Anyone else got anything I can try to increase the thread count/core usage when rendering/decoding please?

Does anyone have any idea about the above please? I'm sorry to bump, but I would really like to see what difference this makes when I utilise my hardware more fully.

TIA

FFmpeg needs to be built with --enable-pthreads. Run 'ffmpeg -version' and see if that is in the configuration flags.

AVCHD decoding can not be parallelized by ffmpeg, so with mpeg-2 or mpeg-4 encoding, which are very fast anyways will not be improved if it constrained by the decoding and video/audio processing. Some gain should be noticeable when encoding H.264. The decoding and processing is serial at the moment; there is an unstable branch in mlt git named parallel-consumer that requires an explicit option, but I do not recommend it at the moment:
http://www.dennedy.org/index.php?option=content&task=view&id=106&Itemid=2

If you press shift-H in top, then you can see each thread.

I just did some testing, and the env var and threads= options are working as expected, but it is hardly noticeable due to the decode/processing thread being pegged at 100%. In doing so, I found a bug that was introducing a superficial overhead and needlessly pegging another thread at 100%. So, with this fix, it would use even much less CPU for you. :-)

If you decide to try out the parallel-consumer branch, then refrain from using effects and add to render the options "buffer=25" and "real_time=-3" to use 3 processing threads with no frame-dropping (a positive value there enables frame-dropping). The gain is more noticeable when it must scale and deinterlace (use a different project resolution than your source material) because decoding must still be serial due to multithreading limitations of ffmpeg. However, with parallel processing it can be decoding in one thread, deinterlacing in another, and scaling in another.

Hi,

Thanks for the info. However, when I look at my ffmpeg install, It already has that switch enabled:

FFmpeg version SVN-r0.5.1-4:0.5.1-1ubuntu1, Copyright (c) 2000-2009 Fabrice Bellard, et al.
configuration: --extra-version=4:0.5.1-1ubuntu1 --prefix=/usr --enable-avfilter --enable-avfilter-lavf --enable-vdpau --enable-bzlib --enable-libgsm --enable-libschroedinger --enable-libspeex --enable-libtheora --enable-libvorbis --enable-pthreads --enable-zlib --disable-stripping --disable-vhook --enable-runtime-cpudetect --enable-gpl --enable-postproc --enable-swscale --enable-x11grab --enable-libdc1394 --enable-shared --disable-static
libavutil 49.15. 0 / 49.15. 0
libavcodec 52.20. 1 / 52.20. 1
libavformat 52.31. 0 / 52.31. 0
libavdevice 52. 1. 0 / 52. 1. 0
libavfilter 0. 4. 0 / 0. 4. 0
libswscale 0. 7. 1 / 0. 7. 1
libpostproc 51. 2. 0 / 51. 2. 0
built on Mar 4 2010 12:41:55, gcc: 4.4.3
FFmpeg SVN-r0.5.1-4:0.5.1-1ubuntu1
libavutil 49.15. 0 / 49.15. 0
libavcodec 52.20. 1 / 52.20. 1
libavformat 52.31. 0 / 52.31. 0
libavdevice 52. 1. 0 / 52. 1. 0
libavfilter 0. 4. 0 / 0. 4. 0
libswscale 0. 7. 1 / 0. 7. 1
libpostproc 51. 2. 0 / 51. 2. 0

I am more interested in rendering performance really, and when I view top with shift-H, I can see there are more ffmpeg threads, just they are all only using ~15-20% cpu... is there anything else I can do to get the most out of my hardware under linux (specifically Ubuntu)?

TIA

Anyone? Here is what I get on thread view in top, when specifying threads=3 in the render profile. Rendering to MPEG-4 12000k with ac3 sound 256k and adding 'threads=3' into the profile...

smokedog@ubuntu:~$ top

top - 01:17:20 up 58 min, 2 users, load average: 2.35, 2.32, 2.21
Tasks: 459 total, 3 running, 456 sleeping, 0 stopped, 0 zombie
Cpu(s): 37.9%us, 0.2%sy, 0.0%ni, 61.9%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Mem: 6118552k total, 6075800k used, 42752k free, 1267420k buffers
Swap: 261112k total, 0k used, 261112k free, 2666748k cached

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
2397 smokedog 20 0 1838m 959m 19m R 99.6 16.1 42:29.60 melt
2398 smokedog 20 0 1838m 959m 19m R 53.8 16.1 20:33.59 melt
2402 smokedog 20 0 1838m 959m 19m S 46.5 16.1 19:04.69 melt
2399 smokedog 20 0 1838m 959m 19m S 43.8 16.1 17:53.19 melt
2401 smokedog 20 0 1838m 959m 19m S 37.8 16.1 16:50.76 melt
2400 smokedog 20 0 1838m 959m 19m S 27.2 16.1 17:13.72 melt
3290 smokedog 20 0 1838m 959m 19m S 2.3 16.1 0:01.99 melt
3291 smokedog 20 0 1838m 959m 19m S 2.0 16.1 0:01.70 melt
3292 smokedog 20 0 1838m 959m 19m S 2.0 16.1 0:01.63 melt
3293 smokedog 20 0 1838m 959m 19m S 1.7 16.1 0:01.72 melt
2042 smokedog 20 0 1680m 525m 37m S 1.3 8.8 2:06.72 kdenlive
2349 root 20 0 15392 1328 748 S 1.3 0.0 0:30.23 mount.ntfs
84 root 20 0 0 0 0 S 0.3 0.0 0:01.14 kswapd0
1243 root 20 0 48300 1440 1036 S 0.3 0.0 0:00.72 atieventsd
3261 smokedog 20 0 19480 1692 1056 R 0.3 0.0 0:00.33 top
3272 smokedog 20 0 496m 68m 28m S 0.3 1.1 0:02.80 firefox-bin
1 root 20 0 23688 1944 1276 S 0.0 0.0 0:01.48 init


Seems there is quite a bit more I should be able to squeeze out here. Is it just down to the render profile I am using?

Looking at it actually - seems pretty much all my RAM is in use... why would that be? I wonder if that is where my bottleneck is instead... there's me thinking 6GB should be plenty... ;)

Seriously though, anyone got any ideas?

You have 6 busy melt threads. I am sorry I can not magically make all of them use 100%. You seem to lack some basic understanding of parallel processing. This is complex software. The things running in parallel have dependencies on other tasks and places where >1 of the same thing can not be running at the same time due to lack of optimization due to placing higher priority on correctness, stability, clarity or other reasons. Many of these limitations exist in FFmpeg. This is what we call non-linear scaling meaning a problem does not speed up by a simple factor of number of CPUs (2x for 2, 3x for 3, etc) or # machines (distributed processing in a cluster). The melt thread running at 99% is holding back the encoding threads because mlt is not internally doing parallel processing, which I already explained is in a development branch.

As for your RAM, melt is using 959 MB resident, which is the most accurate column available in that output. The other RAM is used up by other processes and especially the kernel for caching - mostly disk I/O, which it knows how to free automatically as needed.

This explained my question then searching the forum. My CPU uses almost 100% on one core and various on the others. Totally about roughly 40%. Big thanks to ddennedy for good answers on everything I read in this forum. Have tried kdenlive on and off since version 0.5 something. Now it is becoming very useful. So big thanks to the team.