This forum has been archived. All content is frozen. Please use KDE Discuss instead.

Working with many GB of data in one project?

Tags: None
(comma "," separated)
Chamo
Registered Member
Posts
27
Karma
0
Hi,

I'm not sure if this is the place to ask for such tips and tricks, but...

How do other users handle projects with lots of HD clips? Like for example all the material from one holiday, maybe 5 hours of HD raw material that you want to cut down to one hour? If I load 4 GB in 228 file into kdenlive at once, my system stays usable. Not bad, so far. I can use folders for the clips inside kdenlive so that I don't get too confused. The timeline, on the other hand, can't be segmented or structured, like in chapters for example. Daydream: "Minimize this chapter" in the timeline, or even "unload from RAM" without deleting it from the project. Speaking about RAM: I'm not sure how much kdenlive can load on my less than average hardware. I tried 6,6 GB in 465 files. No fun, kdenlive needs half an hour to load them. 9 GB of data in swap space. Playback (project or clip monitor) not in realtime anymore. Everything I try to do is very slow. I guess I won't try that again.

Any suggestions?

TIA

Chamo

waynevid
Registered Member
Posts
118
Karma
0
What version of Kdenlive do you have?

Waynevid


KoRnholio8
Registered Member
Posts
88
Karma
0
Turf off making of thumbnails waveforms of audio if you don't need them. That takes a lot of time, especially with AVCHD.


Chamo
Registered Member
Posts
27
Karma
0
Hi,

I'm using kdenlive 0.7.7 with mlt 0.5.4. Video output is via VDPAU in mlt.
Thanks for the hint, KoRnholio8. I tried it out and it seems to help a little. The 6,6 GB in 465 files now loaded in 10 minutes, and responsiveness is better. The playback in the project or clip monitor is still not smooth, though.
I think the limit is somewhere with the loading of the clips into the project tree and with creating the thumbnails there.
Hm, what does "Maximum number of clients reached" on the command line mean?

I guess I'll just go on with "only" 2,7 GB in 141 files. It's not that bad. ;-)

Chamo
lordofthestrings
Registered Member
Posts
95
Karma
0
I'm sorry if I go off topic, how do you get output via VDPAU in MLT? Does it means that you have playback/monitor with GPU acceleration in Kdenlive? How can I test it for myself?
I'm running Ubuntu Lucid 64-bit, Kdenlive 0.7.7, I have an NVIDIA 8500GT, default ffmpeg with VDPAU enabled and SMPLAYER using VDPAU correctly leaving CPU usage around 5% even with 1920x1080 videos. AMD Athlon 64 X2 6000+.
I never needed extra power in Kdenlive working with SD miniDV but now I'm waiting for a Canon HV10 (tape HDV) I got from ebay and I already know It will be harder when I'll start adding Frei0r plugins ecc. ...
Thank you,
Ignazio
KoRnholio8
Registered Member
Posts
88
Karma
0
I just finished working on a project with AVCHD files decoded to DHxHD with total size of the clips at about 50gb. I tried setting up VDPAU, but failed. The project however did load in approximately 10-15 seconds from a USB 2 external drive. I'd say your main problems would be AVCHD, VDPAU and the sheer number of small files.


Chamo
Registered Member
Posts
27
Karma
0
Hi,

so the size of the project (the Gigabytes of the clip material) is not an issue in kdenlive. The number of AVCHD clips is. That seems to be related to the "Maximum number of clients reached" message. But what clients, and how is this number defined? Could I increase this number?

About VDPAU support and how to get it: Please see the other thread. That's about all I know about this.
http://www.kdenlive.org/forum/recommended-way-working-avchd-content#comment-9297

Chamo
lordofthestrings
Registered Member
Posts
95
Karma
0
Thank you. I 've read the other thread you linked. That's probably why you have such great performances with AVCHD on a single core machine. I use Ubuntu and installed everything through the package manager. I have 8500GT and an Athlon dual core 3GHz, so, maybe, if I was able to compile everything by myself I could reach even better results... I don't know how to do, I'm frightened to damage my system because I'm a (3 years...) newbie...
Anyway, that's really amazing!
Ignazio
ddennedy
Registered Member
Posts
1315
Karma
1
to see if you have vdpau working, in a terminal run:
$ melt some.mts -debug 2>&1 | grep vdpau

and see if you get a bunch of vdpau_get_buffer and vdpau_release_buffer messages.

There are still major performance problems with seeking on AVCHD. Also, playback is not low cpu as mplayer since Kdenlive must pull the uncompressed images from video memory back into main memory for additional processing. mplayer can just keep them in video memory and tell vdpau when to show them.



lordofthestrings
Registered Member
Posts
95
Karma
0
Thanks for the tip, I tested it!
The terminal gave no messages at all. The video opened up in a window and played but no message in the terminal :-(
lordofthestrings
Registered Member
Posts
95
Karma
0
I uninstalled Kdenlive from the repos and I followed the instruction on this site and compiled MLT and Kdenlive by myself and ffmpeg with this script: http://www.webupd8.org/2010/04/script-to-automatically-download-and.html
I have everything up and running but no VDPAU in MLT (which I also read should be automatically detected)....
This is a killer feature that could help me staying with my dual core for other two years and Kdenlive is my favorite video editor!
ddennedy
Registered Member
Posts
1315
Karma
1
See if it is enabled in FFmpeg. When you run 'ffmpeg -codecs' it should show a h264_vdpau. If not, then check to see if you have libvdpau-dev installed by looking for /usr/include/vdpau/vdpau.h or "locate 'vdpau/vdpau.h'"


lordofthestrings
Registered Member
Posts
95
Karma
0
I'm pretty sure I've installed libvdpau-dev from synaptic and I know that smplayer+vdpau is still working good but when I'll come back home (now I'm at work) I'll check FFmpeg (since I didn't compile it manually) and I'll look for vdpau.h.
Thanks, I'll let you know.
Ignazio
lordofthestrings
Registered Member
Posts
95
Karma
0
vdpau.h is there :-)

And that's what returned the command ffmpeg -codecs:

Image
Chamo
Registered Member
Posts
27
Karma
0
Hi,

the option "codecs" seems to be unknown to ffmpeg. I guess Dan meant "formats":

ffmpeg -formats | grep vdpau

should give you something like

FFmpeg version SVN-r20373, Copyright (c) 2000-2009 Fabrice Bellard, et al.
built on Feb 27 2010 22:27:14 with gcc 4.3.4
configuration: --prefix=/usr --libdir=/usr/lib64 --shlibdir=/usr/lib64 --mandir=/usr/share/man --enable-static --enable-shared --cc=x86_64-pc-linux-gnu-gcc --disable-debug --disable-network --enable-libmp3lame --enable-libvorbis --enable-libx264 --enable-libxvid --disable-indev=v4l --disable-indev=v4l2 --disable-indev=oss --disable-indev=jack --disable-outdev=oss --enable-x11grab --disable-altivec --cpu=athlon64 --enable-gpl --enable-version3 --enable-postproc --enable-avfilter --enable-avfilter-lavf --disable-stripping --enable-hardcoded-tables
libavutil 50. 3. 0 / 50. 3. 0
libavcodec 52.37. 1 / 52.37. 1
libavformat 52.39. 2 / 52.39. 2
libavdevice 52. 2. 0 / 52. 2. 0
libavfilter 1. 4. 1 / 1. 4. 1
libswscale 0. 7. 1 / 0. 7. 1
libpostproc 51. 2. 0 / 51. 2. 0
D V D h264_vdpau H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10 (VDPAU acceleration)
D V DT mpeg1video_vdpau MPEG-1 video (VDPAU acceleration)
D V DT mpegvideo_vdpau MPEG-1/2 video (VDPAU acceleration)
D V D vc1_vdpau SMPTE VC-1 VDPAU
D V D wmv3_vdpau Windows Media Video 9 VDPAU

Cheers

Chamo


Bookmarks



Who is online

Registered users: Bing [Bot], Google [Bot], Yahoo [Bot]