Hi everyone,

I want to share an idea, that could add a very powerful tool to kdenlive.
When working with video projects, I encountered the problem to have some repetitive tasks, that I have to work on manually.
And in times of python scripts, it would not be difficult to automate these tasks.

So my idea is:
pyvideo-tracks in the multi-track viewer.

Next to video-tracks, and audio-tracks there could be a third option for pyvideo-tracks.

These new tracks are not for inserting video or audio snippets, but when you click on them, a text box occurs and you can write python code that determines the video sequence.
By this method, every frame can be calculated individual and with certain conditions.

To be more concrete:
The script has as input a single frame, converts it to a numpy array, then operations can be executed, and a numpy array with the new image is returned and set to the screen.
As there is only one frame, the script takes also only one time point, for example the current timestamp /the vertical line in the multi-track-view.
When rendering the video, the script can then be called many times for every single frame in th video.

Here are four examples, where this could be very useful:

1. Intros
Let's say, in every video project, there should be a starting logo form second 1 to second 3. So there is a condition, that asks for the time of the videoframe. If it is fulfilled, the logo is set to the video, otherwise nothing is set.

Code: Select all

from imageio import imread

def generate_frame(video):
    image = imread("logo.png")
    if  1 < video.time < 3:
        return image
    else:
        return None


2. Image Filtering
Automated burring of a video from track v1

Code: Select all

import cv2
def generate_frame(video):
    image = video.v1.frame
    if type(image) != None:
        cv2.medianBlur(image,5)
        return image
    else:
        return None


3. Insert png file when speed of track is higher
Let say the video runs faster in a certain section, and you want to indicate this faster speed with a png file of a speed icon:

Code: Select all

from imageio import imread
def generate_frame(video):
    image = imread("speed_icon.png")
    if  video.v1.track_speed == 200:
        return image
    else:
        return None


4. Insert image, when audio is higher
You want to see a "clap icon", when the sound is over a threshold?

Code: Select all

from imageio import imread
def generate_frame(video):
    image = imread("clap_icon.png")
    if  video.a1.volume > 0.7:
        return image
    else:
        return None


And as soon as there is a programming interface, creativity is unlimited.
What just comes in my mind are aruco detection for virtual reality that can added to videos, very adjustable custom transitions, and conditions for filters that are just not achievable with an graphical interface.
Or it could be even possible, to add small animated scenes into videos, for example with manim, a python library that 3blue1brown
uses for his videos.
I am curious about your opinions.

i was also thinking about creating some sort of scripting-language.. this to make users able to create effects and make the Kdenlive project grown by itself (by community)

initially i thought about an effect that will extract the sequence of frames(or a single frame oen by one) and send everything to an external application... and wait for the frames to came back... but i've no idea on how the frame can be handled (probably as an image on clipboard?) ..and how the external application can send data to kdenlive to communicate?

BTW this is a long/complicated job and if in future something like that will be created will be necessary to think a lot about functioning mechanisms ...