TL:DR: I found an old (external to Krita) lip sync tool and I'm going to hack on it, does anyone have any requests?
Repo: https://github.com/MaskedRetriever/papagayo-1

(This is my first post to this forum, apologies for immediately spamming with a weird topic, but then again this finally motivated me to try the kde forums...)

The Krita animation tool kit is really exciting, but from what I've heard importing audio is presently off the table for the fairly sensible reason that adding a sound system complete with scrubbing abilities and AV-sync would be way complicated and off the main mission of Krita. Hearing this set my mind to an old project I remembered: Papagayo.

Papagayo is a helper tool for dialog that lets you position words over the top of the waveform of an audio file to locate the exact frame on which it occurs. It implements audio scrubbing and some simple phoneme dictionaries to aid the process, but is mostly manual. That said, it's also pretty similar to how traditional animation has always worked, and for less ambitious animations (the stated goal of Krita Animation is "simple animations" as I recall) it's a great way of meditating on the words and syllables.

But the UI has some problems! And export is kind of unfinished! But it's written in Python, the one language I'm anything approaching "good" at, so I started messing with it:
https://github.com/MaskedRetriever/papagayo-1

So far I've added a shell script to install the libraries my Kubuntu box didn't already have (not exactly a full featured setup script!) and tweaked the UI so frames are easier to locate and clicking on one gives you a big bright indicator of which frame you clicked on. I love that feature, it makes finding sound features much more comfortable!

Planned:
* More UI updates to make the interface more convenient, discoverable and configurable. There's no configuration file or settings menu(!) so adding that is a priority.
* Export image strips. Papagayo has a little mouth icon in the corner that shows phonemes as mouth movements, and exporting those would be great as a reference layer if Krita has an import frames option. Also if I can manage, export dialog as a set of highlighted words/phonemes, useful for much the same reason.Edit! I've added this! Krita 3 alpha has image strip import too...
* If I get very very ambitious/lucky with time and markov chains, auto phoneme detection, but nobody hold their breath on that...

Also, I'd like some feedback on specific kinds of export would interest krita developers. Papagayo has a very limited text field export that's handy for certain kinds of 3D software (Blender python scripting to apply phonemes to models especially) but that might be something a Krita plugin could handle as well. There could be other features I hadn't thought of but which might not be too hard to do once I knew the idea was out there...

It's definitely true that if Krita were to get audio strips as layers, most people would have very little use for this tool, but if that doesn't happen, this tool has potential to be a solid compromise. Also, at least for particularly "traditionalist" animation with lots of doting on frames, it makes you really pay attention to what frame everything happens at *before* you animate, which is good for you, and builds character. So basically if anyone was thinking of building hooks into Krita for attaching data concerning sound (that was not, itself, sound) I'd love to hear about that so I can make better exported content for that...