I find that if I'm moving/copying files and generally cleaning
up my directory tree, after a while strigi and nepomuk and
virtuoso-t go crazy, consuming vast amounts of memory and
pegging the cpu at 100% seemingly forever (I've always had
to turn indexing off so I don't know if it would eventually
settle down.

Would it be possible to have a setting that limits strigi to
X% of cpu (as in the boinc app)? Speed is not of the essence
here.

I'm trying a little program for linux called cpulimit (easy to find
with google). This program limits any process (e.g. virtuoso-t)
to a specified max cpu useage. So far, it seems to be working
OK.

As anyone else tried this. Does it really work?

Also, I am indexing a directory tree whose total size is 8 gb,
plus I guess strigi indexes mail whether I tell it to or not (is
that right?).

But the index size is 1.8 gb. Doesn't that seem a bit large.
The index is some 20% of the size of the files indexed!!
Is that normal??

any comments appreciated ...

ability to limit cpu for Strigi would be a good BrainStorm idea, as would changing the default niceness if it is not already defaulted to a low priority.

ability to limit cpu for Strigi would be a good BrainStorm idea, as would changing the default niceness if it is not already defaulted to a low priority.

adjusting niceness is good (but I assume that's done already - yes
I just checked and virtuoso-t is niced to 19).

cpulimit issues sigstop and sigcont signals ... doesn't affect
the scheduler

If the CPU usage of virtuoso-t/Nepomuk does not settle itself after some time, I would suggest filing a bug report at bugs.kde.org, as it should not need too long to update the index to reflect the removal or moving of files.

The closest topic for my problem.
Time of indexing is extremely high. After around 15 hours, Strigi indexed only 22k of file, and I have more then 90k. And it seem that Strigi is trying to index same file several times, when it's definitely not changing (I disabled all torrent is case it affecting some way, it didn't help). I could understand it on big files, like videos, but songs? Is it a bug or just the way it's working? I would like to give it a shot, but I just can't =(

Check your Strigi release, and ask for an update.

If you run Strigi 0.7.7, the only files that get constant reindexings are FLACs and some MP3 (if you have FFMPEG enabled... almost all distros ship Strigi without it). Ask your distro for it, or install a distro having it (like Chakra Linux).

The issue of Strigi indexing all the time the same files was already fixed.

Well, I run 0.7.7 and ffmpeg is disabled, but I still got that multiple indexing of various files — from mp3's to iso's. Maybe notifier got it all wrong, but it's confusing.

Index all your home is not a good idea. Try to limit the number of folders strigi must index and consider that indexers are very slow with large multimedia files (1 or 2 minutes per file).

Try to not index folders with high file movement and changes, like your downloads folders, the place when you transcoding videos, your build folders, etc...

If you index only things that matters your database and performance will be better and, if you need index more places, you can add more folders lately.

Nepomuk tries to have synchronized your files with your database and this has a extremely high cost because, when you reorganized so many files, nepomuk tries to synchronize all your changes as fast as possible.

My database has over 30k files and I have few performance issues.

Thanks for the advice, I didn't think about that this way. I just don't need index my music or videos, just photos and documents.