Reply to topic

[solved] Backup Nepomuk data

User avatar Ignacio Serantes
Registered Member
Posts
448
Karma
1
OS

[solved] Backup Nepomuk data

Wed Nov 26, 2008 4:34 pm
Hello!

I currently using KDE 4.1.3 over an OpenSuse linux 11.

My problem is that my sesame2 Nepomuk database is out of control and it's size is increased day by day (currently 215 MB).

I want erase it and begin again but I have a some personal information introduced with Dolphin, tags, scores and some comments, and I don't want to loose that.

I playing a little with sopranocmd, currently I know how to backup and restore the database but I don't know how to backup only my data and not the data automatically added by Nepomuk and strigi. I googled but I don't found a howto or script to doing this job.

By the way, is there some option to limit search in strigi & Nepomuk?
I configure strigi since beginning in KDE configuration panel, mail and music directories only, but I look in sesame2 database and I found files from all my system :(. Now I have strigi deactivated but I don't detect variations, sesame2 database grows every day and sometimes CPU is 100% because some nepomukservicestub process.

I think that Nepomuk integration in KDE is really amazing and it's full of good promises but, currently, using it is problematic :).

Thanks in advance!

Last edited by bcooksley on Mon Jan 12, 2009 5:50 am, edited 1 time in total.


Ignacio Serantes, proud to be a member of KDE forums since 2008-Nov.
User avatar bcooksley
Administrator
Posts
18641
Karma
83
OS

RE: Backup Dolphin Nepomuk data

Thu Nov 27, 2008 10:23 am
This problem is caused by Strigi indexing your computer. I would recommend you disable Strigi, despite the fact you have Sesame2, because nothing can really actually make use of the indexing it performs, and you can still use the Nepomuk tagging without it. This issue has been resolved for KDE 4.2.


System Settings and Device Actions KCM maintainer
Image
User avatar Ignacio Serantes
Registered Member
Posts
448
Karma
1
OS

RE: Backup Dolphin Nepomuk data

Thu Nov 27, 2008 4:01 pm
Thanks, I will try KDE 4.2 Beta 1.

On the other side, I found a dirty trick to backup my data using sopranocmd and grep. Apparently, all my data begins with " nedb.backup

rm nedb.tmp
[/code]

Now, database located in "~/.kde4/share/apps/nepomuk/repository/main", can be deleted after stopping nepomuk without loosing personal data.

The last steps are restart nepomuk and restore the backup with:
Code: Select all
sopranocmd --dbus org.kde.NepomukServer --model main import nedb.backup


After this job, my database size is 2.1 MB, a much reasonable size than 215 MB, and Dolphin and nepomuk works faster.

I hope this information will be useful to others changing "/home/ignacio/media".


Ignacio Serantes, proud to be a member of KDE forums since 2008-Nov.
User avatar Tomaz
Registered Member
Posts
86
Karma
0

RE: Backup Dolphin Nepomuk data

Thu Nov 27, 2008 4:52 pm
very good to know indeed. I was suffering from that as well.


Rocs developer. (and no, i'm not proud of it) ;D
DanielW
KDE Developer
Posts
71
Karma
0
OS

RE: Backup Dolphin Nepomuk data

Thu Nov 27, 2008 5:03 pm
Yes, there should be some Backup tool.

The sopranocmd trick is nice, but in the long run no real help. (And it doesn't really solve the problem. If you have own ratings/tags all over the place you not use it to delete the Strigi data without losing your data).

And, well you can disable Strigis indexing (at the moment there is nothing which makes use of the data) for the moment. Or select only some folders for indexing. Too bad, that it doesn't delete things from Index when you deselect some folders later.


DanielW, proud to be a member of KDE forums since 2008-Oct.
User avatar Ignacio Serantes
Registered Member
Posts
448
Karma
1
OS

RE: Backup Dolphin Nepomuk data

Thu Nov 27, 2008 10:12 pm
DanielW wrote:(And it doesn't really solve the problem. If you have own ratings/tags all over the place you not use it to delete the Strigi data without losing your data).


Well, I first try SPARQL but I don't understand very well (SQL I miss you) and grep works for me because I only tagged video and all are in the same location :).

When I learn SPARQL I will try a script with a better solution.


Ignacio Serantes, proud to be a member of KDE forums since 2008-Nov.
DanielW
KDE Developer
Posts
71
Karma
0
OS

RE: Backup Dolphin Nepomuk data

Fri Nov 28, 2008 7:35 pm
OK, i wrote a better script which really deletes all Strigi data and only that.

It also uses grep, so well it is not perfect (a small C++ prog for it would be better, but had no time for that, and you would have to compile it to use it)

Code: Select all
#!/bin/bash                                               

# Script deletes all Data coming from strigi out of the nepomuk store
# it does no check if one step failed                               
# DanielW dw@danielwinter.de                                         

echo -n  "Exporting Nepomuk Data... "
sopranocmd --socket ~/.kde4/share/apps/nepomuk/socket --model main export nedb.tmp &>/dev/null
echo "Done"                                                                                   

echo -n "stopping Nepomuk... "
kquitapp NepomukServer       
echo "Done"                   

echo -n "Moving Nepomuk Data to ~/.kde4/share/apps/nepomuk/repository.backup... "
mv ~/.kde4/share/apps/nepomuk/repository ~/.kde4/share/apps/nepomuk/repository.backup
echo "Done"

echo -n ""out"greping Strigi data... "
grep  -vE " .$" nedb.tmp >nedbwithoutstrigi.tmp
echo "Done"

echo -n "starting Nepomuk... "
nepomukserver &>/dev/null
sleep 20
echo "Done"

echo -n "importing Data into nepomuk... "
sopranocmd --socket ~/.kde4/share/apps/nepomuk/socket --model main import nedbwithoutstrigi.tmp &>/dev/null
echo "Done"

echo -n "Cleaning up... "
rm nedbwithoutstrigi.tmp nedb.tmp

echo  "Done"

echo
echo "All done, if all went well you can now delete the Backuped data in ~/.kde4/share/apps/nepomuk/repository.backup"


If you want to make the things manualy: it works the same as the OPs one with two changes:

It uses --socket for sopranocmd because that is faster.

And it uses this changed grep command:

grep -vE " .$" nedb.tmp >nedbwithoutstrigi.tmp

Last edited by DanielW on Fri Nov 28, 2008 7:39 pm, edited 1 time in total.


DanielW, proud to be a member of KDE forums since 2008-Oct.
User avatar Ignacio Serantes
Registered Member
Posts
448
Karma
1
OS
Well, I can see that socket is fast and dbus very slow.

Thanks for the tip and for the script.


Ignacio Serantes, proud to be a member of KDE forums since 2008-Nov.
User avatar bcooksley
Administrator
Posts
18641
Karma
83
OS
Stuck, as with KDE 4.2 Nepomuk will be storing peoples search index and will be equipped with frontend(s) to expose this functionality. Therefore they may wish to be able to backup their Nepomuk tags / etc, without the search index.


System Settings and Device Actions KCM maintainer
Image
iamsorandom
Registered Member
Posts
32
Karma
0

Re: [solved] Backup Nepomuk data

Sat Jun 22, 2013 7:56 pm
Hi, I noticed this was a sticky - I backup my files regularly ( the whole home folder, with deja-dup) but is it recommended to backup the nepomuk database separately?? Is there a particular risk of losing tags etc.?
User avatar bcooksley
Administrator
Posts
18641
Karma
83
OS

Re: [solved] Backup Nepomuk data

Tue Jun 25, 2013 9:01 am
Nepomuk itself won't "lose" your tags, but if your system crashes then the database could possibly be lost - which would include the tags on all of your files. I'm not sure how good a backup of the Nepomuk Virtuoso database would be while it is running though - so running the separate backup program every so often might be advised.


System Settings and Device Actions KCM maintainer
Image
iamsorandom
Registered Member
Posts
32
Karma
0

Re: [solved] Backup Nepomuk data

Tue Jun 25, 2013 1:43 pm
Ok - thanks! would be good if this could be automated to ensure beginner users don't forget...
User avatar bcooksley
Administrator
Posts
18641
Karma
83
OS

Re: [solved] Backup Nepomuk data

Wed Jun 26, 2013 9:43 am
Indeed, automation of backups would probably be a good idea (as long as they don't use up all the disk space of course).
You might want to file a feature request in the Brainstorm about that, to see what others think.


System Settings and Device Actions KCM maintainer
Image

 
Reply to topic

Bookmarks



Who is online

Registered users: Andreas Mair, Baidu [Spider], Bing [Bot], capslock, cloose, edmael, Exabot [Bot], Google [Bot], GreatEmerald, HmpfCBR, johngates, koriun, Majestic-12 [Bot], piedro, psbot [Picsearch], sledz, Sogou [Bot], TheraHedwig, tparrott, Yahoo [Bot]