Reply to topic

virtuoso -t

User avatar wakou222
Registered Member
Posts
14
Karma
0
OS

virtuoso -t

Mon Sep 09, 2013 2:05 am
I got a warning that my /home was full. I found that my soprano-virtuoso.db was 20Gb of a 65Gb /home dir. I deleted it, but it came back, and is now at 30Gb. If I do a htop I see a number of processes of the form:
Code: Select all
4791 ?        SNl  248:40 /usr/bin/virtuoso-t +foreground +configfile /tmp/virtuoso_Cj4345.ini +wait


Nepomuk is only set to index /home (not hidden files) and a 100Gb media store containing music and video (I have given up with kmail as it was broken) What is causing this db to bloat uncontrollably? Why are the instances of virtuoso -t hogging resources and hanging around for so long?

(The above had been running for 4 hours)

How do I stop this happening?
Is there an alternative to the whole akonadi/strigi/nepomuk/virtuoso mess? ie a simple file indexer that can be relied upon?

I use opensuse 12.3 with KDE 4.11
Image


openSUSE 12.3(kernel 3.11.xxxx/KDE 4.11/nVidia 6150se IGP/nVidia driver 304.108/AMD Athlon 64 X2 Dual Core Processor 5600/4Gb DDR2 RAM
User avatar bcooksley
Administrator
Posts
19763
Karma
87
OS

Re: virtuoso -t

Mon Sep 09, 2013 7:22 am
To clear this large database, run the following commands, one after the other:
Code: Select all
qdbus org.kde.NepomukServer /nepomukserver quit
sleep 60s
rm -rf $(kde4-config --localprefix)/share/apps/nepomuk/repository


With regards to htop - do you have it set to display the threads of user processes? If so, try setting it to hide them. If this reveals more than one instance of virtuoso-t for a single user, this is likely the cause of the issue as only one process should be running.


KDE Sysadmin
[img]http://forum.kde.org/content/bcooksley_sig.png[/img]
User avatar wakou222
Registered Member
Posts
14
Karma
0
OS

Re: virtuoso -t

Mon Sep 09, 2013 9:22 am
Thank you, I will delete that directory, using your instructions,





With regards to htop - do you have it set to display the threads of user processes? If so, try setting it to hide them. If this reveals more than one instance of virtuoso-t for a single user, this is likely the cause of the issue as only one process should be running.


I only set HTOP to that view to show that there was more than one instance.... (If I understand you correctly)


openSUSE 12.3(kernel 3.11.xxxx/KDE 4.11/nVidia 6150se IGP/nVidia driver 304.108/AMD Athlon 64 X2 Dual Core Processor 5600/4Gb DDR2 RAM
User avatar wakou222
Registered Member
Posts
14
Karma
0
OS

Re: virtuoso -t

Mon Sep 09, 2013 11:12 am
Just a FYI, I deleted the db and the dir in which it resides, as above, at approx !0:30.
It has gradually grown since then, and is now 740Mb. (12:00)

Also this morning I did a zypper up, and some virtuoso/nepomuk etc components were upgraded, so I am going to reboot and hope that (!) something has improved.

I am not sure that I understood bcooksley 's point re. htop, but here is another....

Image

edit/ps soprano-virtuoso.db now at 826Mb, grown that much whilst I was writing this...


openSUSE 12.3(kernel 3.11.xxxx/KDE 4.11/nVidia 6150se IGP/nVidia driver 304.108/AMD Athlon 64 X2 Dual Core Processor 5600/4Gb DDR2 RAM
User avatar wakou222
Registered Member
Posts
14
Karma
0
OS

Re: virtuoso -t

Mon Sep 09, 2013 1:51 pm
..... now at 2.3Gb and growing...


openSUSE 12.3(kernel 3.11.xxxx/KDE 4.11/nVidia 6150se IGP/nVidia driver 304.108/AMD Athlon 64 X2 Dual Core Processor 5600/4Gb DDR2 RAM
User avatar wakou222
Registered Member
Posts
14
Karma
0
OS

Re: virtuoso -t

Tue Sep 10, 2013 8:11 am
Sigh, now 20 or so hours later and the .db is 15.5Gb. I shall delete it again, but is there really no way to find out what it is doing or what is making it choke?


openSUSE 12.3(kernel 3.11.xxxx/KDE 4.11/nVidia 6150se IGP/nVidia driver 304.108/AMD Athlon 64 X2 Dual Core Processor 5600/4Gb DDR2 RAM
User avatar bcooksley
Administrator
Posts
19763
Karma
87
OS

Re: virtuoso -t

Tue Sep 10, 2013 9:13 am
I would suggest disabling file indexing in your case. If you have any source code trees, you may wish to exclude them from indexing if you need to keep file indexing enabled.

After disabling (or reconfiguring) file indexing, you will need to wipe the database again as I previously described.

If this does not alleviate the issue, I strongly suspect that Nepomuk is somehow starting twice on your system - particularly as you mention you have it set to hide threads yet have multiple copies of virtuoso-t and nepomukstorage running.


KDE Sysadmin
[img]http://forum.kde.org/content/bcooksley_sig.png[/img]
User avatar wakou222
Registered Member
Posts
14
Karma
0
OS

Re: virtuoso -t

Fri Sep 13, 2013 9:15 am
So I switched off file indexing via desktop configuration, and deleted everything under ~/.kde4/share/apps/nepomuk/repository. Everything is now behaving nicely with the .db sitting at 10Mb, But then it would (!)
But of course I do not now have a convenient search tool! Are there any alternatives? I am not sure that I have or will ever have need of an 'ontological' anything, but I like the IDEA of being able to find files and resources on my PC quickly and efficiently. Also, if the package is shipped as default, then it must work, surely? And so I would like to find out why it does not on my machine, and even help out those who are working on it. Also, like many of us converts to linux, I regularly recommend linux (and KDE) to friends and indeed anyone who will listen, I can hardly do this in good faith when I know that a major plank of of the DE is broken, and known to be broken, and that the only advice I can offer is to switch it off!
Should I report this as a bug? If so, to whom and on which app? (opensuse, KDE, virtuoso, nepomuk?)
So where do I go from here? All (polite) suggestions gratefully considered!
Much thanks for reading and thank you to bcooksley for his/her advice.


openSUSE 12.3(kernel 3.11.xxxx/KDE 4.11/nVidia 6150se IGP/nVidia driver 304.108/AMD Athlon 64 X2 Dual Core Processor 5600/4Gb DDR2 RAM
User avatar bcooksley
Administrator
Posts
19763
Karma
87
OS

Re: virtuoso -t

Fri Sep 13, 2013 9:44 am
One thing you could check for is recursive symlinks on your system. Nepomuk is supposed to have some protections against this, but I suspect there may be limits to that.

Example:
Code: Select all
Folder at: ~/documents/stuff/
Symlink at: ~/documents/stuff/2011/work to ~/documents/stuff/


KDE Sysadmin
[img]http://forum.kde.org/content/bcooksley_sig.png[/img]
User avatar wakou222
Registered Member
Posts
14
Karma
0
OS

Re: virtuoso -t

Fri Sep 13, 2013 9:59 am
bcooksley wrote:One thing you could check for is recursive symlinks on your system. Nepomuk is supposed to have some protections against this, but I suspect there may be limits to that.

Example:
Code: Select all
Folder at: ~/documents/stuff/
Symlink at: ~/documents/stuff/2011/work to ~/documents/stuff/


Thank you, I will look in to that. I doubt very much that it is the case, as I never insert symlinks myself, and the indexer is only looking at /home (non-hiddens) and a drive which I call 'Store' mounted in fstab as /Store, which contains music and archived radio broadcasts, and some other media... So if I understand correctly, nothing should have inserted any symlinks without my knowledge.
Is there any way to look in the .db file, to see what exactly is filling it up?


openSUSE 12.3(kernel 3.11.xxxx/KDE 4.11/nVidia 6150se IGP/nVidia driver 304.108/AMD Athlon 64 X2 Dual Core Processor 5600/4Gb DDR2 RAM
User avatar wakou222
Registered Member
Posts
14
Karma
0
OS

Re: virtuoso -t

Fri Sep 13, 2013 10:37 am
I was wrong! (showing my ignorance :( ) I have gazillions of symlinks. Now to do some researching as to what my cmd:

Code: Select all
ls -lR /home/stephen/ | grep ^l

is actually showing me, and try to work out whether any of them recurse. Happily it is raining here, so gardening is off. Puzzling about on my system is ON!


openSUSE 12.3(kernel 3.11.xxxx/KDE 4.11/nVidia 6150se IGP/nVidia driver 304.108/AMD Athlon 64 X2 Dual Core Processor 5600/4Gb DDR2 RAM
User avatar bcooksley
Administrator
Posts
19763
Karma
87
OS

Re: virtuoso -t

Mon Sep 16, 2013 10:23 am
You can also use the following command to find symlinks on your system:
Code: Select all
find -type l


Symlinks under ~/.local or ~/.kde4 are expected and can be safely ignored.


KDE Sysadmin
[img]http://forum.kde.org/content/bcooksley_sig.png[/img]
User avatar wakou222
Registered Member
Posts
14
Karma
0
OS

Re: virtuoso -t

Mon Sep 16, 2013 11:09 am
Thank you..

I have installed a package called symlinks to make this a little clearer. Nothing untoward is reported using any of the various methods of testing for danglers or recursing symlinks.
There is a bunch of them inside various wine projects, but these are all under 'hidden' directories, and so should not be followed or even 'seen' by nepomuk.


I have re-enabled nepomuk file index to look ONLY at my 'store' It seems to be 'working' but ONLY for new files, it has not indexed (AFAICS) what is there already.
The .db file remains at 10Mb


openSUSE 12.3(kernel 3.11.xxxx/KDE 4.11/nVidia 6150se IGP/nVidia driver 304.108/AMD Athlon 64 X2 Dual Core Processor 5600/4Gb DDR2 RAM
User avatar bcooksley
Administrator
Posts
19763
Karma
87
OS

Re: virtuoso -t

Mon Sep 16, 2013 11:12 am
Okay, that is quite odd. It may need a logout / login cycle possibly, or the creation of a fresh database to trigger it to reindex the Store directory only.


KDE Sysadmin
[img]http://forum.kde.org/content/bcooksley_sig.png[/img]
User avatar wakou222
Registered Member
Posts
14
Karma
0
OS

Re: virtuoso -t

Mon Sep 16, 2013 11:26 am
OK, So I delete all below ~/kde4/share/apps/nepomuk/repository again, then reboot?

(I have logged out/in many times, and rebooted at least daily, since I re-enabled the indexing)


openSUSE 12.3(kernel 3.11.xxxx/KDE 4.11/nVidia 6150se IGP/nVidia driver 304.108/AMD Athlon 64 X2 Dual Core Processor 5600/4Gb DDR2 RAM

 
Reply to topic

Bookmarks



Who is online

Registered users: ajmoss, Baidu [Spider], Bing [Bot], Exabot [Bot], flipwise, Google [Bot], kyrhammer, mathpfvr, OlafLostViking, Saru, Snudl