Ok, I wasn't entirely sure where to put this, but here goes;

I imagine a few of you know of my K.I.I.M idea? Well, we might as well break it down into smaller chunks now. For those that don't know, KIIM is an idea for a next generation system, somewhat similar to a science-fiction setup (though it's quite possible to do)

The main part of KIIM, however, is understanding and parsing natural language.

Examples;
If the user types into the text entry field of this Conversational Interface;
"Ivan's email address is ivan@gmail.com"
Then the system should automatically add that email to a contact named 'Ivan' in the addressbook.

"Here to Jack's house by foot."
Intergrating with online services to provide geolocation of the user. Intergration with KAddressBook to find out 'Jack's location. Google Maps (or another similar service) to plot the route.

"Show me all documents created yesterday"
Would show all text files with the creation date of yesterday.

"Transfer all documents about cars into my USB stick."

Using NEPOMUK/Scribo to find documents on the system about cars, the system then locates a USB stick on the system and copies them to it.


The benefits of this interface are;

1. No GUI needed, a tool wouldnt need to clutter up the DE by using it's own GUI. As such, this also saves time on development.

2. Computers are usable to more people; many people have trouble with computers, especially when it comes to the Linux terminal. This provides the most natural and most easy way to command your computer ever created.

3. More functions. I have thought of many useful functions that would be very useful, but which is very hard to justify a full GUI for, or would make the GUI an impractical solution. (Such as being to get the computer to automatically use Harvard referencing methods when trying to reference another document.)



Now, I am on the verge of pleading here, I think this could be SO useful to so many people (not to mention a rather unique way of working with computers). I'm quite willing to pay for it's development (I don't earn much, but what I can give to fund this project, I will do)

If there is ANYONE out there who wants to help create this, please, get in contact.

-Dante

For a full, proper report on conversational interfaces; look at

http://www.dgp.toronto.edu/people/byron/papers/nli.html

Remember that item was written in 1994, many of the problems that would face such a system have been solved.

I'd be willing to get involved, though I'm not sure I'd be of much help with this as I've never really worked on large projects or anything, but I'd be very interested in keeping up to date with this, and would be more than happy to offer some patches or feedback if I could.

At the very least I could test it out once it gets to a usable state and find and try to fix bugs.

Just wondering though, how do you plan to actually implement this (If you've thought about that already), would it be a complicated and ambitious project to actually understand the languages, or would it work on keywords or Regex?

Personally I think (and I apologise if you've already thought of these details) that being able to recognise patterns (Using Regex) would probably be the way to go.
For example if it could recognise:

Code: Select all

<A>s <B> is <C>

Then it could basically go through the following steps:

What is B, and do I have somewhere to store it?
If B isn't recognised it could just use it's own Database to store "<A>s <B> = <C>".
However in the case of B being "email address" or something it knows that it can use KAddressBook instead of having to use it's own database, in which case it could search for people with name <A> and store the email address, or create a new person if it's not found.
If more than one person with name <A> (It could check first and surnames as well as trying to find full match) it could provide a list for the user to choose.

If it used these patterns then you could have different ways to say the same thing, for example "<A>s <B> is <C>" and "<C> is <A>s <B>" could both be supported simply by adding another regex pattern.

Translation should be simple enough as with each language it would simply be a case of translating the Regex's and providing info as to what each variable represents (For example a language which Uses "Verb-Subject-Object" word order instead of "Subject-Verb-Object" might not translate directly and may need extra information to communicate this to the program).

There could be a list of recognised verbs and verbals forms to cut down on the amount of regexes you have to test it on (Only test for the above mentioned cases if the word "is" appears for example, so you don't have to test every sentence with every regex).
This way it could be easier to add support for new programs as well as new languages, as each new plugin (assuming it were to be plugin based) could provide a list of regex expressions and the key word for each.
An Amarok plugin would be a list of regexes such as "Listen to <X>", "Play <X>", "Play music by <X>", etc. and the keywords "Listen" and "Play" to identify these.
If the word "listen" is encountered in the sentence it will test the first regex, if the word "play" is encountered it would test the second (You could, in some cases, such as these, only care if the keyword is at the start of the sentence, but not in all cases)


I see from the other thread that you're considering using keywords but I think that it may provide too many "false positives" and end up doing unintended actions.
I think the regex system I briefly mention above might be better? It could cover most cases if the expressions are chosen carefully and it could fall back to a keyword system if this fails, as keywords for functions will have already been defined if the above is followed (I'd recommend asking for confirmation if it can't be sure what user means though).

Would be interested to see what you think of this, you might be able to see flaws in it that I can't, or have reasons for preferring keywords that i haven't considered.

(And I'm aware the examples above aren't actually regex but I haven't used it in a while and just wanted to give examples without having to look it up and worry about whether or not it's right.)

The reason I chose keywords was because they would be easier for developers to add to it.

As for the database; no point keeping copies of information already stored, is there? The system should act as an interface to the computer, not a replacment for the large-scale work already being done. (I'm also assisting on the NEPOMUK-KDE project, hence my lack of time to respond to this thread, my apologies)

I really can't see the difference between the regex and keywords, the goal of the system is that it can take commands/queries like:

"Show me all documents I worked on yesterday" or "Format my external hard-drive to ext3", rather then "gksudo something-something-something" (which is, lets face it, not the most intuitive way of doing things)

Dante Ashton wrote:As for the database; no point keeping copies of information already stored, is there? The system should act as an interface to the computer, not a replacment for the large-scale work already being done. (I'm also assisting on the NEPOMUK-KDE project, hence my lack of time to respond to this thread, my apologies)

I meant that the database (if there is one) would only store information that it doesn't know how to store any other way, so it wouldn't store email addresses or birtthdays or anything, but if the user typed "Marks favourite food is Pizza" or something (possibly bad example) it wouldn't know how to store "Favourite food" so would resort to it's own database.

Oh and no worries about that, just wanted to make sure you saw the post.

I really can't see the difference between the regex and keywords, the goal of the system is that it can take commands/queries like:

"Show me all documents I worked on yesterday" or "Format my external hard-drive to ext3", rather then "gksudo something-something-something" (which is, lets face it, not the most intuitive way of doing things)

The regex I was thinking would just be a way to make sure that the most common commands are 100% recognised, keywords rely on the system "guessing" what is wanted (albeit cleverly if done right).
While it probably wouldn't be a problem, it may occur that when adding new features or updating how the system works it might slightly change how it matches keywords, and commands that worked in older versions may not work with newer ones.
With regexes this can't happen (unless there's a clear bug).

Of course it could always support both, and fall back to keywords if no matches are found?
Or perhaps it might even be best to just work on keywords for now and then consider regex at some point later, but I was thinking that if this was done after a large amount of features were added it would mean a lot of work, but then again, if both were supported not every function would need a pattern, just the most common or anything that seems tricky to do with just keywords (perhaps due to other functions using the same keywords).

The idea is that a developer does not need to write in every possible query to use a program/function, rather, it retains some 'fuzzy' aspect. With fuzzy systems, yes, there is a chance of false positives, but it's easy for such a system to identify when 'it isnt sure' and thus, can prompt a user.

About the Mark likes Pizza bit, that is, eventally, the kind of data NEPOMUK will store.

I guess that makes sense, something like how KRunner works might be useful, as in while you type it shows a list of the 2 or 3 commands that match your query the most (with most likely on top) so you can see before you press enter what it would do, and can choose another choice if that's not what you want.

Would need to give more verbose output than KRunner though probably, since it would be doing more than just opening the program, it would also be making it perform a specific function.

http://picasaweb.google.com/MentalOmega ... directlink

In that link are a few mockups depicting KRunner acting as the interface.


I beleive there was also a plasmaoid not too long ago on either KDE apps or KDE look. that could provide this functionailty.

Hmmm that looks good, but I'm pretty sure that once you press enter the KRunner interface will disappear, as I haven't seen it stay open for anything before.
So you would have to type in the command and then sit and wait for it to complete (unless I'm wrong about the way KRunner works or it can be overridden)?

Oh and I can't find the plasmoid you describe in the 21 pages of plasmoids on kde-look. So either it's somewhere else or it's been removed.

KRunner could be altered to adjust for it.

Remember those are mockups. :P

Hmm...have you tried KDE apps? I'll look as well, of course.

I haven't looked on kde-apps.org but there doesn't seem to be a category for plasmoids there so i assumed it couldn't be there.