This is an idea of mine I've been thinking about for quite a while, the more I think about it, the more I like it.

{{{{Remember, this an idea only, there is NO work to implement this as of yet. I'm hoping people would be interested in it and help me work on it}}}}


Meet Kiim, or rather, KDE's Intelligent Information Manager. (K.I.I.M)

Kiim is (or rather, will be) an intelligent agent. To specify; a natural language front-end between the computer, you and your information.

Take Kontact, a hub around other tools; Kmail, Kopete, Akregator and other personal information tools. Imagine Kiim doing a very similar task. Tasks are added by telling her about them.

What would this mean?


It would mean, in the end, a system that talks to you; gives you the news, alerts and even possibly reads out emails. Can talk with someone when your away through Kopete, reminds you of appointments...makes appointments for you (example through Kopete;
Jack: Hey Dan, I'm going to the bowling alley in 20 minutes, wanna come?
When I come back, Kiim tells me Jack's message, depending on how much I 'trust' Jack, Kiim could make this an appointment. Also possible would be Kiim creating tasks based on what was said through email/IM)

The above example is quite simple; keywords are a Natural Language Processor's best friend, in this case, such keywords are;

Going/Bowling/20 minutes/?


Who would it serve?

Just about everyone!
The idea here is to get your computer working alongside you instead of you working within it.



Whats more, it could monitor system health, manage the installation of applications, play media, manage the computer...all through natural language. Imagine that!

What's more, it could perform face recognition to log you in. (god knows there are plenty of resources online that can be freely used)

It could use festival to speak.

Put it in context for me.


As I am apt to do as a writer, let's stare into the crystal ball here...

You get up, aching and groaning. You turn your computer on. The system boots up. The camera turns on;

"Good morning, John. Today is the 14th of May. You have 11 new messages, two from your Mother. Do not forget that in six hours you have an appointment with the Dentist."

You read the messages, replying to a few from friends and family.

"How long until the appointment?" you ask Kiim.

"Six hours, twenty seven minutes, John."

"Give me directions to it" (Home's current position has been given to Kiim)

"Here is a map and a list of directions. Estimated time of journey; 8 minutes" (In this case, Kiim assembled data about the local services, gathered from online services, such as Google Maps. The map and directions, as well as the estimated travel time, are also gathered via the same route)

"Please print it."

"As you wish...I have printed it" (The pause there was indicative of Kiim getting feedback from the printer saying it had finished it's task)
---------------------------------
7 Hours later
---------------------------------
"Good evening, John. I trust you went to your Dentist appointment?"

"I did" (Kiim marks it off as 'completed') "but I've been given another one at midday this Saturday" (Kiim creates a new task, and knows it is another Dentist appointment by the eariler conversation)

"Very well. I shall alert you when it is time."
----------------------------------
"Kiim, I fancy listening to MUSE, can you put some on, please?" (Kiim goes through the music library, finds MUSE and starts it, with the most played tracks first)

15 minutes later...

(music dips in volume, Kiim speaks) "John, there is an email from Kate, concerning photos. Shall I bring it up?"

"Yes Please"

(John reads through the email and notes that Kate wants a copy of the photos of their recent camping trip)

"Kiim, please send a copy of all photos with the tags 'Kate' and 'Camping' from the last three weeks to Kate."

"It is done" (Kiim went through the photos, found pictures (she knew they were pictures due to the extensions) tagged 'Kate' and 'Camping' and sent them to a contact named Kate in the address book.
----------------------------------------------

"Kiim, bring up the document I was working on yesterday."
(Kiim looks for documents with the creation date of yesterday)

----------------------------------------------

"Kiim, send all my documents from yesterday to the disk" ("The Disk" is taken to mean the most recently inserted USB Drive, Kiim looks for any documents that were modified/created yesterday)

"Done."

------------------------------------------

"Kiim, translate this to Russian and send it Luba"

"....Translation done, sending....sent."

------------------------------------------

"God, I hate my brother..."

"Why do you hate your brother?"

(Kiim calls upon a clone of ELIZA to do the talking :P)

----------------------------------------------


There is a lot more that can be done.

Umm...well, that made for interesting reading, but...why?

Kiim has a lot of potential, and as our world expands, we must find ways to automate tasks, keep our lives fairly simple and make sense out of more and more and MORE data. Kiim would be the groundbase for it.

Such a project has never been attempted before in the history of FOSS, or even in the history of proprietary software. It's a massive undertaking, I agree. Thanks to the world of FOSS, a good deal of work is not needed. We're not going to be building from scratch, mainly just linking things toghter under a (code-wise, rather simple) interface.

I can do a lot of things, I know a lot about AI tech and I can assist. But I need developers. I would LOVE it if developers think this is a really good idea and start to think about it. AI is rather lacking in the one place it would be most useful; the desktop. I want to change that, but I need people of whatever skill level. I want to help make KDE a great thing. I want to make Linux available to everyone.

But...surely the desktop as it stands could do all that?

The Desktop metaphor as it stands does not easily lend itself to automation. This is to create an interface that can be used by anyone, as long as they know how to type, it can be used by them. Also consider the practical benefits this would have to the disabled. It also comes to mind that this interface would not be out of place in a Home Automation system. A future proof interface, basically put.


Well....maybe your on to something, got any specs?

I'm still in the process of writing them. PM me should you wish to discuss the more technical aspect of it further.

PS; I have spent the whole day writing and rewriting this post for clarity. If you've read all the way down here, thanks

Any and all feedback appreciated.

Hello Dante,

first, really impressive post, thanks for sharing us with your thoughts. Basically i feel this is a thing i'd love to see on my desktop (reminds of 2-3 movies i watched about future computers use ).

Technically, i would say most of it is already possible, at least the backends for it are on a good way. Just thinking of Akonadi, Nepomuk etc as the libraries to use. So the fundamentals should already be there. The most problematic could be the AI part, when it comes to speaking into a microfone in natural language and getting some commands out of it. Is there anything you already have in mind in this way?

P.S.: Eliza is a really funny application, i love it

Thanks for replying! . I was beginning to think this was being ignored...anyway;

Speech recognition is very difficult. There are one or two FOSS speech recognition systems available, notably, CMU Sphinx. Sphinx is more or less ready, but it lacks a critical component, the vocal model. Without this, no speech recognition program can understand speech.

Whilst I would love to include Sphinx in this idea, the only real way to get it working is to provide a trainable interface. That involves neural nets (So the training in basically, simple, but a lot of work to implement)

A rather more concrete and successful approach would be entering commands via text. (I admit this isn't the best, but when the project finally gets around to Sphinx, it'll be a simply matter of just hooking it up to the text input field)

As for the AI part, it's rather simple. As I said before, keywords. :P. There is a plethora of Natural Language Processing systems available. Perhaps most notable is the very basic code called; AIML (Artificial Intelligence Markup Langauge) which, whilst not conforming exactly to the needs of Kiim, is a good start.

There is a wide range of technologies in FOSS Kiim can use. Fuzzy data searching, natural langauge, machine translation...and a huge amount of online services to complement it. Most of the work behind Kiim would be just linking everything toghter...

I hope, neverendingo, your response is the first of many. I wonder if any of the devs will comment?

This sounds very well thought out.
And yes, i think the comments will come. You were just too impressive with such a big and well thought post
Keep it up, this is a really good thought, i will read up on the language systems you provided.

You've made my day!

Do you think I should put this into the Brainstorm section? The reason I put it in Discussions and Opinions first was because I really wanted to gauge the response to it.

Sure!

But to repeat it, as it was stated somewhere else, you won't need the respond from somewhere else here to put something into Brainstorm. Brainstorm IS MEANT to get feedback for your ideas from the user base.
If it is not valid, the moderators will act accordingly, but if it is, there is no problem with it (apart from the user feedback after submitting).

And if the users show you that they really like it we will find one or the other way to get the attention to the devs (without being specific here). It just will go its way from the "do we really need that?" to "we need it, help is needed to implement it" way.

So, please, go ahead, and let it test.

Thank you for your support.

I've posted it to the Brainstorm section.

neverendingo; do me a favour, think about how Kiim would grow in the coming years as the Semantic Desktop arrives and there is more processing power available. Think about the new forms of data coming in (like the location of restaurants). Then tell me you aren't dazzled by that prospect...I am :P

Thanks.

heh, sure i am dazzled

And that is why i approved your idea myself. So good luck with it. We'll see what comes out of it.

Hopefully a miniature revolution!

Thank you very much, you've been very supportive of my idea.

I hope the devs can see the possibilities of Kiim, and why it's worth building her (or him, or it)

Good luck for that.

As this idea is now shifted to Brainstorm, it would be nice if you could close this thread (just use the "it's solved" button.
Thanks in advance.