Hello,

I've added a new idea to the brainstorm about another approach to access to remote and local resources:

brainstorm.php?mode=idea&i=84975

In short: my idea is a construction build with the automounter and a fuse module, which creates for every user logged in a map with connections to all kind of resources, local and remote. Mounts of SMB (Samba or Windows) shares are mirrored into this map, making it look like these shares are mounted there.

The reactions talk about different issues, one of them is the need for kde applications to know it's a remote or a local file. How do they find out normally?
Does it take a socket or something to communicate with
the applications?

Stef

KIOSlaves are spawned by KLauncher on request from the application. It then passes back the appropriate socket and the application and slave interact this way.

KIOSlaves have a *.protocol file associated with each protocol they support which details information such as if they operate locally or remotely.

For maximum support you would probably be best to use some form of KIO FUSE type system, however do note that KIO may not support enough to map the POSIX file system API to it.

Thanks,

some questions still. So how does the application know which protocol to use??
I guess they only know which to use, if the url given by the user (and/or environment) contains the protocol like in an URI:

smb://someserver/someshare/somefile

It then knows about the kio slave to use, kio_smb in this case, which tries to perform the requested action, via libsmbclient.so. It does not perform a mount (where my construction does, and makes use of the network fs cifs..).

I must say, this is stateless, where my construction is statefull. You mention that the kio subsystem does not support enough to make the POSIX file system API to it. I do not know what you exactly mean by that, do you mean locking, ACL/ATTR, notify, all the permissions/ownership available??

I see there are two approaches to remote resources.
First my "network" tree in the homedirectory, presented by my fuse module:
/home/sbon/Workspace/Network/Windows Network/Workgroups/Servers/Shares/

versus uri notation:

smb://Servers/Shares/

Isn't it possible to make the kio slave kio_smb make use of the mounted shares, (mounted by automount and the fs cifs) in stead of setting up an entirely (or is it entirly?) new connection. I'm looking for a way to make the kio slaves work together with my construction, making use of the mounting of smb shares via cifs, doing the real IO not via the fuse module but directly on the mounted share.

Note that this is not only about smb.

Stef Bon

I think the primary weakness of KIO in this case is seeking.

Permissions, Ownership and ACL's are likely all well supported.

I am not aware of any capability in kio_smb ( or KIO in general ) to use already existing connections by the Operating System.

ACL/ATTR supported by kio_smb?? Notify? Sure?
Locking? Connections made by the kio slaves are no mounts.

What do you mean by seeking. Seeking in a file?
Or seeking like searching?

Stef

Seeking as in a file.

While I am not sure that kio_smb supports notifications, ACL's and attributes, I do know that kio_file does, so KIO must include the needed infrastructure.

I believe using FUSE to mount KIO slaves has been attempted before, the code is in KDE Playground.

Ok,

a remark, you say "using FUSE to mount KIO slaves has been attempted before".
I suggest here just the other way arround, kio slaves should make of fuse (read: this construction)

A question, would it make a chance when I come with this idea to kde developers to make the kio system make use of existing mounts, or better,
"resource records". The construction I've build is based on these resource records. On the computer you're working on every resource the user wants to make use of or has/wants access to is created first. No resource record, no connection. An example, the record with at /var/cache/mount.md5key/0d3b3c4884ba3cd3f84a792d01f8f220 contains:

RECORD_type=smb
RECORD_security=private
RECORD_creator=discover-smb-resources.sh
RECORD_resource_uri="smb://LFS20060812/bononline/"
LOCAL_user=gast
SMB_workgroup=BONONLINE
SMB_name=LFS20060812
SMB_share=bononline
SMB_ip=192.168.0.2
SMB_auth_method=private
SMB_credential_file=/home/gast/.smb/mount.cred

This record is used by the automounter, to mount the resource (here: smb//LFS20060812/bononline/) in a directory specially for this user (gast):

/mnt/mount.md5key/gast/mount/0d3b3c4884ba3cd3f84a792d01f8f220

and with the help of a fuse module I've developed, fuse-workspace-union, this resource is made available in the homedirectory of the user, in this case "gast" like:

/home/gast/Workspace/Network/Windows Network/BONONLINE/LFS20060812/bononline

Now, this is an example of one resource, in practice there are a lot of records, not only smb related, but also FTP, SSH, IPX (Novell Netware) and even local resources like USB is possible, giving you as user a tree:

/home/gast/Workspace/Devices/...usb.sticks...
Network/FTP/...servers..
Novell Netware/....
SSH access/....
Windows Network/..workgroups../servers/shares


Now, what I propose is that the kio subsystem should make use of these resource record. Whenever an app tries to connect to a remote source to open a file for example, first look at these records, to try to make
the connection through this construction, for example the file tesfile on share bononline on server LFS20060812:

smb://LFS20060812/bononline/testfile

At this moment the kio subsystem uses the slave for this protocol, kio_smb, which connects to server LFS20060812 (with the help of libsmbcient.so).

I'm not suggesting to replace this, but to make it possible to make the kio system make use of the resource records and the fact that resources are available on the system via autofs managed directories.

I'm working with this construction on my machine, with KDE 4.4aplha, and it works very good, but it lacks integration with KDE.

So, does this idea make a chance??


Stef BOn

bcooksley wrote:KIOSlaves have a *.protocol file associated with each protocol they support which details information such as if they operate locally or remotely.

While it is true that KDE has these meta information about protocols, I am not sure it is used or widely used to check for locality.
The most common way is to check KUrl::isLocalFile() (but this could be accessing the protocol meta information, not sure).

kio_smb could probably be made to delegate to kio_file in case something is already mounted locally and use direct connection if it is not.
But again I am no expert in KIO internals, chaining might have some limitations.
And interesting question would still be how this kind of forwarding will impact error handling, e.g. how does the IO slave know that the connection to the SMB server got lost.

The best approach so far I've seen for the scope of GUI applications is GVFS, which uses a similar concept of helper processes (mount daemons) but shares them per protocol rather than per document.

While I know if attempts to map it into KIO I personally see more value in making a KDE based client library instead, thus allowing native access (Qt sockets, event loop, probably job based API) to GVFS mount daemons.
This would allow to evaluate the two approaches side by side for different scenarios without getting anyone into "will be replaced" panic.

anda_skoa wrote:kio_smb could probably be made to delegate to kio_file in case something is already mounted locally and use direct connection if it is not.
But again I am no expert in KIO internals, chaining might have some limitations.
And interesting question would still be how this kind of forwarding will impact error handling, e.g. how does the IO slave know that the connection to the SMB server got lost.

The best approach so far I've seen for the scope of GUI applications is GVFS, which uses a similar concept of helper processes (mount daemons) but shares them per protocol rather than per document.

While I know if attempts to map it into KIO I personally see more value in making a KDE based client library instead, thus allowing native access (Qt sockets, event loop, probably job based API) to GVFS mount daemons.
This would allow to evaluate the two approaches side by side for different scenarios without getting anyone into "will be replaced" panic.

What do you mean with "shares them per protocol rather than per document". The kio slaves do things per document, while GVFS - you say - uses mount helpers. With simular you refer to my construction, where the actual mounting is done by autofs the automounter??

What do you mean with two approaches side by side? Which two approaches?

Your last remark makes sense, cause I've put a lot of effort in this construction, and can handle critics, but if it is just left out without any comments, or developers of KDE choose to develop something new without reacting on suggestions of me, I will not like that!

I hope for a constructive discussion.

Thanks,

Stef Bon

bonassis wrote:What do you mean with "shares them per protocol rather than per document". The kio slaves do things per document, while GVFS - you say - uses mount helpers. With simular you refer to my construction, where the actual mounting is done by autofs the automounter??

As far as I understood GVFS it uses one mount daemon per protocol so it can share reuse connections that go to the same server.
KIO uses one IO slave per request, basically a per document approach.

bonassis wrote:What do you mean with two approaches side by side? Which two approaches?

Native GVFS access and KIO, but this basically applies to any technology which is tendend to do something like KIO.

bonassis wrote:Your last remark makes sense, cause I've put a lot of effort in this construction, and can handle critics, but if it is just left out without any comments, or developers of KDE choose to develop something new without reacting on suggestions of me, I will not like that!

Basically the most important thing is to not claim one is creating a KIO replacement. People now how KIO works, how they can use it etc.
Any other system will first have to prove that it is as easy to use and can do at least the same things before people will consider moving away from KIO.
It is therefore better to have something that can be evaluated independently, e.g. a demo program that does something an application would use KIO for.

Cheers,
_

About connections with my construction, it makes use of (at SMB ) the automounter to do the mounting, and cifs for the connection.
Cifs on linux also tries to have one connection in stead of one for every process. If more than one user is working on the machine making use of cifs I do not yet know if this is resulting in still one connection, but two.

Futher I'm still busy developing, and working on how it handles locks.
Fuse has the ability to perform locks.

Also this construction is already able to do also USB mounts.

And working on good documentation to explain step by step how my construction works, and how one is able to make it work on the own host, and how to automate it.

Thanks for your reply,

and I'll report back here when I'm ready with the documentation.

Stef Bon
the Netherlands