Hi everybody,

TLDR - Short version:
We are reworking part of the "Save As" code, and we are adding the long-requested Save button! The old "internal implicit save" behavior is being dropped and we need your input about what to do with data saved internally by previous Okular versions.

Long version:
Okular has always been "secretly" saving your annotations and forms in XML files in the ~/.kde/share/apps/okular/docdata/ folder (using the file name as key). However, if your file is a PDF, in recent Okular versions you can also export them natively to a new PDF file through "File -> Save As". But, as many of you know, even the latest released versions have no plain "Save" button.

This behavior was initially designed because of several reasons. One of them was that there was no native save support initially, because Poppler -the library we use to work with PDFs- used to have no PDF writing capabilites. It also gave us the "feature" of being able to annotate document stored in read-only locations (e.g. HTTP websites) and another important design goal was to be able to annotate documents whose format does not support saving annotations natively, basically every format but PDF.

However, we were (and still are) saving into a "secret" folder using the document file name as key, and this makes it particularly hard to share annotations/form data to other users. Especially, a lot of user complained that when they rename their documents or send them to someone else their annotations and form data seem to disappear.

For this reason, the "Document Archive" format (.okular) was created long ago. It basically packs the original document and the annotations/form data in a single file, enabling users to easily share their work with other Okular users. This format works well, but it has been a "second-class format" until now because you have to click on "File -> Export As -> Document Archive" in order to create such a file.

Because of these issues, in the last Okular sprint we've been working on what we called "become a text editor", in other words, we're planning to have no secret files behind the scenes anymore and let the user always "Save" or "Save as" explicitly, just like in a text editor, in the best format possibile (either natively or as .okular, if the native format is not able to store the annotations/form data).

So the big question is: the docdata folder is going away, but what should we do with existing annotations stored by previous Okular versions in that folder? We don't want you to lose your data, so we need to find a way to ask our users to move their annotations and form data out of that folder.

The basic idea is that you should "Save As" as soon as possible to migrate to our new approach, and from there carry on with the new "Save" button every time you make a change. After you "Save As", the corresponding docdata/ XML file will be deleted and your annotations will only exist in the file you just saved to.

We ask your opinion about when you should be asked to save your annotations to a new file.

OPTION 1 - When you open a previously annotated file (blocking)
You get a blocking popup window saying "Okular has changed the way it stores annotations and forms, please Save As now in order to migrate and be able to continue annotating". The dialog will have two buttons "Save As now" and "Ignore/Remind me later". The former will let you save to a new file and continue editing, the latter will open the document in read-only and disallow annotation and form modifications, and you will get the same dialog the next time you reopen the same document.

OPTION 2 - When you open a previously annotated file (non-blocking)
Similar to the previous option, but open the document in read-only mode by default and show a "blablabla please save as now" with a "Save As" button in a KMessageWidget (similar to the current "Show forms" bar) to unlock editing capabilites.

OPTION 3 - When you first edit an annotation or form (blocking)
After you first add/edit/remove an annotation or edit a form, you get a "please save as now" with "Save As" and "Cancel". If you cancel we don't know how to save your modification, so we will have to revert it (i.e. we will do a forced undo).

OPTION 4 - When you close the document
If you have made any modification, you get a "Your changes will be lost if you don't Save As now" when you try to close the document, with "Save As", "Discard" and "Cancel" options. In case of "Discard", we won't delete the docdata/ XML file, so that the old annotations/form data will still be there the next time you reopen the document.

I would suggest to add another "button" to any of the four options above in order to allow one time migration of all annotations. This option would convert most files at once and prevent users of being bothered every time they open another edited doc. This would also help to migrate docs to the new format more quickly as there might be documents not opened for very long periods (or never). After this migration it would be nice to show users which documents where not found and allow them to remove them or keep their information (if this docs are stored in removable media) for a later migration.

Will Okular support saving annotations to files (PDF, ODF, OOXML) ? If so, there should be a "Save" button next to "Save As" when possible, because my guess is most users will just wish to save their annotations inside the original document.

Louis

I second Louis's proposal to include a "Save" button, if possible.

An option to migrate all docdata at once would also be nice, but I am not sure, whether I would be brave enough to use it
(I've no idea what files I've added comments to and I am not sure whether It would be safe to modify all of these files).

Ugh, I'm not a fan of this change in general. The secret automatic saving was very clever and useful. I have a lot of PDFs in a cloud service, and use Okular to highlight important areas. Those changes don't affect the file itself, hence don't trigger the cloud update and other people I have the files shared with don't get my annotations. So it's very nice and easy, I open the PDF and highlight things, and I see it next time I open it, all without conflicting with others.

So my preference would be to keep both ways. By default, auto-save "uncommitted" changes in an internal local share directory, and then commit them to file only if the person presses "save". If not saved, on exit have a notify pop up, saying that changes haven't been committed to the file, asking to either save or continue using the internal saves, with an option "do not ask again".

I kind of side with GreatEmerald here too.

I do almost the opposite of him and store my 'annotations' in the Cloud so that they're the same on Desktop or Laptop.

However, the .pdf (or other) files themselves live on my Desktop and only get copied across to my laptop when I need them.

In any case though, I'm sure I will adapt.

(Also: Thanks for making such an excellent piece of software! Cleanest PDF viewer I've ever used!)

I am a heavy Okular-PDF-annotator and I am concerned about if PDF will always be saved as PDF (best possible format) or .okular. I will only be able to open .okular on subset of Linux systems running Okular.

Personally, I would only support formats that Okular is 100% confident that it could write in natively. No software should do anything that it isn't confident of 100% results.

Very much pleased about these future changes!

Thank you all for your input and keep voting!
Let's answer some questions.

aitorpazos wrote:I would suggest to add another "button" to any of the four options above in order to allow one time migration of all annotations. This option would convert most files at once and prevent users of being bothered every time they open another edited doc. This would also help to migrate docs to the new format more quickly as there might be documents not opened for very long periods (or never). After this migration it would be nice to show users which documents where not found and allow them to remove them or keep their information (if this docs are stored in removable media) for a later migration.

This was our first thought too, but we decided that asking to migrate all documents when you first open the new Okular version is too "blocking" for the user. Also, as you said, there would be issues with temporarily unavailable/deleted/moved/renamed files. So we decided that this cannot be the only way to migrate, maybe we'll add it later as an extra migration way, if there's enough request for it.

louis94 wrote:Will Okular support saving annotations to files (PDF, ODF, OOXML) ? If so, there should be a "Save" button next to "Save As" when possible, because my guess is most users will just wish to save their annotations inside the original document.

Yes, there is going to be a Save button. But Okular currently only supports saving natively to PDF. For the other formats, "Save" you tell you that annotations cannot be saved natively and it will let you save as .okular archive, which can save annotations no matter the format of the original document.

GreatEmerald wrote:Ugh, I'm not a fan of this change in general. The secret automatic saving was very clever and useful. I have a lot of PDFs in a cloud service, and use Okular to highlight important areas. Those changes don't affect the file itself, hence don't trigger the cloud update and other people I have the files shared with don't get my annotations. So it's very nice and easy, I open the PDF and highlight things, and I see it next time I open it, all without conflicting with others.

So my preference would be to keep both ways. By default, auto-save "uncommitted" changes in an internal local share directory, and then commit them to file only if the person presses "save". If not saved, on exit have a notify pop up, saying that changes haven't been committed to the file, asking to either save or continue using the internal saves, with an option "do not ask again".

I see your point, however, your workflow doesn't work with all kinds of files. For some files saving to docdata is not possible and Okular already forces you to save externally (e.g. PDFs with embedded videos or with embedded PDF annotations). So we are kind of unifying the behavior for all kinds of files. Also, as I said in the initial post, this "secret file" behavior seems to confuse most users, who expect exactly the opposite, i.e. having their annotations be synchronized on the cloud or sent via email along with the PDF etc...

jimtendo wrote:I kind of side with GreatEmerald here too.

I do almost the opposite of him and store my 'annotations' in the Cloud so that they're the same on Desktop or Laptop.

However, the .pdf (or other) files themselves live on my Desktop and only get copied across to my laptop when I need them.

In any case though, I'm sure I will adapt.

(Also: Thanks for making such an excellent piece of software! Cleanest PDF viewer I've ever used!)

Uhm, so you symlinked the docdata folder to the cloud storage, but you don't want to synchronize PDFs, correct? Yes, I'm afraid this will no longer be possible.

donniezazen wrote:I am a heavy Okular-PDF-annotator and I am concerned about if PDF will always be saved as PDF (best possible format) or .okular. I will only be able to open .okular on subset of Linux systems running Okular.

Personally, I would only support formats that Okular is 100% confident that it could write in natively. No software should do anything that it isn't confident of 100% results.

You will have both choices, but yes, for PDFs the default format is going to be PDF. For the other formats, native, if you have not modified the document (this will be just a plain file copy), or .okular, if you have made modifications.
It will also be possible to open .okular files and save them natively, so something like this becomes possible. Note that saving back to native will embed the annotations if your file is a PDF, or drop them if it is not (there is going to be a big warning about it).

To keep supporting the two workflows, one could re-use the old docdata format to allow saving annotations only. This way, an user could save his/her annotations to the folder (s)he wants, which would be less confusing.

Since saving to PDF is supported, I'll cope with it.

Louis

I think non-blocking is the way to go. Also a way to migrate all existing annotations to the new format in one go would be useful (also non-blocking, something like amarok does when scanning its collection).

I don't see how option 3 fits here?
The other options refer to the old docdata whereas option 3 describes the general approach for a changed document, so I don't see how these are exclusive...

In general:
please don't use blocking / popping modal dialogs - they are really disturbing, but instead use the from-top-sliding-in bar, like in gwenview, rekonq, kwrite, firefox, etc... which give a much less interrupting (and thus smoother) user experience.

About the transition, I must say I don't care this much since I stopped using the feature, because of said problems (rename / moving of files, lost annotations). Why wasn't a checksum based approach used here?

Also one final question, I didn't quite understand it from the OP: Do you support now (ie. save to) pdfs with annotations or will it all still be .okular in the future?


Otherwise thanks a lot for this fine program, I use it alot!