This forum has been archived. All content is frozen. Please use KDE Discuss instead.

Save as XML - transaction sequence ?

Tags: None
(comma "," separated)
jehoshua
Registered Member
Posts
100
Karma
0
OS
Currently looking at importing some old QIF files ( see viewtopic.php?f=69&t=170819 ). In the past I used to be able to:

1. * Backup KMM file
2. * Import QIF data
3. * Backup KMM file
4. * Compare the uncompressed KMM data from steps 1 and 3 (Beyond Compare)

However, due to some sort of 'security' feature, the KMM data is not stored in strict sequential order. At least that is how I remember the (new) format.

So, have done a "save as" of format XML and then opened with Kate. It seems to be in date/time sequence.

Is it only the .KMY data that can be in random order, and the XML data will stay in strict date/time sequence please ?


Currently running Kubuntu 22.04, Plasma 5.24.4, Frameworks 5.92.0, Qt 5.15.3, kernel 5.15.0-46-generic (64-bit), GeForce 840M Graphics
jehoshua
Registered Member
Posts
100
Karma
0
OS
So, I added only one transaction that had 2 splits, saved that as XML, and comparing the 2 files there are no less than 24,798 differences. There is therefore NO way I'm going to be able to reconcile as I used to. :<

Is there a setting in the config for KMyMoney to make this so called 'security" feature an OPTION ? Thereby keeping the transaction sequence in order, not just line by line, but for every component of the XML structure.


Currently running Kubuntu 22.04, Plasma 5.24.4, Frameworks 5.92.0, Qt 5.15.3, kernel 5.15.0-46-generic (64-bit), GeForce 840M Graphics
User avatar
ipwizard
KDE Developer
Posts
1359
Karma
6
OS
In fact what you experience is caused by a security feature of the underlying Qt framework. What happens is that the order of the XML attributes is random, hence the many differences in your file when saving it.

See the Qt documentation on how you can possibly turn off the randomization for your purpose. I have not verified that method, though.


ipwizard, proud to be a member of the KMyMoney forum since its beginning. :-D
openSuSE Leap 15.4 64bit, KF5
jehoshua
Registered Member
Posts
100
Karma
0
OS
ipwizard wrote:In fact what you experience is caused by a security feature of the underlying Qt framework. What happens is that the order of the XML attributes is random, hence the many differences in your file when saving it.

See the Qt documentation on how you can possibly turn off the randomization for your purpose. I have not verified that method, though.


Great, thank you for that link. I see ..

This randomization of QHash is enabled by default. Even though programs should never depend on a particular QHash ordering, there may be situations where you temporarily need deterministic behavior, for example for debugging or regression testing. To disable the randomization, define the environment variable QT_HASH_SEED to have the value 0. Alternatively, you can call the qSetGlobalQHashSeed() function (https://doc.qt.io/qt-5/qhash.html#qSetGlobalQHashSeed ) with the value 0.


I see I'm using

Qt
Version 5.15.3 (built against 5.15.3)


in KMyMoney version 5.1.2

As to how/where to specify the correct QT variable , it seems this is a good start - https://forum.qt.io/search?term=QT_HASH ... itlesposts


Currently running Kubuntu 22.04, Plasma 5.24.4, Frameworks 5.92.0, Qt 5.15.3, kernel 5.15.0-46-generic (64-bit), GeForce 840M Graphics
jehoshua
Registered Member
Posts
100
Karma
0
OS
It seems all I had to do was to set the environment variable QT_HASH_SEED to zero. Used https://linuxconfig.org/how-to-set-and- ... s-on-linux as a guide.

$ printenv QT_HASH_SEED
$
$ export QT_HASH_SEED="0"
$ printenv QT_HASH_SEED
0

So, now that is setup. In KMM, did the 'save as XML', then added a transaction, did the 'save as XML' again , different file of course. Checked in Beyond Compare and the randomisation is still happening. :<

Maybe KMM wasn't picking up the temporary environment variable ? Later, .. reading up a bit and that does seem to be the case, so I need to modify /etc/environment and test again.


Currently running Kubuntu 22.04, Plasma 5.24.4, Frameworks 5.92.0, Qt 5.15.3, kernel 5.15.0-46-generic (64-bit), GeForce 840M Graphics
User avatar
ipwizard
KDE Developer
Posts
1359
Karma
6
OS
Did you save it once after you have setup the env var or twice and compared the two versions after you have set it up? Turning off the randomization in the first step will of course bring a different result as the seed for the hash is different. Only two versions after the env var is set will use the same seed (if that feature works as expected).


ipwizard, proud to be a member of the KMyMoney forum since its beginning. :-D
openSuSE Leap 15.4 64bit, KF5
jehoshua
Registered Member
Posts
100
Karma
0
OS
ipwizard wrote:Did you save it once after you have setup the env var or twice and compared the two versions after you have set it up? Turning off the randomization in the first step will of course bring a different result as the seed for the hash is different. Only two versions after the env var is set will use the same seed (if that feature works as expected).


Thanks for your reply. I decided to do it all (properly) again, as follows:

1. Full Backup to external drive

2. Modify /etc/environment to add

QT_HASH_SEED=0

as per https://superuser.com/questions/339617/ ... 652#339652

3. Log off, login and check value

printenv QT_HASH_SEED
0

4. KMM - Open Australian.kmy and 'save as XML' - filename qt1.xml

5. KMM - Open Australian.kmy again and add a transaction, save and backup

6. 'save as XML' - filename qt2.xml

7. Beyond Compare - compare qt1.xml and qt2.xml

Found only six different sections. There was the transaction added, it's splits and then 5 other modifications to values. Great to see it all in nice and neat 'order'.

8. Hold backup from step 5. Open Australian.kmy again, make a modification to two transactions, save and backup

9. 'save as XML' - filename qt3.xml

10. Beyond Compare - compare qt2.xml and qt3.xml

Found only 14 different sections. As the 2 transactions updated were for a reconciliation flag, the changes reflected for the accounts, payees and splits,etc.

The ONLY section/s of the XML file that were (now) out of order were to do with 'category ID' - see https://pasteboard.co/2CEGJnR7e771.png. If that is the only random/disorder that is okay for testing and reconciliation, however it will obviously be better to stick with the "only twice" rules as "ipwizard" suggested.

11. Cleanup - modify /etc/environment to remove

QT_HASH_SEED=0

log off, login and check the environment variable

After log off, login ..

$ printenv QT_HASH_SEED
0

$ cat /etc/environment
PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin"

Tried another log off, log on. Environment variable was still there. Cache maybe ?? Rebooted and env. variable was now gone.


Currently running Kubuntu 22.04, Plasma 5.24.4, Frameworks 5.92.0, Qt 5.15.3, kernel 5.15.0-46-generic (64-bit), GeForce 840M Graphics
jehoshua
Registered Member
Posts
100
Karma
0
OS
This was a success, adding the environment variable caused the XML data to be in strict order, before and after a QIF import, or any sort of transaction modification. ;D


Currently running Kubuntu 22.04, Plasma 5.24.4, Frameworks 5.92.0, Qt 5.15.3, kernel 5.15.0-46-generic (64-bit), GeForce 840M Graphics


Bookmarks



Who is online

Registered users: Bing [Bot], claydoh, Evergrowing, Google [Bot], rblackwell