I'm not sure if it's just me, but developing Krita in windows (MinGW) is a serious pain because of the slow compile times and slow run speeds. Just launching the debugger sets me back a couple of minutes. Thank goodness there's jom (comes with Qt Creator) which allows compiling with multiple cores. Compilation was much slower without it.

What I do now is to develop first in linux (compiles 3-4 times faster, and it runs almost instantaneously), and then only make sure it works in windows.

Anyhow. I noticed that the windows final krita build was pretty fast. Can I get that kind of performance also? What do I need to do?

Well, slow build C++ speeds in windows is not a surprising thing. I played with blender code before and it's the same thing. Just wondered if any of you windows developers here have magic workarounds to speed things up.

Windows 10 64-bit, TDM-GCC 5.1.0 64-bit

The big slow-down is the ntfs file system. It's a bit better on an SSD, but it's a seriously slow file system and there are about 3000 c++ files that need to be compiled.

Linux ext4 wins then. Hahaha! No wonder large C++ projects are so slow in windows.

would it be faster if FAT is used? Not sure if there are other file systems that windows can use which would make it faster. Perhaps microsoft should consider adding ext4 support for windows.

"A bit better" on SSD doesn't sound very convincing...

Bruceoutdoors wrote:Thank goodness there's jom (comes with Qt Creator) which allows compiling with multiple cores. Compilation was much slower without it.

mingw32-make does allow parallel jobs...

I was trying to get a clang (gnu/mingw-w64) build, but I haven't got anywhere.

There appears to be some compiler bugs which prevents me from building Qt (not that building Qt with clang gnu toolchain is supported at all lol.) Lots of hacks and changes to make the build system work, but then hit by a compiler bug with linking... how wonderful. At least it's not something you'd rebuild often, so I probably can just use the one built with mingw-w64 instead.

Some 3rd party deps have problems with clang too. At first there were only some minor problems, but then I seem to have hit the same or similar compiler bug. It didn't help that the latest mingw-w64 I was using couldn't compile some of the 3rd party deps either, failing witih ICEs (wtf) which are probably compiler bugs too. Without the 3rd party deps I can't even try building Krita at all. (Though even if I could, I'd probably get hit by the same compiler bug again.)

I may try some older stable mingw-w64 builds and previous clang versions if I have time, but meh, I've already wasted enough time with these. Perhaps trying with the official clang (MSVC) build would make more sense, but I don't want to touch that. Probably would be another pile of issues.

FYR this looks like to be related to the compiler bug I was getting: https://llvm.org/bugs/show_bug.cgi?id=27966

boudewijn wrote:The big slow-down is the ntfs file system. It's a bit better on an SSD, but it's a seriously slow file system and there are about 3000 c++ files that need to be compiled.

Any real-time antivirus scanners (including Microsoft Defender/MSE) could also be a factor. You can try adding the build directories to the exclusion list.

Hm... so it is. mingw32-make does support multiple cores (using the -jN option). I read somewhere that it doesn't, but then I didn't read that it was due to some bugs with qmake - well, Krita uses cmake, so that is not an issue.

I also added the ignore directories to windows defender as suggested. Turns out that it was also actively eating up computational resource during compilation (ugh). It speeds it up a bit I guess (didn't measure), but still way behind linux compile times. I'm doubtful using clang would be that much faster, and judging by the amount of hassle you went through to no avail I'd rather not try.

The ideal solution would be to have Microsoft officially support ext4 and have windows installed there, but that would be wishful thinking.

Thanks!

Well, the build system continuously spawning new processes (recursive mingw32-make, g++, cmd) doesn't help either. Starting new processes on Windows is way slower than fork on *nix. I doubt using another filesystem will change much.
Just for kicks, perhaps try white-listing the gcc files (bin/lib/include/etc.) too?

If you want speed you probably should try cross-compiling on Linux. I've never touched that stuff with Krita though, and, boud couldn't make it work properly with the new vc.

Just FYR, I compile on a laptop with ivy bridge i5, 8GB of RAM and on an external USB 3.0 hard drive. It's not fun.
(I might also have to do a speed comparison of TDM-GCC 5.1 and a newer versions of mingw-w64 5.4, perhaps the newer would compile quicker?)

GUYS. THIS IS REVELATIONAL. I got some huge build speed increase using ninja (https://github.com/ninja-build/ninja/releases)

I did some update build tests to compare the speed. Here's what I'm using:

• Windows 10 64-bit, i5-3337U (1.8GHz), 8GB RAM
• Jom v1.1.0
• Ninja v1.7.1
• mingw32-make - TDM-GCC 5.1.0

So all I did was add a redundant line in one file and update the build (Krita 3.0.2 alpha). Bear in mind these a rough numbers since I only ran it twice.

elapsed times:

• jom install : 02:28
• mingw32-make install -j5 : 01:30
• ninja install -j5 : 00:15

Duuuuude. 15 seconds?? That's even faster the building in linux. This is way too awesome guys. I'm not even sure if it is actually updating the build or just printing pretty messages, so please try it out also to validate my results.

Drawbacks!! I'm using Qt Creator 4.1.0 and it hard coded the configure command to only use MinGW build files. This makes my ninja build incompatible with Qt Creator. So right now I have 2 builds: mingw and ninja, and I can only open Krita source code using the mingw build. So what I did was used a custom process step in "build steps" , but doing so error messages doesn't show up in the "Issues" tab in Qt Creator. I had to check the compile log. That sucks, but I would still use ninja anyway.

Update: My bad. Had to change the cmake generator to "CodeBlocks - Ninja" in my Kits in Qt Creator 4.1.0. I also noticed that the cmake configure step (runs everytime I load the project in Qt Creator) is also noticeably faster with Ninja compared to MinGW. Error and warning messages still doesn't show up in the Issues pane though (it's not that bad I mean, I can still search the compile log). Update: I wipe out the build and configuration files, reconfigure and rebuild and now the errors show up in the Issues Pane again.

Yeah, ninja is much faster. I have seen it not update some targets properly on some kind of changes though, so if you see errors like that, it could be ninja.

@Scummos Ah, I'll keep that in mind.

There's another thing I'm still curious though: how did the Krita binaries in the distribution release (the binaries that get extracted from the installer) become so small?

I have set the 3rdParty libraries to optimize for size in krita/3rdparty/CMakeLists.txt:

Code: Select all

set(GLOBAL_BUILD_TYPE MinSizeRel)

It definitely shrinks the size, but it is still no where as small as the distributed binaries.

It also had me wondering why "RelWithDebInfo" is the default option. The binaries it output are really big (up to 10 times or more) relative to the Release option. Is debugging information really needed for 3rdparty libraries?

Actually I'm not sure if the size of the binaries caused the slow down. Is it possible with the right tweaks, that the performance of the development version match the distributed release in windows? It definitely felt that way in linux. I just wish it could work the same for windows also. But you guys would know best.

Bruceoutdoors wrote:@Scummos Ah, I'll keep that in mind.

There's another thing I'm still curious though: how did the Krita binaries in the distribution release (the binaries that get extracted from the installer) become so small?

[...]

It also had me wondering why "RelWithDebInfo" is the default option. The binaries it output are really big (up to 10 times or more) relative to the Release option. Is debugging information really needed for 3rdparty libraries?

Actually I'm not sure if the size of the binaries caused the slow down. Is it possible with the right tweaks, that the performance of the development version match the distributed release in windows? It definitely felt that way in linux. I just wish it could work the same for windows also. But you guys would know best.

The release binaries are stripped during packaging. We build with `RelWithDebInfo` because we also publish builds with debug info for typical users to help get stack traces.

The binary size would certainly affect the loading speed. Bigger files = more time to read from disk. Though I might try to work on a way to split the debug info into its own files.

Bruceoutdoors wrote:I have set the 3rdParty libraries to optimize for size in krita/3rdparty/CMakeLists.txt:

Code: Select all

set(GLOBAL_BUILD_TYPE MinSizeRel)

It definitely shrinks the size, but it is still no where as small as the distributed binaries.

AFAIK `MinSizeRel` uses gcc's -Os flag. It will optimize for smaller code but with performance tradeoffs, I would certainly not want that for Krita.

To add to that, "packaging\windows\package.bat" is the packaging script for release, though I am trying to optimize it a bit more, which may or may not be merged in the future.

@alvinwong hm. I'd understand if "RelWithDebInfo" was set for the build of Krita itself, but the 3rd party libraries?

Also another question (this may be a bit off topic), as a developer how did you set your build type in Krita to "Debug" in windows? The default build type "RelWithDebInfo" will optimize out certain info like the value of QStrings and other stuff (in GDB it just shows <optimized out>). I'm not sure if setting CMAKE_BUILD_TYPE in Krita's build configuration does change the build type (setting CMAKE_BUILD_TYPE in 3rd party libraries is actually a redundant step; unless you modify CMakeLists.txt it'll always use RelWithDebInfo).

In linux what I do is change a line in KRITA_SOURCE_ROOT/CMakeLists.txt:

Code: Select all

set(CMAKE_CXX_FLAGS_KRITADEVS "-O3 -g" CACHE STRING "" FORCE)

to

Code: Select all

set(CMAKE_CXX_FLAGS_KRITADEVS "-O0 -g" CACHE STRING "" FORCE)

but that same hack doesn't work in windows (or at least I think it didn't work...). I tried explicitly defining CMAKE_CXX_FLAGS during configuration but it also doesn't seem to do anything.

I saw in another krita build guide using Emerge (https://community.kde.org/Krita/Emergebuild), in configuring Krita they set the option "-DCMAKE_BUILD_TYPE=Debug". Why is it in the guide in Krita's git repo is it "-DCMAKE_BUILD_TYPE=RelWithDebInfo"?

AFAIK `MinSizeRel` uses gcc's -Os flag. It will optimize for smaller code but with performance tradeoffs, I would certainly not want that for Krita.

Ah good point. I won't use that in that case. It still doesn't explain how the distribution release could become even smaller than MinSizeRel though.

UPDATE: I just checked out the packaging script "packaging\windows\package.bat". So, turns out that it is the strip command that drastically reduces the size of these binaries! Running

Code: Select all

strip *.dll

On the RelWithDebInfo binaries actually shrinks the size waaaay down, and matches the size in the distributed release binaries. There's really little need to use Release build after all. What's more surprising is that stripped binaries of RelWithDebInfo builds are smaller that Release builds.

Man. I never know how long it would have took me if I just went about trying to figure it out myself. Thanks @alvinwong!

Well, Release will compile faster than RelWithDebInfo, so if you strip them anyways, use Release.

-Os (MinSizeRel) is not useful outside microcontrollers imo.