Skip to main content Skip to navigation

Status report: Community Bonding

Categories:

Good morning everyone!

I’m checking in today to let you know what I did in my GSoC project these past weeks. This Community Bonding period was really wonderful; although I’ve been more or less involved with the project since 2016, I’ve acquainted myself with the efforts of each of the members, and so far it’s been a wonderful experience.

During these past weeks, I’ve been preparing for the coding period by talking with Boudewijn and Wolthera about the particulars of Krita’s file format and build system. The objectives for the past two meetings were:

  • speccing the new layer
  • integrating SeExpr into Krita’s dependency build system
  • document the process

This post summarizes my efforts on each of these bits. I apologise for the late post; but this week I got to attend (virtually) my first Eurographics and LGM sessions, and with the homework load, it’s been some wild days.

Without further ado, let’s see how deep this rabbit hole goes.


Integration: success!

I’m glad to report that SeExpr can be successfully built as a dependency! Currently, the build process points to my own fork of SeExpr. This is for two reasons: first, for GSoC evaluation purposes, my mentors need to be notified of every commit I make (in this case, both in Krita and in the SeExpr library itself). But the other, and most important, is that SeExpr, as designed by Disney, is natively incompatible with our compilation toolchain in Windows. When Disney developers wrote the library, they made a critical assumption, that a Windows compilation would always be made with Visual C++. Also, since their parser is written with Bison and FLEX, they required Windows users to copy bundled, pre-generated files before being able to compile the library.

My fixes were four-fold:

With this process, I was able to get successful builds on both Windows and macOS. (There’s a bug that Iván fixed quickly in the osxbuild.sh script, but as it was easily worked around, I’ve yet to cherry-pick it into my branch.)

Building AppImages: oh my…

The process for Linux wasn’t so smooth, unfortunately. The build process I demoed in a previous post uses Manjaro’s system libraries, and since it’s rolling release distribution, it guarantees that almost no one will be able to run a demo. Since tiar usually asks users to test using AppImages, I decided to take that route, and attempted to build them… blindly.

I spent a whole weekend fixing the ensuing mess.

The AppLauncher mess

To begin with, Manjaro comes with an AppImage integration called AppLauncher that hooks into any AppImage that tries to run and helps you integrate it with your system. The bad news is that, as a system application, it uses the local version of Qt, 5.14. Why is this important?

Turns out, the build scripts use a packer, linuxdeployqt. I’d naïvely thought this was bundled with Qt (like macdeployqt and windeployqt), but it’s an AppImage on its own right. And since the build scripts patch LD_LIBRARY_PATH, in order for it to find our (heavily patched) Qt 5.12 distribution, I got very unhelpful crashes from AppLauncher. I uninstalled it once I recognized the Qt version mismatch, but it kept crying “file or directory not found”.

tl;dr: uninstall AppLauncher and reboot your system before using linuxdeployqt.

Packing… not

My Ryzen 7 box packs quite a punch so, thankfully, the above didn’t waste too much time. The real problem arose, when I noticed that linuxdeployqt automatically detects and refuses to run if your system is too new. This is to prevent incompatibilities between the AppImage and the version of the glibc library it needs.

I needed a way to isolate my Krita build and use a least common denominator distribution. Thanks to my best friend’s sysadmin tales, I knew I had a solution in the shape of a LXC container.

tl;dr: I installed LXD (the Snap version),

sudo snap install lxd

added my account to the lxd group to skip the sudo bits,

sudo usermod -a -G lxd amalia

After the customary reboot, I initialized LXD with all the defaults,

lxd init

created a new container (Ubuntu 18.04 is still too new for AppImages, but it helped to satisfy my learning needs),

lxc launch ubuntu:18.04 krita -c security.privileged true

bound the ~/krita folder as a writable device (which is why I need security.privileged),

lxc config device add krita src disk source=$HOME/krita path=/krita

and relaunched the container:

lxc stop krita
lxc start krita

With the container set up, I could start building the dependencies… But wait, what packages do we actually need?

Whack-a-Qt

Yes, this step felt like another whack-a-mole of mistakes. The dependencies’ build script is notoriously unhelpful:

# Prerequisites: cmake git build-essential libxcb-keysyms1-dev plus all deps for Qt5

After a whole day of getting complaints from each step, and installing the required packages, I managed to determine exactly which dependencies are needed for Krita. For the build process in general,

sudo apt-get install build-essential cmake libgl1-dev libglu1-mesa-dev

and additionally, for the KDE libraries,

sudo apt-get install libxcb-keysyms1-dev libxcb-res0-dev libxrender-dev libxkbcommon-dev

But there are hidden dependencies that aren’t listed anywhere, perhaps because we bundle them in 3rdparty yet we do not use them in Linux. The GMic plugin needs:

sudo apt-get install libtiff5-dev

The exiv2 library needs:

sudo apt-get install zlib1g-dev

And, finally, Qt itself:

sudo apt-get install libssl-dev gettext libfontconfig1-dev

With the dependencies installed, I added the two remaining utilities, linuxdeployqt and patchelf:

wget -c -nv "https://github.com/probonopd/linuxdeployqt/releases/download/continuous/linuxdeployqt-continuous-x86_64.AppImage" -O /usr/local/bin/linuxdeployqt
chmod a+x /usr/local/bin/linuxdeployqt

cd /tmp/
wget  -c -nv https://nixos.org/releases/patchelf/patchelf-0.9/patchelf-0.9.tar.bz2
tar -xf patchelf-0.9.tar.bz2
cd patchelf-0.9
./configure -prefix=/usr/local
make -j4 install
cd ~
rm -rf /tmp/patchelf-*

By this time, I had realised there was an official KDE repo listing everything I needed to set up such a container, sysadmin/ci-tooling. You can save time and headaches by pulling the kdeorg/appimage-ubuntu1604 Docker image, or by using Dmitry’s scripts.

Dmitry and Agata asked:

  • why I don’t follow the Docker route
  • how I’d code and debug Krita under this process
  • and whether there was any difference between Docker and LXC.

I’d like to clarify that the main development will be done under Windows, as that’s the most clean environment possible; it doesn’t ship Qt nor any other libraries that could interfere. For macOS, I have my MacBook Pro with Mojave. And for the AppImages, the LXC container resides in a separate SSD with the Manjaro install I run.

Finally, I prefer LXC to Docker because Docker (without e.g. docker-compose), is oriented to ephemeral containers, that need a lot of command line flags to set themselves up. I’m also too prone to docker image prune --all, and I wouldn’t want to download a multi-GB image each time I build Krita . And since the LXC container runs in the background, I can have it always up and pull the demonstration AppImage very easily – with just a copy-paste operation.

The new addition: a ‘SeExpr’ generator plugin

Two weeks ago, Wolthera sent me her MR for the KRA file format documentation. The gist of this is that each KRA file is a Zip container. It has a manifest in maindoc.xml listing, among other things, the component layers’ properties.

To the above, I’ll add a new layer type identified as SeExpr. My plan was to store the code in the layer node itself. In heavily simplified terms, it would look like this:

<layer nodetype="SeExpr">
<![CDATA[
$val=voronoi(5*[$u,$v,.5],4,.6,.2);
$color=ccurve($val,
    0.000, [0.141, 0.059, 0.051], 4, 
    0.185, [0.302, 0.176, 0.122], 4, 
    0.301, [0.651, 0.447, 0.165], 4,  
    0.462, [0.976, 0.976, 0.976], 4);
$color
]]>
</layer>

However, Boudewijn suggested I should instead follow the existing pattern and move the SeExpr rendering code to a separate file. This should make it easier to repair KRA files if they are corrupted. I have not done any further work on this, especially the UX bits; hopefully next week, I’ll be able to show you something

What’s next?

I have some more work to do as regards dependencies. I noticed Krita doesn’t maintain a list of the supported libraries for each operating system; for instance, the exiv2 library uses zlib, yet we don’t ship it in Linux. I’d like to sit down and document the whole list, and the necessary fixes for ensuring a consistent feature set. Next, I’ll be dissecting the SeExpr library, and why I elected to bundle only a limited set of features.

In the meanwhile, please chip in on David’s thread at Krita-Artists.org and let me know your expectations!

Until next time,

~amyspark