Status report: Week 1

Hey all! This is my first report of the project’s Coding Period.

The project’s objectives for this week are:

define the new generator
build SeExpr
and try calling it from within Krita

I had also promised in the previous post to:

dissect SeExpr
write up a list of the supported libraries in each OS.

Krita’s library support state

I’ve dissected each platform’s build scripts into this Google Docs sheet. This piece of work was done against the 4.3 branch, not taking into account some fixes I wrote for the AppImage.

SeExpr prototype

I’m glad to say, I’ve been mostly successful!

Windows and macOS: success! Linux, not so much...

The new generator

0638ce85 introduces a new type of layer generator identified like its namesake, seexpr.

File format-wise, it’s expressed in the manifest as a layer of type generatorlayer:

<layer opacity="255" channelflags="" locked="0" filename="layer2" uuid="{1b9dbcc4-7dbc-4c57-95c5-3b3353ce0eac}" selected="true" generatorname="seexpr" x="0" compositeop="normal" nodetype="generatorlayer" colorlabel="0" y="0" name="Layer 3" visible="1" collapsed="0" intimeline="1" generatorversion="1"/>

while its script is stored in the layer’s configuration file (layers/layer2.filterconfig):

<!DOCTYPE params>
<params version="1">
 <param name="script" type="string"><![CDATA[$val=voronoi(5*[$u,$v,.5],4,.6,.2); 
 $color=ccurve($val,
    0.000, [0.141, 0.059, 0.051], 4,
    0.185, [0.302, 0.176, 0.122], 4,
    0.301, [0.651, 0.447, 0.165], 4,
    0.462, [0.976, 0.976, 0.976], 4);
$color
]]></param>
</params>

Code-wise, the new generator is a mixture of:

a port of Disney’s initialization code, adapted to use Qt’s types (QMap and sanity assertions),
the existing Simplex Noise generator, which I used as a basis to understand how to get its configuration and report progress.

A quick dissection of SeExpr

The library itself can be accessed with two bits of code. The first has been named SeExprExpressionContext; it’s a subclass of SeExpr’s Expression class, adapted to provide four variables so far:

u and v are the current pixel’s normalized, centerpoint coordinates;
w and h are the image’s width and height.

The SeExprExpressionContext can be initialized directly, with just the string that’s retrieved from the configuration. On each iteration, I get the pixel’s coordinates, update them in the expression context, and evaluate the expression. And since the library outputs floating-point normalized RGB, I ship these values directly as a instance of Qt’s QColor.

void KisSeExprGenerator::generate(KisProcessingInformation dstInfo,
                                  const QSize &size,
                                  const KisFilterConfigurationSP config,
                                  KoUpdater *progressUpdater) const
{
    KisPaintDeviceSP device = dstInfo.paintDevice();

    Q_ASSERT(!device.isNull());
    Q_ASSERT(config);

    if (config)
    {
        QString script = config->getString("script");

        QRect bounds = QRect(dstInfo.topLeft(), size);
        const KoColorSpace *cs = device->colorSpace();
        KisSequentialIteratorProgress it(device, bounds, progressUpdater);

        SeExprExpressionContext expression(script);

        expression.m_vars["u"] = new SeExprVariable();
        expression.m_vars["v"] = new SeExprVariable();
        expression.m_vars["w"] = new SeExprVariable(bounds.width());
        expression.m_vars["h"] = new SeExprVariable(bounds.height());

        if (expression.isValid() && expression.returnType().isFP(3))
        {
            double pixel_stride_x = 1. / bounds.width();
            double pixel_stride_y = 1. / bounds.height();
            double &u = expression.m_vars["u"]->m_value;
            double &v = expression.m_vars["v"]->m_value;

            while(it.nextPixel())
            {
                u = pixel_stride_x * (it.x() + .5);
                v = pixel_stride_y * (it.y() + .5);

                const qreal* value = expression.evalFP();
                QColor color;

                // SeExpr already outputs normalized RGB
                color.setRedF(value[0]);
                color.setGreenF(value[1]);
                color.setBlueF(value[2]);

                cs->fromQColor(color, it.rawData());
            }
        }
    }
}

The Linux problem

It works really nice, under Windows and macOS. Linux, however…

Under Linux, SeExpr’s interpreter doesn’t work correctly when invoked from another namespace. Inside Krita, it truncates all floating-point values in its internal state, as shown from this dump (look at the fp section):

Parse tree desired type lifetime_error Float[3] actual varying Float[3]
  '$val=voronoi(5*[$u,$v,.5],4,.6,.2);
$color=ccurve($val,
    0.000, [0.141, 0.059, 0.051], 4, 
    0.185, [0.302, 0.176, 0.122], 4, 
    0.301, [0.651, 0.447, 0.165], 4,  
    0.462, [0.976, 0.976, 0.976], 4);
$color' N7SeExpr214ExprModuleNodeE type=varying Float[3]
    '$val=voronoi(5*[$u,$v,.5],4,.6,.2);
$color=ccurve($val,
    0.000, [0.141, 0.059, 0.051], 4, 
    0.185, [0.302, 0.176, 0.122], 4, 
    0.301, [0.651, 0.447, 0.165], 4,  
    0.462, [0.976, 0.976, 0.976], 4);
$color' N7SeExpr213ExprBlockNodeE type=varying Float[3]
      '$val=voronoi(5*[$u,$v,.5],4,.6,.2);' N7SeExpr28ExprNodeE type=varying None
        '$val=voronoi(5*[$u,$v,.5],4,.6,.2);' N7SeExpr214ExprAssignNodeE type=varying None
          'voronoi(5*[$u,$v,.5],4,.6,.2)' N7SeExpr212ExprFuncNodeE type=varying Float[3]
            '5*[$u,$v,.5]' N7SeExpr216ExprBinaryOpNodeE type=varying Float[3]
              '5' N7SeExpr211ExprNumNodeE type=constant Float
              '[$u,$v,.5]' N7SeExpr211ExprVecNodeE type=varying Float[3]
                '$u' N7SeExpr211ExprVarNodeE type=varying Float
                '$v' N7SeExpr211ExprVarNodeE type=varying Float
                '.5' N7SeExpr211ExprNumNodeE type=constant Float
            '4' N7SeExpr211ExprNumNodeE type=constant Float
            '.6' N7SeExpr211ExprNumNodeE type=constant Float
            '.2' N7SeExpr211ExprNumNodeE type=constant Float
        '$color=ccurve($val,
    0.000, [0.141, 0.059, 0.051], 4, 
    0.185, [0.302, 0.176, 0.122], 4, 
    0.301, [0.651, 0.447, 0.165], 4,  
    0.462, [0.976, 0.976, 0.976], 4);' N7SeExpr214ExprAssignNodeE type=varying None
          'ccurve($val,
    0.000, [0.141, 0.059, 0.051], 4, 
    0.185, [0.302, 0.176, 0.122], 4, 
    0.301, [0.651, 0.447, 0.165], 4,  
    0.462, [0.976, 0.976, 0.976], 4)' N7SeExpr212ExprFuncNodeE type=varying Float[3]
            '$val' N7SeExpr211ExprVarNodeE type=varying Float[3]
            '0.000' N7SeExpr211ExprNumNodeE type=constant Float
            '[0.141, 0.059, 0.051]' N7SeExpr211ExprVecNodeE type=constant Float[3]
              '0.141' N7SeExpr211ExprNumNodeE type=constant Float
              '0.059' N7SeExpr211ExprNumNodeE type=constant Float
              '0.051' N7SeExpr211ExprNumNodeE type=constant Float
            '4' N7SeExpr211ExprNumNodeE type=constant Float
            '0.185' N7SeExpr211ExprNumNodeE type=constant Float
            '[0.302, 0.176, 0.122]' N7SeExpr211ExprVecNodeE type=constant Float[3]
              '0.302' N7SeExpr211ExprNumNodeE type=constant Float
              '0.176' N7SeExpr211ExprNumNodeE type=constant Float
              '0.122' N7SeExpr211ExprNumNodeE type=constant Float
            '4' N7SeExpr211ExprNumNodeE type=constant Float
            '0.301' N7SeExpr211ExprNumNodeE type=constant Float
            '[0.651, 0.447, 0.165]' N7SeExpr211ExprVecNodeE type=constant Float[3]
              '0.651' N7SeExpr211ExprNumNodeE type=constant Float
              '0.447' N7SeExpr211ExprNumNodeE type=constant Float
              '0.165' N7SeExpr211ExprNumNodeE type=constant Float
            '4' N7SeExpr211ExprNumNodeE type=constant Float
            '0.462' N7SeExpr211ExprNumNodeE type=constant Float
            '[0.976, 0.976, 0.976]' N7SeExpr211ExprVecNodeE type=constant Float[3]
              '0.976' N7SeExpr211ExprNumNodeE type=constant Float
              '0.976' N7SeExpr211ExprNumNodeE type=constant Float
              '0.976' N7SeExpr211ExprNumNodeE type=constant Float
            '4' N7SeExpr211ExprNumNodeE type=constant Float
      '$color' N7SeExpr211ExprVarNodeE type=varying Float[3]
Eval strategy is interpreter
---- ops     ----------------------
    (null) 0x7ffff7eff210 ( 2 4)
    (null) 0x7ffff7eff210 ( 3 5)
    (null) 0x7ffff7f05d50 ( 4 5 6 7)
    _ZN7SeExpr27PromoteILi3EE1fEPiPdPPcRSt6vectorIiSaIiEE 0x7ffff7edbf90 ( 3 10)
    (null) 0x7ffff7eef900 ( 10 7 13)
    _ZN7SeExpr214ExprFuncSimple6EvalOpEPiPdPPcRSt6vectorIiSaIiEE 0x7ffff7edc5f0 ( 4 5 20 19 13 16 17 18)
    (null) 0x7ffff7f04b40 ( 20 0)
    (null) 0x7ffff7f05d50 ( 27 28 29 30)
    (null) 0x7ffff7f05d50 ( 35 36 37 38)
    (null) 0x7ffff7f05d50 ( 43 44 45 46)
    (null) 0x7ffff7f05d50 ( 51 52 53 54)
    _ZN7SeExpr214ExprFuncSimple6EvalOpEPiPdPPcRSt6vectorIiSaIiEE 0x7ffff7edc5f0 ( 6 7 59 58 0 26 30 33 34 38 41 42 46 49 50 54 57)
    (null) 0x7ffff7f04b40 ( 59 23)
---- opdata  ----------------------
opData[0]= 2
opData[1]= 4
opData[2]= 3
opData[3]= 5
opData[4]= 4
opData[5]= 5
opData[6]= 6
opData[7]= 7
opData[8]= 3
opData[9]= 10
opData[10]= 10
opData[11]= 7
opData[12]= 13
opData[13]= 4
opData[14]= 5
opData[15]= 20
opData[16]= 19
opData[17]= 13
opData[18]= 16
opData[19]= 17
opData[20]= 18
opData[21]= 20
opData[22]= 0
opData[23]= 27
opData[24]= 28
opData[25]= 29
opData[26]= 30
opData[27]= 35
opData[28]= 36
opData[29]= 37
opData[30]= 38
opData[31]= 43
opData[32]= 44
opData[33]= 45
opData[34]= 46
opData[35]= 51
opData[36]= 52
opData[37]= 53
opData[38]= 54
opData[39]= 6
opData[40]= 7
opData[41]= 59
opData[42]= 58
opData[43]= 0
opData[44]= 26
opData[45]= 30
opData[46]= 33
opData[47]= 34
opData[48]= 38
opData[49]= 41
opData[50]= 42
opData[51]= 46
opData[52]= 49
opData[53]= 50
opData[54]= 54
opData[55]= 57
opData[56]= 59
opData[57]= 23
----- fp --------------------------
fp[0]= 0
fp[1]= 0
fp[2]= 0
fp[3]= 5
fp[4]= 0
fp[5]= 0
fp[6]= 0.5
fp[7]= 0
fp[8]= 0
fp[9]= 0.5
fp[10]= 5
fp[11]= 5
fp[12]= 5
fp[13]= 0
fp[14]= 0
fp[15]= 2.5
fp[16]= 4
fp[17]= 0.6
fp[18]= 0.2
fp[19]= 4
fp[20]= 0
fp[21]= 0
fp[22]= 0
fp[23]= 0
fp[24]= 0
fp[25]= 0
fp[26]= 0
fp[27]= 0.141
fp[28]= 0.059
fp[29]= 0.051
fp[30]= 0.141
fp[31]= 0.059
fp[32]= 0.051
fp[33]= 4
fp[34]= 0.185
fp[35]= 0.302
fp[36]= 0.176
fp[37]= 0.122
fp[38]= 0.302
fp[39]= 0.176
fp[40]= 0.122
fp[41]= 4
fp[42]= 0.301
fp[43]= 0.651
fp[44]= 0.447
fp[45]= 0.165
fp[46]= 0.651
fp[47]= 0.447
fp[48]= 0.165
fp[49]= 4
fp[50]= 0.462
fp[51]= 0.976
fp[52]= 0.976
fp[53]= 0.976
fp[54]= 0.976
fp[55]= 0.976
fp[56]= 0.976
fp[57]= 4
fp[58]= 13
fp[59]= 0
fp[60]= 0
fp[61]= 0
---- str     ----------------------
s[0] reserved for datablock = 0
s[1] is indirectIndex = 0
s[2]= 0x��UUUU '��UU...'
s[3]= 0x��UUUU '��UU...'
s[4]= 0x�)��� '�)��...'
s[5]= 0x�)��� '�)��...'
s[6]= 0x�*��� '�*��...'
s[7]= 0x +��� ' +��...'
ending with isValid 1
parse error

Parse tree desired type lifetime_error Float[3] actual varying Float[3]
  '$val=voronoi(5*[$u,$v,.5],4,.6,.2); 
 $color=ccurve($val,
    0.000, [0.141, 0.059, 0.051], 4,
    0.185, [0.302, 0.176, 0.122], 4,
    0.301, [0.651, 0.447, 0.165], 4,
    0.462, [0.976, 0.976, 0.976], 4);
$color' N7SeExpr214ExprModuleNodeE type=varying Float[3]
    '$val=voronoi(5*[$u,$v,.5],4,.6,.2); 
 $color=ccurve($val,
    0.000, [0.141, 0.059, 0.051], 4,
    0.185, [0.302, 0.176, 0.122], 4,
    0.301, [0.651, 0.447, 0.165], 4,
    0.462, [0.976, 0.976, 0.976], 4);
$color' N7SeExpr213ExprBlockNodeE type=varying Float[3]
      '$val=voronoi(5*[$u,$v,.5],4,.6,.2);' N7SeExpr28ExprNodeE type=varying None
        '$val=voronoi(5*[$u,$v,.5],4,.6,.2);' N7SeExpr214ExprAssignNodeE type=varying None
          'voronoi(5*[$u,$v,.5],4,.6,.2)' N7SeExpr212ExprFuncNodeE type=varying Float[3]
            '5*[$u,$v,.5]' N7SeExpr216ExprBinaryOpNodeE type=varying Float[3]
              '5' N7SeExpr211ExprNumNodeE type=constant Float
              '[$u,$v,.5]' N7SeExpr211ExprVecNodeE type=varying Float[3]
                '$u' N7SeExpr211ExprVarNodeE type=varying Float
                '$v' N7SeExpr211ExprVarNodeE type=varying Float
                '.5' N7SeExpr211ExprNumNodeE type=constant Float
            '4' N7SeExpr211ExprNumNodeE type=constant Float
            '.6' N7SeExpr211ExprNumNodeE type=constant Float
            '.2' N7SeExpr211ExprNumNodeE type=constant Float
        '$color=ccurve($val,
    0.000, [0.141, 0.059, 0.051], 4,
    0.185, [0.302, 0.176, 0.122], 4,
    0.301, [0.651, 0.447, 0.165], 4,
    0.462, [0.976, 0.976, 0.976], 4);' N7SeExpr214ExprAssignNodeE type=varying None
          'ccurve($val,
    0.000, [0.141, 0.059, 0.051], 4,
    0.185, [0.302, 0.176, 0.122], 4,
    0.301, [0.651, 0.447, 0.165], 4,
    0.462, [0.976, 0.976, 0.976], 4)' N7SeExpr212ExprFuncNodeE type=varying Float[3]
            '$val' N7SeExpr211ExprVarNodeE type=varying Float[3]
            '0.000' N7SeExpr211ExprNumNodeE type=constant Float
            '[0.141, 0.059, 0.051]' N7SeExpr211ExprVecNodeE type=constant Float[3]
              '0.141' N7SeExpr211ExprNumNodeE type=constant Float
              '0.059' N7SeExpr211ExprNumNodeE type=constant Float
              '0.051' N7SeExpr211ExprNumNodeE type=constant Float
            '4' N7SeExpr211ExprNumNodeE type=constant Float
            '0.185' N7SeExpr211ExprNumNodeE type=constant Float
            '[0.302, 0.176, 0.122]' N7SeExpr211ExprVecNodeE type=constant Float[3]
              '0.302' N7SeExpr211ExprNumNodeE type=constant Float
              '0.176' N7SeExpr211ExprNumNodeE type=constant Float
              '0.122' N7SeExpr211ExprNumNodeE type=constant Float
            '4' N7SeExpr211ExprNumNodeE type=constant Float
            '0.301' N7SeExpr211ExprNumNodeE type=constant Float
            '[0.651, 0.447, 0.165]' N7SeExpr211ExprVecNodeE type=constant Float[3]
              '0.651' N7SeExpr211ExprNumNodeE type=constant Float
              '0.447' N7SeExpr211ExprNumNodeE type=constant Float
              '0.165' N7SeExpr211ExprNumNodeE type=constant Float
            '4' N7SeExpr211ExprNumNodeE type=constant Float
            '0.462' N7SeExpr211ExprNumNodeE type=constant Float
            '[0.976, 0.976, 0.976]' N7SeExpr211ExprVecNodeE type=constant Float[3]
              '0.976' N7SeExpr211ExprNumNodeE type=constant Float
              '0.976' N7SeExpr211ExprNumNodeE type=constant Float
              '0.976' N7SeExpr211ExprNumNodeE type=constant Float
            '4' N7SeExpr211ExprNumNodeE type=constant Float
      '$color' N7SeExpr211ExprVarNodeE type=varying Float[3]
Eval strategy is interpreter
---- ops     ----------------------
    (null) 0x7fffe42c6210 ( 2 4)
    (null) 0x7fffe42c6210 ( 3 5)
    (null) 0x7fffe42ccd50 ( 4 5 6 7)
    _ZN7SeExpr27PromoteILi3EE1fEPiPdPPcRSt6vectorIiSaIiEE 0x7fffe42a2f90 ( 3 10)
    (null) 0x7fffe42b6900 ( 10 7 13)
    _ZN7SeExpr214ExprFuncSimple6EvalOpEPiPdPPcRSt6vectorIiSaIiEE 0x7fffe42a35f0 ( 4 5 20 19 13 16 17 18)
    (null) 0x7fffe42cbb40 ( 20 0)
    (null) 0x7fffe42ccd50 ( 27 28 29 30)
    (null) 0x7fffe42ccd50 ( 35 36 37 38)
    (null) 0x7fffe42ccd50 ( 43 44 45 46)
    (null) 0x7fffe42ccd50 ( 51 52 53 54)
    _ZN7SeExpr214ExprFuncSimple6EvalOpEPiPdPPcRSt6vectorIiSaIiEE 0x7fffe42a35f0 ( 6 7 59 58 0 26 30 33 34 38 41 42 46 49 50 54 57)
    (null) 0x7fffe42cbb40 ( 59 23)
---- opdata  ----------------------
opData[0]= 2
opData[1]= 4
opData[2]= 3
opData[3]= 5
opData[4]= 4
opData[5]= 5
opData[6]= 6
opData[7]= 7
opData[8]= 3
opData[9]= 10
opData[10]= 10
opData[11]= 7
opData[12]= 13
opData[13]= 4
opData[14]= 5
opData[15]= 20
opData[16]= 19
opData[17]= 13
opData[18]= 16
opData[19]= 17
opData[20]= 18
opData[21]= 20
opData[22]= 0
opData[23]= 27
opData[24]= 28
opData[25]= 29
opData[26]= 30
opData[27]= 35
opData[28]= 36
opData[29]= 37
opData[30]= 38
opData[31]= 43
opData[32]= 44
opData[33]= 45
opData[34]= 46
opData[35]= 51
opData[36]= 52
opData[37]= 53
opData[38]= 54
opData[39]= 6
opData[40]= 7
opData[41]= 59
opData[42]= 58
opData[43]= 0
opData[44]= 26
opData[45]= 30
opData[46]= 33
opData[47]= 34
opData[48]= 38
opData[49]= 41
opData[50]= 42
opData[51]= 46
opData[52]= 49
opData[53]= 50
opData[54]= 54
opData[55]= 57
opData[56]= 59
opData[57]= 23
----- fp --------------------------
fp[0]= 0
fp[1]= 0
fp[2]= 0
fp[3]= 5
fp[4]= 0
fp[5]= 0
fp[6]= 0
fp[7]= 0
fp[8]= 0
fp[9]= 0
fp[10]= 5
fp[11]= 5
fp[12]= 5
fp[13]= 0
fp[14]= 0
fp[15]= 0
fp[16]= 4
fp[17]= 0
fp[18]= 0
fp[19]= 4
fp[20]= 0
fp[21]= 0
fp[22]= 0
fp[23]= 0
fp[24]= 0
fp[25]= 0
fp[26]= 0
fp[27]= 0
fp[28]= 0
fp[29]= 0
fp[30]= 0
fp[31]= 0
fp[32]= 0
fp[33]= 4
fp[34]= 0
fp[35]= 0
fp[36]= 0
fp[37]= 0
fp[38]= 0
fp[39]= 0
fp[40]= 0
fp[41]= 4
fp[42]= 0
fp[43]= 0
fp[44]= 0
fp[45]= 0
fp[46]= 0
fp[47]= 0
fp[48]= 0
fp[49]= 4
fp[50]= 0
fp[51]= 0
fp[52]= 0
fp[53]= 0
fp[54]= 0
fp[55]= 0
fp[56]= 0
fp[57]= 4
fp[58]= 13
fp[59]= 0
fp[60]= 0
fp[61]= 0
---- str     ----------------------
s[0] reserved for datablock = 0
s[1] is indirectIndex = 0
s[2]= 0x�41�� '�41�...'
s[3]= 0x�41�� '�41�...'
s[4]= 0x��/�� '��/�...'
s[5]= 0xؙ/�� 'ؙ/�...'
s[6]= 0xȚ/�� 'Ț/�...'
s[7]= 0x �/�� ' �/�...'
ending with isValid 1
parse error

Dumps of SeExpr's interpreter state of the Voronoi-lava program. On the left, the state as produced under the imageSynth2 demo program. On the right, the state as produced under Krita's generator. The fp section shows every relevant value has been truncated.

I must own that I don’t really know how to solve this yet. I’ve tested with a more direct port of the initialization (using std::map, const double instead of const real), but it still corrupts its internal state. I’ll file a question in Disney’s repo and see if they have any idea what could be going on.

What’s next?

In trying to make sure I could test everything, I covered four weeks in one (Week 1, 2, 3, and 7 😊) I’ll take things slower now, seeing that it (mostly) works, and have a good looks at the innards of SeExpr. I’ll also work on adding a test suite.

Ideas that I have? Probably adding a resource explorer (like the pattern’s), and adding syntax highlighting. SeExpr already provides dependency-free plugins to do this, but I don’t want to complicate things yet.

Until next time,

~amyspark