SeExpr

Image Editor Tutorial

## User Interface Tutorial

The user interface components included in the SeExpr distribution provide a useful way to visualize the results of your expression evaluations and speed up the process of building expressions. This tutorial uses some of the main UI components to build a simple app for editing and previewing expressions for image synthesis.

The finished example code can be found in ImageEditorDialog Class Reference.

See Simple ASCII Grapher Tutorial for getting started using SeExpr.

## Problem Overview: Image Synthesis

We'd like to be able to use a GUI expression editor to preview resulting images as we are building our expressions. For example, the evaluation of this:

    $val=voronoi(5*[$u,$v,.5],4,.6,.2);
    $color=ccurve($val,
        0.000, [0.141, 0.059, 0.051], 4,
        0.185, [0.302, 0.176, 0.122], 4,
        0.301, [0.651, 0.447, 0.165], 4,
        0.462, [0.976, 0.976, 0.976], 4);
    $color

looks like this:

In this tutorial, we are going to write an interface for doing image synthesis using expressions. This example uses the Qt Toolkit and SeExpr libraries. See referenced documentation for more detail.

The components of the interface will include:

- an expression editor
- a panel for control widgets
- an expression library browser
- an image previewer

## Main Dialog

We will first create a new ImageEditorDialog class based on the QDialog class:

    #include <QtGui/QDialog>

    class ImageEditorDialog: public QDialog
    {
    public:
        ImageEditorDialog(QWidget *parent=0);
    };

    ImageEditorDialog::ImageEditorDialog(QWidget *parent)
        :QDialog(parent)
    {
        this->setWindowTitle("Image Synthesis Editor");
    }

A simple main application will show the dialog:

    #include <QtGui/QApplication>
    #include "ImageEditorDialog.h"

    int main(int argc, char *argv[]){
        QApplication app(argc, argv);
        ImageEditorDialog *dialog = new ImageEditorDialog(0);
        dialog->show();
        app.exec();
        return 0;
    }

## Widget Controls and Text Editor

Now let's add some controls and an expression editor. The ExprEdControlCollection class provides the ability to add UI controls (sliders, ramps, etc.) for various types of variables (int, float, color, curves, etc.). The ExprEditor class provides the ability to manually edit the expression script.

We will include two new header files, in addition to the needed Qt header files:

    #include <QtGui/QVBoxLayout>
    #include <QtGui/QScrollArea>
    #include <ExprEdControlCollection.h>
    #include <ExprEditor.h>

and add a private member to the ImageEditorDialog class definition for the editor (we will use the editor's contents later to generate the preview image):

    private:
        ExprEditor *_editor;

Next we'll add 2 new components in the constructor and lay them out. The QScrollArea has specific parameters to properly display the controls. The ExprEditor constructor takes a pointer to the ExprEdControlCollection object, so it can connect signals between them:

        // Expression controls
        ExprEdControlCollection *controls = new ExprEdControlCollection();
        QScrollArea* scrollArea=new QScrollArea();
        scrollArea->setMinimumHeight(100);
        scrollArea->setFixedWidth(450);
        scrollArea->setWidgetResizable(true);
        scrollArea->setWidget(controls);

        // Expression editor
        _editor = new ExprEditor(this, controls);

        // Layout widgets
        QVBoxLayout *rootLayout = new QVBoxLayout();
        this->setLayout(rootLayout);
        rootLayout->addWidget(scrollArea);
        rootLayout->addWidget(_editor);

Already we can use the demo app to create some expressions.

Clicking the Add Widget button will give us a dialog to choose different variable types, with the corresponding expression output displayed in the editor:

For example, adding a color widget creates 3 sliders for RGB values which we can edit either from the control sliders or from the text editor:

## Expression Library Browser

Next let's add an expression library browser so we can look at example expressions and add our own. We will use the ExprEdBrowser class and connect it to the editor.

First, we will need an additional header:

    #include <ExprEdBrowser.h>

The browser reads from a config.txt file to locate expression files. An example can be found in ./build/src/demos/imageEditor/config.txt. We will create the browser in the ImageEditorDialog constructor:

        // Expression browser
        ExprEdBrowser *browser = new ExprEdBrowser(0, _editor);

        // Add user expressions, example expressions to browser list.
        browser->addUserExpressionPath("imageEditor");
    #ifdef IMAGE_EDITOR_ROOT
        std::string exPathStr = IMAGE_EDITOR_ROOT;
        exPathStr += "/share/SeExpr/expressions";
        browser->addPath("Examples", exPathStr);
    #else
        browser->addPath("Examples", "./src/demos/imageEditor");
    #endif
        browser->update();

and adjust the layout to make room for it:

        // Layout widgets: top section containing left and right, and bottom section
        QVBoxLayout *rootLayout = new QVBoxLayout();
        this->setLayout(rootLayout);

        QWidget* topWidget=new QWidget();
        QHBoxLayout* topLayout=new QHBoxLayout();
        topLayout->setContentsMargins(0,0,0,0);
        topWidget->setLayout(topLayout);

        QWidget *leftWidget=new QWidget();
        QVBoxLayout *leftLayout=new QVBoxLayout();
        leftLayout->setContentsMargins(0,0,0,0);
        leftWidget->setLayout(leftLayout);
        leftLayout->addWidget(scrollArea,1);

        QWidget *bottomWidget=new QWidget();
        QVBoxLayout *bottomLayout=new QVBoxLayout();
        bottomLayout->setContentsMargins(0,0,0,0);
        bottomWidget->setLayout(bottomLayout);

        topLayout->addWidget(leftWidget);
        topLayout->addWidget(browser,1);

        bottomLayout->addWidget(_editor);

        rootLayout->addWidget(topWidget);
        rootLayout->addWidget(bottomWidget);

We now have the ability to browse expression files. Selecting a file from the browser list will load its contents into the editor and create any associated widgets:

## Image Previewer

The last component is the image previewer. This borrows heavily from the imageSynth demo code.

We will use a QLabel with a pixmap image generated by some of the code from the imageSynth program.

First some additional headers:

    #include <string>
    #include <QtGui/QLabel>
    #include <QtGui/QImage>
    #include <QtGui/QMessageBox>

and another new data member in the ImageEditorDialog class for the image label:

    private:
        QLabel *_imageLabel;

We'll create the image label in the constructor and add it to the top left of the layout, above the controls (scrollArea):

        // Image Previewer
        _imageLabel = new QLabel();
        _imageLabel->setFixedSize(256,256);
        _imageLabel->setAlignment(Qt::AlignHCenter | Qt::AlignVCenter );
        QImage image("./src/doc/html/seexprlogo.png"); // just a fun default
        QPixmap imagePixmap = QPixmap::fromImage(image);
        imagePixmap = imagePixmap.scaled(256, 256, Qt::KeepAspectRatio);
        _imageLabel->setPixmap(imagePixmap);

        leftLayout->addWidget(_imageLabel);
        leftLayout->addWidget(scrollArea,1);

The layout should nowlook something like this:

To generate an image preview, we will want to evaluate the contents of the editor. We'll need to add an apply button that will evaluate the expression.

First, let's create some classes to handle the image synthesis. This code is a subset of the imageSynth demo code mentioned earlier. The differences are that the pixel order is slightly different for generating a QImage than for generating a PNG file, and the evaluateExpression() method returns a pointer to the image rather than writing the image to file.

    #include <png.h>
    #include <Expression.h>

    double clamp(double x){return std::max(0.,std::min(255.,x));}

    // Simple image synthesizer expression class to support demo image editor
    class ImageSynthExpression:public Expression
    {
    public:
        // Constructor that takes the expression to parse
        ImageSynthExpression(const std::string& expr)
            :Expression(expr)
        {}

        // Simple variable that just returns its internal value
        struct Var:public ExprVarRef
        {
            Var(const double val)
                : SeExpr2::ExprVarRef(SeExpr2::ExprType().FP(1).Varying()),
                  val(val) {}
            Var()
                : SeExpr2::ExprVarRef(SeExpr2::ExprType().FP(1).Varying()),
                  val(0.0) {}

            double val; // independent variable
            void eval(double* result){result[0]=val;}
            void eval(const char** result){assert(false);}
        };

        // variable map
        mutable std::map<std::string,Var> vars;

        // resolve function that only supports one external variable 'x'
        ExprVarRef* resolveVar(const std::string& name) const
        {
            std::map<std::string,Var>:iterator i=vars.find(name);
            if(i != vars.end()) return &i->second;
            return 0;
        }
    };

    class ImageSynthesizer
    {
    public:
        ImageSynthesizer();
        unsigned char *evaluateExpression(const std::string &exprStr);
    private:
        int _width;
        int _height;
    };

    ImageSynthesizer::ImageSynthesizer()
    {
        _width = 256;
        _height = 256;
    }

    unsigned char *ImageSynthesizer::evaluateExpression(const std::string &exprStr)
    {
        ImageSynthExpression expr(exprStr);

        // make variables
        expr.vars["u"]=ImageSynthExpression::Var(0.);
        expr.vars["v"]=ImageSynthExpression::Var(0.);
        expr.vars["w"]=ImageSynthExpression::Var(_width);
        expr.vars["h"]=ImageSynthExpression::Var(_height);

        // check if expression is valid
        bool valid=expr.isValid();
        if(!valid){
            std::cerr<<"Invalid expression "<<std::endl;
            std::cerr<<expr.parseError()<<std::endl;
            return NULL;
        }

        // evaluate expression
        std::cerr<<"Evaluating expression..."<<std::endl;
        unsigned char* image=new unsigned char[_width*_height*4];
        double one_over_width=1./_width,one_over_height=1./_height;
        double& u=expr.vars["u"].val;
        double& v=expr.vars["v"].val;
        unsigned char* pixel=image;
        for(int row=0;row<_height;row++){
            for(int col=0;col<_width;col++){
                u=one_over_width*(col+.5);
                v=one_over_height*(row+.5);
                Vec3d result=expr.evaluate();
                pixel[0]=clamp(result[0]*256.);
                pixel[1]=clamp(result[1]*256.);
                pixel[2]=clamp(result[2]*256.);
                pixel[3]=255;
                pixel+=4;
            }
        }

        return image;
    }

Now we're ready to create an apply button and add it to the bottom of the layout:

    #include <QtGui/QPushButton>
    #include <QtGui/QMessageBox>

        // Create apply button and connect to image preview.
        QPushButton *applyButton=new QPushButton("Apply");
        connect(applyButton, SIGNAL(clicked()), (ImageEditorDialog*)this, SLOT(applyExpression()));

        QWidget *buttonWidget=new QWidget();
        QHBoxLayout *buttonLayout = new QHBoxLayout(0);
        buttonWidget->setLayout(buttonLayout);
        buttonLayout->addWidget(applyButton);

        bottomLayout->addWidget(_editor);
        bottomLayout->addWidget(buttonWidget);

We will need to write the applyExpression() function to call the ImageSynthesizer object to evaluate the expression inside the editor. First, we need to include the Q_OBJECT macro in the ImageEditorDialog class definition, since we're going to be connecting signals and slots:

    class ImageEditorDialog: public QDialog
    {
        Q_OBJECT
        ...

Next, we'll add a new data member for the ImageSynthesizer instance as well as the applyExpression() function to the ImageEditorDialog class definition:

    private:
        ImageSynthesizer *_imageSynthesizer;
    private slots:
        void applyExpression();

and, initialize _imageSynthesizer in the constructor:

        _imageSynthesizer = new ImageSynthesizer();

Now we can write the function to evaluate the expression and generate a preview image:

    // Apply expression, if any, from the editor contents to the preview image
    void ImageEditorDialog::applyExpression()
    {
        std::string exprStr = _editor->getExpr();
        if( exprStr.empty() )
        {
            QMessageBox msgBox;
            msgBox.setText("No expression entered in the editor.");
            msgBox.exec();
        } else {
            QImage image(_imageSynthesizer->evaluateExpression(exprStr),
                         256,
                         256,
                         QImage::Format_RGB32);
            if( image.isNull() )
            {
                QMessageBox msgBox;
                msgBox.setText("Error evaluating expression to create preview image.");
                msgBox.exec();
            } else {
                QPixmap imagePixmap = QPixmap::fromImage(image);
                _imageLabel->setPixmap(imagePixmap);
            }
        }
    }

Finally, we'll have to pull out the ImageEditorDialog class definition into its own header, ImageEditorDialog.h, so the Qt MOC files will build correctly:

    #include <QtGui/QDialog>

    class QLabel;
    class ExprEditor;
    class ImageSynthesizer;

    class ImageEditorDialog: public QDialog
    {
        Q_OBJECT
    public:
        ImageEditorDialog(QWidget *parent=0);
    private:
        QLabel *_imageLabel;
        ExprEditor *_editor;
        ImageSynthesizer *_imageSynthesizer;
    private slots:
        void applyExpression();
    };

Now we can load expressions from a library, create new expressions manually as well as by creating new controls, and see all our changes in an image preview as we go:

A next simple step would be to add a Save button to save expressions from the editor to your own ~/imageEditor/expressions directory. The library browser will pick these up automatically, based on the context (imageEditor) and the hard-coded directory (expressions). This exercise is left to the reader to implement.