Optimize QScriptEngine repeated action

Question

I'm trying to optimize QScriptEngine operations in one of my functions.

The function is named executeCustomJSOperation and it executes the same JS code in multiple files. However each file needs to change a global variable named $xmlData. The function basicaly loads a XML file to memory using the $xmlData variable and then always apply the same javascript code (jsString) to edit this XML file using javascript. In the end the $xmlData variable is updated with the edited XML again.

I've got a 2.5 speedup using only an OpenMP parallel for over the for loop that processes each XML file. But now I don't know how to proceed to improve this function speed further.

The code is the following:

// allows user to echo js variables to check them in terminal using cout
QScriptValue echo(QScriptContext *context, QScriptEngine *engine)
{
    std::cout << context->argument(0).toString().toUtf8().constData() << std::endl; 
    return "";
}

void executeCustomJSOperation(const QString &jsString, const QStringList &filesToProcess){  
    QString rexmlString, jsxmlString;
    QFile rexmlfile(":/resources/libs/rexml.js"); // load javascript libraries as strings to memory
    QFile jsxmlfile(":/resources/libs/jsxml.js");

    rexmlfile.open(QFile::ReadOnly | QFile::Text);
    jsxmlfile.open(QFile::ReadOnly | QFile::Text);

    rexmlString=QTextStream(&rexmlfile).readAll();
    jsxmlString=QTextStream(&jsxmlfile).readAll();

    // Process all XmlFiles
#pragma omp parallel for // 2.5 speedup in my pc
    for(int i=0; i<filesToProcess.size(); i++){

        QString currXmlFileString;

        QScriptEngine engine;
        QScriptValue engineResult;

        // Add echo function so user can debug the code
        QScriptValue echoFunction = engine.newFunction(echo);
        engine.globalObject().setProperty("echo", echoFunction);

        engine.evaluate(rexmlString); // load js libraries in js engine
        engine.evaluate(jsxmlString);

        QFile currXmlFile(filesToProcess[i]);

        currXmlFileString=QTextStream(&currXmlFile).readAll();

        currXmlFile.close(); // close reading

        engine.globalObject().setProperty("$xmlData",currXmlFileString);

        engine.evaluate("main(); function main() {"+jsString+"}"); // main function allows to use return to exit prematurely from user code

        engineResult=engine.globalObject().property("$xmlData");

        QTextStream(&currXmlFile) << engineResult.toString(); // retreive the modified xml by javascript and save it to the file
    }
}

Do you think it is possible to optimize further this code? If you have any doubt please ask.

score 1 · Answer 1 · answered Feb 15 '14 at 22:02

1

You can construct a QScriptProgram, put all JS code in it and evaluate it using QScriptEngine::evaluate. It could speed up executing because parsing JS code will be done only once. However, QScriptProgram is not documented as reentrant or thread-safe, so you can't be sure that it will work correctly in multiple threads even if each thread uses its own QScriptProgram object.

answered Feb 15 '14 at 22:02

Pavel Strakhov

39,123
5
88
127

Hi, thank for the tip. I've tried it but I couldn't improve (the cpu) performance with it. Also as you said `QScriptProgram` isn't thread safe so I can't just use it with threads unless I use one for each thread which parses the js after its creation. I've also tried use the same objects (including qengine and qscriptprogram) for all threads and for this I've tried some synchronization methods like omp critical, qmutex and EnterCriticalSection but they slow down too much the program. The fastest implementation so far stills the one I posted where each thread have individual engines/variables. – RandomGuy Feb 15 '14 at 23:33
Try to create one `QScriptProgram` from JS code, then create several copies of it (which should be cheap) and distribute them to all threads. – Pavel Strakhov Feb 16 '14 at 01:25
Unfortunally seems that option doesn't work either with threads. I get segmentation fault when I use that method. Seems that the constructor `QScriptProgram::QScriptProgram(const QScriptProgram & other)` doesn't make a deep copy of the QScriptProgram. You can check the implemented code [here](http://pastebin.com/6n4urL0G). Note that this code include the complete code and not only the simplified one that I posted here (includes error checking for instance). – RandomGuy Feb 16 '14 at 13:14

score 1 · Accepted Answer · edited Nov 03 '14 at 20:23

1

Why are you creating / initializing a separate QScriptEngine for each iteration? I'd suggest moving everything up to your line

engine.evaluate(jsxmlString);

to outside the for()-loop.

True, this will make things more difficult WRT threading. Essentially you'd have to set up n worker threads, and create one script engine per thread (not per file). For starters a simple single threaded version should give you a first idea of what speedup to expect, and if that is worth the trouble.

Of course, if your JS code really is single use, only, QScriptProgram is your only hope of optimization. Again, you'd set up a limited number of worker threads, each with its own QScriptProgram (and one QScriptEngine per iteration, as in your current code).

edited Nov 03 '14 at 20:23

lennon310

12,503
11
43
61

answered Nov 03 '14 at 20:03

Tfry

194
6

Thanks! I haven't updated the question in a while with the changes that I have made with the time. I have wrote exactly like you suggested, I only create one `QScriptEngine` per thread and moved the redundant code out of the loop. I haven't made the measurements but I know is faster. I have tried the `QScriptProgram` tip but I haven't noticed any significant change in the performance, I'm allocating QScriptPrograms in a vector in the heap where each position is used by each thread. I also have tried the new `QJSEngine` which I believe uses `V8` JSengine but I have got much slower results. :| – RandomGuy Nov 09 '14 at 22:37

Optimize QScriptEngine repeated action

2 Answers2

Linked