How to use future / async in cppyy

Question

I'm trying to use future from C++ STL via cppyy (a C++-python binding packet).
For example, I could run this following code in C++ (which is adapted from this answer)

#include <future>
#include <thread>
#include <chrono>
#include <iostream>

using namespace std;
using namespace chrono_literals; 

int main () {
    promise<int> p;
    future<int> f = p.get_future();
    thread t([&p]() {
        this_thread::sleep_for(10s);
        p.set_value(2);
    });
    auto status = f.wait_for(10ms);
    if (status == future_status::ready) {
        cout << "task is read" << endl;
    } else {
        cout << "task is running" << endl;
    }
    t.join();
    return 0;
}

A similar implementation of the above in Python is

import cppyy

cppyy.cppdef(r'''
#include <future>
#include <thread>
#include <chrono>
#include <iostream>

using namespace std;

int test () {
    promise<int> p;
    future<int> f = p.get_future();
    thread t([&p]() {
        this_thread::sleep_for(10s);
        p.set_value(2);
    });
    auto status = f.wait_for(10ms);
    if (status == future_status::ready) {
        cout << "task is read" << endl;
    } else {
        cout << "task is running" << endl;
    }
    t.join();
    return 0;
}
''')


cppyy.gbl.test()

And the above code yields

IncrementalExecutor::executeFunction: symbol '__emutls_v._ZSt15__once_callable' unresolved while linking symbol '__cf_4'!
IncrementalExecutor::executeFunction: symbol '__emutls_v._ZSt11__once_call' unresolved while linking symbol '__cf_4'!

It looks like it's caused by using future in cppyy. Any solutions to this?

`cppyy` is more than a Python binding tool. It actually runs a C++ interpreter. Threading in the interpreter doesn't mix with threading in a compiled app. I believe this is the same problem. https://sft.its.cern.ch/jira/browse/ROOT-9277 — Tim Roberts, Aug 20 '22 at 04:31
@TimRoberts Yes, indeed. I also tried to use `async` in cppyy and got the same error message. So, do you think this is a problem that cppyy needs to solve? Or, I need to change my code? — Max Wong, Aug 20 '22 at 04:36
I think the answer is, you can't use `cppyy` with `async` C++ code. There are many other ways to hook up Python and C++, including the boost Python code. It's not as automatic as `cppyy`, but it will let you do what you want. — Tim Roberts, Aug 20 '22 at 06:08

score 2 · Accepted Answer · answered Aug 20 '22 at 15:29

2

Clang9's JIT does not support thread local storage the way the modern g++ implements it, will check again when the (on-going) upgrade to Clang13 is finished, which may resolve this issue.

Otherwise, cppyy mixes fine with threaded code (e.g. the above example runs fine on MacOS, with Clang the system compiler). Just that any TLS use needs to sit in compiled library code while the JIT has this limitation.

answered Aug 20 '22 at 15:29

Wim Lavrijsen

3,453
1
9
21

Thanks. Would it be possible that I compile a shared library that uses `async` functionality and include that in `cppyy`? – Max Wong Aug 20 '22 at 16:26
Yes: if you put `test()` in a library, you can use it like any other function. Do remember to set `__release_gil__` (https://cppyy.readthedocs.io/en/latest/misc.html?highlight=release%20gil#special-variables) for it to run asynchronous from Python code – Wim Lavrijsen Aug 20 '22 at 17:57

How to use future / async in cppyy

1 Answers1