Is it possible to create a function dynamically, during runtime in C++?

Question

C++ is a static, compiled language, templates are resolved during compile time and so on...

But is it possible to create a function during runtime, that is not described in the source code and has not been converted to machine language during compilation, so that a user can throw at it data that has not been anticipated in the source?

I am aware this cannot happen in a straightforward way, but surely it must be possible, there are plenty of programing languages that are not compiled and create that sort of stuff dynamically that are implemented in either C or C++.

Maybe if factories for all primitive types are created, along with suitable data structures to organize them into more complex objects such as user types and functions, this is achievable?

Any info on the subject as well as pointers to online materials are welcome. Thanks!

EDIT: I am aware it is possible, it is more like I am interested in implementation details :)

Answer 1

Yes, of course, without any tools mentioned in the other answers, but simply using the C++ compiler.

just follow these steps from within your C++ program (on linux, but must be similar on other OS)

1. write a C++ program into a file (e.g. in /tmp/prog.cc), using an ofstream
2. compile the program via system("c++ /tmp/prog.cc -o /tmp/prog.so -shared -fPIC");
3. load the program dynamically, e.g. using dlopen()

Answer 2

You can also just give the bytecode directly to a function and just pass it casted as the function type as demonstrated below.

e.g.

byte[3] func = { 0x90, 0x0f, 0x1 }
*reinterpret_cast<void**>(&func)()

Answer 3

Yes, JIT compilers do it all the time. They allocate a piece of memory that has been given special execution rights by the OS, then fill it with code and cast the pointer to a function pointer and execute it. Pretty simple.

EDIT: Here's an example on how to do it in Linux: http://burnttoys.blogspot.de/2011/04/how-to-allocate-executable-memory-on.html

Answer 4

Below an example for C++ runtime compilation based on the method mentioned before (write code to output file, compile via system(), load via dlopen() and dlsym()). See also the example in a related question. The difference here is that it dynamically compiles a class rather than a function. This is achieved by adding a C-style maker() function to the code to be compiled dynamically. References:

• https://www.linuxjournal.com/article/3687
• http://www.tldp.org/HOWTO/C++-dlopen/thesolution.html

The example only works under Linux (Windows has LoadLibrary and GetProcAddress functions instead), and requires the identical compiler to be available on the target machine.

baseclass.h

#ifndef BASECLASS_H
#define BASECLASS_H
class A
{
protected:
    double m_input;     // or use a pointer to a larger input object
public:
    virtual double f(double x) const = 0;
    void init(double input) { m_input=input; }
    virtual ~A() {};
};
#endif /* BASECLASS_H */

main.cpp

#include "baseclass.h"
#include <cstdlib>      // EXIT_FAILURE, etc
#include <string>
#include <iostream>
#include <fstream>
#include <dlfcn.h>      // dynamic library loading, dlopen() etc
#include <memory>       // std::shared_ptr

// compile code, instantiate class and return pointer to base class
// https://www.linuxjournal.com/article/3687
// http://www.tldp.org/HOWTO/C++-dlopen/thesolution.html
// https://stackoverflow.com/questions/11016078/
// https://stackoverflow.com/questions/10564670/
std::shared_ptr<A> compile(const std::string& code)
{
    // temporary cpp/library output files
    std::string outpath="/tmp";
    std::string headerfile="baseclass.h";
    std::string cppfile=outpath+"/runtimecode.cpp";
    std::string libfile=outpath+"/runtimecode.so";
    std::string logfile=outpath+"/runtimecode.log";
    std::ofstream out(cppfile.c_str(), std::ofstream::out);

    // copy required header file to outpath
    std::string cp_cmd="cp " + headerfile + " " + outpath;
    system(cp_cmd.c_str());

    // add necessary header to the code
    std::string newcode =   "#include \"" + headerfile + "\"\n\n"
                            + code + "\n\n"
                            "extern \"C\" {\n"
                            "A* maker()\n"
                            "{\n"
                            "    return (A*) new B(); \n"
                            "}\n"
                            "} // extern C\n";

    // output code to file
    if(out.bad()) {
        std::cout << "cannot open " << cppfile << std::endl;
        exit(EXIT_FAILURE);
    }
    out << newcode;
    out.flush();
    out.close();

    // compile the code
    std::string cmd = "g++ -Wall -Wextra " + cppfile + " -o " + libfile
                      + " -O2 -shared -fPIC &> " + logfile;
    int ret = system(cmd.c_str());
    if(WEXITSTATUS(ret) != EXIT_SUCCESS) {
        std::cout << "compilation failed, see " << logfile << std::endl;
        exit(EXIT_FAILURE);
    }

    // load dynamic library
    void* dynlib = dlopen (libfile.c_str(), RTLD_LAZY);
    if(!dynlib) {
        std::cerr << "error loading library:\n" << dlerror() << std::endl;
        exit(EXIT_FAILURE);
    }

    // loading symbol from library and assign to pointer
    // (to be cast to function pointer later)
    void* create = dlsym(dynlib, "maker");
    const char* dlsym_error=dlerror();
    if(dlsym_error != NULL)  {
        std::cerr << "error loading symbol:\n" << dlsym_error << std::endl;
        exit(EXIT_FAILURE);
    }

    // execute "create" function
    // (casting to function pointer first)
    // https://stackoverflow.com/questions/8245880/
    A* a = reinterpret_cast<A*(*)()> (create)();

    // cannot close dynamic lib here, because all functions of the class
    // object will still refer to the library code
    // dlclose(dynlib);

    return std::shared_ptr<A>(a);
}


int main(int argc, char** argv)
{
    double input=2.0;
    double x=5.1;
    // code to be compiled at run-time
    // class needs to be called B and derived from A
    std::string code =  "class B : public A {\n"
                        "    double f(double x) const \n"
                        "    {\n"
                        "        return m_input*x;\n"
                        "    }\n"
                        "};";

    std::cout << "compiling.." << std::endl;
    std::shared_ptr<A> a = compile(code);
    a->init(input);
    std::cout << "f(" << x << ") = " << a->f(x) << std::endl;

    return EXIT_SUCCESS;
}

output

$ g++ -Wall -std=c++11 -O2 -c main.cpp -o main.o   # c++11 required for std::shared_ptr
$ g++ -ldl main.o -o main
$ ./main
compiling..
f(5.1) = 10.2

Answer 5

Have a look at libtcc; it is simple, fast, reliable and suits your need. I use it whenever I need to compile C functions "on the fly".

In the archive, you will find the file examples/libtcc_test.c, which can give you a good head start. This little tutorial might also help you: http://blog.mister-muffin.de/2011/10/22/discovering-tcc/

#include <stdlib.h>
#include <stdio.h>
#include "libtcc.h"

int add(int a, int b) { return a + b; }

char my_program[] =
"int fib(int n) {\n"
"    if (n <= 2) return 1;\n"
"    else return fib(n-1) + fib(n-2);\n"
"}\n"
"int foobar(int n) {\n"
"    printf(\"fib(%d) = %d\\n\", n, fib(n));\n"
"    printf(\"add(%d, %d) = %d\\n\", n, 2 * n, add(n, 2 * n));\n"
"    return 1337;\n"
"}\n";

int main(int argc, char **argv)
{
    TCCState *s;
    int (*foobar_func)(int);
    void *mem;

    s = tcc_new();
    tcc_set_output_type(s, TCC_OUTPUT_MEMORY);
    tcc_compile_string(s, my_program);
    tcc_add_symbol(s, "add", add);

    mem = malloc(tcc_relocate(s, NULL));
    tcc_relocate(s, mem);

    foobar_func = tcc_get_symbol(s, "foobar");

    tcc_delete(s);

    printf("foobar returned: %d\n", foobar_func(32));

    free(mem);
    return 0;
}

Ask questions in the comments if you meet any problems using the library!

Answer 6

In addition to simply using an embedded scripting language (Lua is great for embedding) or writing your own compiler for C++ to use at runtime, if you really want to use C++ you can just use an existing compiler.

For example Clang is a C++ compiler built as libraries that could be easily embedded in another program. It was designed to be used from programs like IDEs that need to analyze and manipulate C++ source in various ways, but using the LLVM compiler infrasructure as a backend it also has the ability to generate code at runtime and hand you a function pointer that you can call to run the generated code.

• Clang
• LLVM

Answer 7

Essentially you will need to write a C++ compiler within your program (not a trivial task), and do the same thing JIT compilers do to run the code. You were actually 90% of the way there with this paragraph:

I am aware this cannot happen in a straightforward way, but surely it must be possible, there are plenty of programing languages that are not compiled and create that sort of stuff dynamically that are implemented in either C or C++.

Exactly--those programs carry the interpreter with them. You run a python program by saying python MyProgram.py--python is the compiled C code that has the ability to interpret and run your program on the fly. You would need do something along those lines, but by using a C++ compiler.

If you need dynamic functions that badly, use a different language :)

Answer 8

A typical approach for this is to combine a C++ (or whatever it's written on) project with scripting language.
Lua is one of the top favorites, since it's well documented, small, and has bindings for a lot of languages.

But if you are not looking into that direction, perhaps you could think of making a use of dynamic libraries?

Answer 9

Yes - you can write a compiler for C++, in C++, with some extra features - write your own functions, compile and run automatically (or not)...

Answer 10

Have a look into ExpressionTrees in .NET - I think this is basically what you want to achieve. Create a tree of subexpressions and then evaluate them. In an object-oriented fashion, each node in the might know how to evaluate itself, by recursion into its subnodes. Your visual language would then create this tree and you can write a simple interpreter to execute it.

Also, check out Ptolemy II, as an example in Java on how such a visual programming language can be written.

Answer 11

You could take a look at Runtime Compiled C++ (or see RCC++ blog and videos), or perhaps try one of its alternatives.

Answer 12

Expanding on Jay's answer using opcodes, the below works on Linux.

1. Learn opcodes from your compiler:
   • write own myfunc.cpp, e.g.

     double f(double x) { return x*x; }
     

   • compile with

     $ g++ -O2 -c myfunc.cpp
     

   • disassemble function f

     $ gdb -batch -ex "file ./myfunc.o" -ex "set disassembly-flavor intel" -ex "disassemble/rs f"
     Dump of assembler code for function _Z1fd:
        0x0000000000000000 <+0>:     f2 0f 59 c0     mulsd  xmm0,xmm0
        0x0000000000000004 <+4>:     c3      ret    
     End of assembler dump.
     

     This means the function x*x in assembly is mulsd xmm0,xmm0, ret and in machine code f2 0f 59 c0 c3.
2. Write your own function in machine code:
   • opcode.cpp

     #include <cstdlib>          // EXIT_FAILURE etc
     #include <cstdio>           // printf(), fopen() etc
     #include <cstring>          // memcpy()
     #include <sys/mman.h>       // mmap()
     
     // allocate memory and fill it with machine code instructions
     // returns pointer to memory location and length in bytes
     void* gencode(size_t& length)
     {
         // machine code
         unsigned char opcode[] = {
             0xf2, 0x0f, 0x59, 0xc0,         // mulsd  xmm0,xmm0
             0xc3                            // ret
         };
         // allocate memory which allows code execution
         // https://en.wikipedia.org/wiki/NX_bit
         void* buf = mmap(NULL,sizeof(opcode),PROT_READ|PROT_WRITE|PROT_EXEC,
                          MAP_PRIVATE|MAP_ANON,-1,0);
         // copy machine code to executable memory location
         memcpy(buf, opcode, sizeof(opcode));
         // return: pointer to memory location with executable code
         length = sizeof(opcode);
         return buf;
     }
     
     // print the disassemby of buf
     void print_asm(const void* buf, size_t length)
     {
         FILE* fp = fopen("/tmp/opcode.bin", "w");
         if(fp!=NULL) {
             fwrite(buf, length, 1, fp);
             fclose(fp);
         }
         system("objdump -D -M intel -b binary -mi386 /tmp/opcode.bin");
     }
     
     int main(int, char**)
     {
         // generate machine code and point myfunc() to it
         size_t length;
         void* code=gencode(length);
         double (*myfunc)(double);   // function pointer
         myfunc = reinterpret_cast<double(*)(double)>(code);
     
         double x=1.5;
         printf("f(%f)=%f\n", x,myfunc(x));
         print_asm(code,length);     // for debugging
         return EXIT_SUCCESS;
     }
     
     

   • compile and run

     $ g++ -O2 opcode.cpp -o opcode
     $ ./opcode
     f(1.500000)=2.250000
     
     /tmp/opcode.bin:     file format binary
     
     
     Disassembly of section .data:
     
     00000000 <.data>:
        0:   f2 0f 59 c0             mulsd  xmm0,xmm0
        4:   c3                      ret    
     

Answer 13

The simplest solution available, if you're not looking for performance is to embed a scripting language interpreter, e.g. for Lua or Python.

Answer 14

It worked for me like this. You have to use the -fpermissive flag. I am using CodeBlocks 17.12.

#include <cstddef>

using namespace std;
int main()
{
    char func[] = {'\x90', '\x0f', '\x1'};
    void (*func2)() = reinterpret_cast<void*>(&func);
    func2();
    return 0;
}