Compiling C++ code with g++ from Python script on Windows

Question

I'm trying to write a Python script that calls g++.exe via subprocess.Popen() and uses it to compile a .cpp file into an .exe. The problem is that no matter how I try to pass the path to the source file, I get the following error:

g++.exe: error: CreateProcess: No such file or directory

My directory structure is as follows:

D:/Test/test.py
D:/Test/external/mingw64/g++.exe
D:/Test/c/client/client.cpp

And my code is:

import os, subprocess

class builder():
    def __init__(self):
        self.gccPath = os.path.abspath("external/mingw64/g++.exe")
        self.sourceDir = os.path.abspath("c/client")
        self.fileName = "client.cpp"
        self.sourceFile = os.path.join(self.sourceDir, self.fileName)

    def run(self):
        command = [self.gccPath, self.sourceFile , "-o", "client.exe"]
        print command
        process = subprocess.Popen(command, shell=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
        n=1
        while True:
            nextLine = process.stdout.readline()
            if nextLine == '' and process.poll() != None:
                break
            if nextLine != "" and nextLine != None:
                print n, nextLine
            n=n+1

builder = builder()
builder.run()

Just some of the ways I've tried to pass the path:

Command: ["D:\\Test\\external\\mingw64\\g++.exe", "c/client/client.cpp", "-o", "client.exe"]
Command: ["D:\\Test\\external\\mingw64\\g++.exe", "c\\client\\client.cpp", "-o", "client.exe"]
Command: ["D:\\Test\\external\\mingw64\\g++.exe", "D:\\Test\\c\\client\\client.cpp", "-o", "client.exe"]

I also tried passing cwd to Popen:

command = [self.gccPath, "client.cpp", "-o", "client.exe"]
process = subprocess.Popen(command, shell=True, cwd=self.sourceDir, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)

Always the same error. I've used Popen plenty of times before and it's usually a trivial matter, so I'm pretty stumped right now as to what I'm doing wrong.

@PM2Ring - But I'm not joining two absolute paths with that line, I'm joining an absolute path with a string. I'm not too worried about path looks, they're only being printed out for debug purposes. If and when I get this working all prints will be removed anyways. — Natsukane, Jun 12 '16 at 14:28
Sounds like you are trying to make a build-system. Why not just use an existing one like SCons (http://scons.org/) or CMake (https://cmake.org/) ? — Jesper Juhl, Jun 12 '16 at 14:48

score 0 · Answer 1 · answered Jun 12 '16 at 14:04

0

It is not the client.cpp file that is not found, but the g++.exe. You can know that because it is CreateProcess that generates the error. If it was the cpp file, CreateProcess would succeed, and only then would the compiler return with an error.

os.path.abspath("external/mingw64/g++.exe")

This builds an absolute path from the relative path you give. Relative means relative to current directory, not to the directory of the python file.

If your g++ is in a fixed tree, a better way should be to construct the path from the script name, like this:

os.path.join(os.path.dirname(__file__), "external/mingw64/g++.exe")

Holds true for other places where you use abspath for something not relative to current working directory.

answered Jun 12 '16 at 14:04

spectras

13,105
2
31
53

Trying that, I just get "The system cannot find the path specified". I'm assuming this time it's an actual compiler error and the client.cpp is the issue? – Natsukane Jun 12 '16 at 14:18
Could be. Is an exception raised, and if so, what is it? – spectras Jun 12 '16 at 14:22
Also, if you still have `cwd=self.sourceDir`, check that it's a valid path as well (constructing it using `__file__` the same way you just changed `gccPath`). – spectras Jun 12 '16 at 14:23
No exceptions, the only output is `1 The system cannot find the path specified`. When I try to construct self.sourceDir that way, there's an exception from subprocess (_execute_child); WindowsError: [Error 267] The directory name is invalid. – Natsukane Jun 12 '16 at 14:33
If no exceptions, then `Popen` succeeds, and the error you see is from `g++.exe`. So you successfuly fixed the path to g++. Next step, fix the path to `client.cpp`. – spectras Jun 12 '16 at 14:39

score 0 · Answer 2 · edited May 23 '17 at 12:08

I was able to solve my own problem and get a working .exe with the following code:

import os, subprocess, json, glob

class client():
    def __init__(self):
        self.gccDir = os.path.abspath("external/mingw64")
        self.sourceDir = "c/client"
        self.fileName = "client.cpp"
        self.sourceFile = os.path.join(self.sourceDir, self.fileName)
        self.destFile = self.sourceFile.replace(".cpp", ".exe")

    def run(self):
        srcFiles = glob.glob(os.path.join(self.sourceDir+"/*.cpp"))
        srcFiles.remove(self.sourceFile)
        myEnv = os.environ.copy()
        myEnv["PATH"] = myEnv["PATH"]+";"+self.gccDir
        command = ["g++.exe", self.sourceFile, " ".join([x for x in srcFiles]), "-std=c++11", "-Os", "-o", self.destFile]
        process = subprocess.Popen(command, shell=True, env=myEnv, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
        n=1
        while True:
            nextLine = process.stdout.readline()
            if nextLine == '' and process.poll() != None:
                break
            if nextLine != "" and nextLine != None:
                print n, nextLine
            n=n+1

The command ends up being:

['g++.exe', 'c/client\\client.cpp', 'c/client\\utils.cpp', '-std=c++11', '-Os', '-o', 'c/\\client.exe']

Paths look ugly but work. The manual removing of sourceFile from srcFiles is a bit clumsy, but it seems necessary that the main file is the first to be referenced in the command.

This answer was very useful and allowed me to temporarily set the PATH environment variable to whatever directory I had g++.exe in. Thanks to everyone for trying to help.

Compiling C++ code with g++ from Python script on Windows

2 Answers2