3

Greetings learned fellows. Running 32bit Python2.7 on Windows 7.

I'm have a question regarding including GDAL executables in a pyinstaller build. I am making a system call to run two GDAL functions from the FWTools release. These functions are in the PATH variable on windows C:\Program Files (x86)\FWTools2.4.7\bin and so it runs fine from the Python27 environment. However, this path is not carried over to the pyinstaller build.

The code in question is calling a GDAL function to re-translate an image onto different geospatial co-ordinates.

os.system("gdal_translate -of GTiff -a_ullr  694440.7939 6403967.2406  696438.7261 6404791.6774 -a_srs EPSG:28355 site.tif siteGR.tif")

os.system("gdalinfo siteGR.tif")

The runtime works well until it hits the above lines and then returns the following error:

'gdal_translate' is not recognized as an internal or external command,
operable program or batch file.

'gdalinfo' is not recognized as an internal or external command,
operable program or batch file.

I have tried to include both gdal_translate.exe and gdalinfo.exe in the build folder as binaries, much like you would do for a .dll, but since it is failing after the script has got going, I don't think it is referring to them.

I have included the spec file below. I could use some advice on how to get the pyinstaller build to recognize executables run from the system in a python script.

Spec:

# -*- mode: python -*-

a = Analysis(['gis_helper.py'],
             pathex=['C:\\Users\\Hp\\PycharmProjects\\GISdev'],
             hiddenimports=['scipy.linalg.cython_blas', 'scipy.linalg.cython_lapack', 'scipy.special._ufuncs_cxx', 'ctypes.util', 'pandas.util', 'distutils.util', 'shapely', '_socket', '_proj', 'multiprocessing', '_multiprocessing', 'multiprocessing.process', 'multiprocessing.util'],
             hookspath=['C:\\Python27\\Lib\\site-packages\\PyInstaller\\hooks'], 
             runtime_hooks=None)


a.binaries1=['geos_c.dll', 'geos_c.dll', 'BINARY'],
a.binaries2=['python27.dll', 'python27.dll', 'BINARY'],
a.binaries3=['_socket.pyd', '_socket.pyd', 'BINARY'],
a.binaries4=['win32api.pyd', 'win32api.pyd', 'BINARY'],
a.binaries5=['pywintypes27.dll', 'pywintypes27.dll', 'BINARY'],
a.binaries6=['pythoncom27.dll', 'pythoncom27.dll', 'BINARY'],
a.binaries7=['_imaging.pyd', '_imaging.pyd', 'BINARY'],
a.binaries8=['_fblas.pyd', '_fblas.pyd', 'BINARY'],
a.binaries9=['gdal_translate.exe', 'gdal_translate.exe', 'BINARY'],
a.binaries10=['gdalinfo.exe', 'gdalinfo.exe', 'BINARY'],

import mpl_toolkits.basemap
import os

src_basedata = os.path.join(mpl_toolkits.basemap.__path__[0], "data")
tgt_basedata = os.path.join('mpl_toolkits', 'basemap', 'data')

pyz = PYZ(a.pure)
exe = EXE(pyz,
          a.scripts,
          [('v',None,'OPTION')],
          a.binaries,  # This needs to be included
          a.binaries1,
          a.binaries2,
          a.binaries3,
          a.binaries4,
          a.binaries5,
          a.binaries6,
          a.binaries7,
          a.binaries8,
          a.binaries9,
          a.binaries10,
          a.zipfiles,
          a.datas + Tree(src_basedata, prefix=tgt_basedata),
          name='gis_helper.exe',
          debug=True,
          strip=None,
          upx=True,
          console=True )
Praxis
  • 934
  • 2
  • 17
  • 31
  • Couple of checks before I answer. I'm assuming you don't get an error like `IOError: [Errno 2] No such file or directory: 'gdal_translate.exe'` when you run pyinstaller? I ask because you're adding the binaries without a path for pyinstaller to find them - that would usually fail... Assuming that they add OK - do you see `gdal_translate.exe` and `gdalinfo.exe` in your `dist` directory after running `pyinstaller`? – J Richard Snape Oct 08 '15 at 14:26
  • @JRichardSnape Sorry for the late reply, things keep getting in the way. The `gdal` files are included in the folder where I execute pyinstaller from, so that is probably why it isn't crashing during build. No IOError: message though. The script runs until it hits the `os.system` calls. There aren't any files in the build folder after it has compiled, only the executable. Should I be running pyinstaller with a flag to activate this? I think the issue may depend on system path variables? Although if the `gdal` files were included in the compile, you would think it would just find them. – Praxis Oct 11 '15 at 08:10
  • OK - I can replicate now. BTW - to help your own debugging, you could try with a non-onefile option first (with `EXE` and `COL` sections) which, if you put the gdal exes in the col section will convince you that they are being collected and included. However - the reason they don't run is because the `exe` can't "find" them within itself. There is a workaround - can't remember it right now but will look up and answer / point you to it tomorrow – J Richard Snape Oct 11 '15 at 20:44
  • OK - here are 3 questions basically addressing your problem. If you can't get it working with one of them, i'll write up an answer... http://stackoverflow.com/questions/13946650/pyinstaller-2-0-bundle-file-as-onefile http://stackoverflow.com/questions/7674790/bundling-data-files-with-pyinstaller-onefile http://stackoverflow.com/questions/19669640/bundling-data-files-with-pyinstaller-2-1-and-meipass-error-onefile – J Richard Snape Oct 11 '15 at 20:50
  • @JRichardSnape. Many thanks for assisting with this. Adding in `chdir(sys._MEIPASS)` to my script directed the runtime path to the correct destination where the exe was being unpacked and run from. the GDAL plugins were then able to be found. After `a.binaries +=['gdal_translate.exe', 'gdal_translate.exe', 'DATA'],` was added to the spec file of course. – Praxis Oct 15 '15 at 09:20
  • Brilliant. Maybe we should put an answer down – J Richard Snape Oct 15 '15 at 13:24

1 Answers1

0

From GDAL 2.1 you can use the 'librified' version of gdal_translate in the python bindings, which may be more robust and prevent you from having to muck about with the path:

import gdal
gdal.Translate(<options>)

Some examples of it being used can be found in the test suite of the gdal repository: https://svn.osgeo.org/gdal/trunk/autotest/utilities/test_gdal_translate_lib.py

You can pip install gdal and I believe it is the 2.1 release (on linux). For windows, the easiest way to install is with conda.

This should prevent you from having to mess around including binaries/paths in pyinstaller too much as it can be treated like any python module

jramm
  • 6,415
  • 4
  • 34
  • 73