2

I have an app where i had two executables: A Flask-SocketIO-Server and CefPython browser. I bundled the two executables with PyInstaller. The Flask-Server with --onefile option and the cefpython with --onedir option, because i couldnt make it with --onefile. Now i decided to have only executable for both codes (Flask and CEFpython), so my flask server has code to run the CEF graphical user interface:

if __name__ == '__main__':

    if len(sys.argv) > 1 and sys.argv[1] == 'dev':
        print "Running Flask-SocketIO on dev mode"
    else:
        print "Running Flask-SocketIO on production mode"
        path = os.getcwd()
        gui_path = path + '\\display_react\\display_react.exe'
        print 'Running Graphical User Interface...'
        thread.start_new_thread(display_react.main, ())  # Baterias
        print 'Initializing server'


    socketio.run(app, debug=False)

The code works fine, but when i try to bundle this code with PyInstaller with --onefile option, the generated executable doesnt work cause some of CEF dependencies. Here the errors when running Pyinstaller:

Running Flask-SocketIO on production mode Running Graphical User Interface... Initializing server [wxpython.py] CEF Python 57.1 [wxpython.py] Python 2.7.14 64bit [wxpython.py] wxPython 4.0.1 msw (phoenix) [0727/125110.576:ERROR:main_delegate.cc(684)] Could not load locale pak for en-US [0727/125110.576:ERROR:main_delegate.cc(691)] Could not load cef.pak [0727/125110.578:ERROR:main_delegate.cc(708)] Could not load cef_100_percent.pak [0727/125110.582:ERROR:main_delegate.cc(717)] Could not load cef_200_percent.pak [0727/125110.582:ERROR:main_delegate.cc(726)] Could not load cef_extensions.pak [0727/125110.648:ERROR:content_client.cc(269)] No data resource available for id 20418 [0727/125110.648:ERROR:content_client.cc(269)] No data resource available for id 20419 [0727/125110.650:ERROR:content_client.cc(269)] No data resource available for id 20420 [0727/125110.655:ERROR:content_client.cc(269)] No data resource available for id 20421 [0727/125110.656:ERROR:content_client.cc(269)] No data resource available for id 20422 [0727/125110.656:ERROR:content_client.cc(269)] No data resource available for id 20417 [0727/125110.680:ERROR:extension_system.cc(72)] Failed to parse extension manifest. C:\Users\Ricardo\AppData\Local\Temp_MEI95~1\display_react.py:118: wxPyDeprecationWarning: Call to deprecated item EmptyIcon. Use :class:Icon instead

Here the .spec file i am using:

# -*- mode: python -*-

block_cipher = None

def get_cefpython_path():
    import cefpython3 as cefpython

    path = os.path.dirname(cefpython.__file__)
    return "%s%s" % (path, os.sep)

cefp = get_cefpython_path()


a = Analysis(['server.py'],
             pathex=['C:\\Users\\Ricardo\\addvolt-scanning-tool\\backend'],
             binaries=[],
             datas=[('PCANBasic.dll', '.'), ('o.ico', '.')], #some dlls i need for flask
             hiddenimports=['engineio.async_gevent'], #engineio hidden import for Flask usage
             hookspath=[],
             runtime_hooks=[],
             excludes=[],
             win_no_prefer_redirects=False,
             win_private_assemblies=False,
             cipher=block_cipher)
pyz = PYZ(a.pure, a.zipped_data,
             cipher=block_cipher)
exe = EXE(pyz,
          a.scripts,
          a.binaries,
          a.zipfiles,
          a.datas + [('locales/en-US.pak', '%s/locales/en-US.pak' % cefp, 'DATA')], # my try to fix that missing dependencies
          name='server',
          debug=False,
          strip=False,
          upx=True,
          runtime_tmpdir=None,
          console=True )

EDIT: SOLVED

Thanks to @cztomczak i got this working. The problem was not on PyInstaller, but on the way wxpython.py was looking for the locale, resources and subprocess stuff. Although all files were on the 'temp/dir/_MEIxxx', the wxpython was looking for these files on the directory of the executable. So the way i got to inform the code to look for these files on temp directory was:

dir_temp = tempfile.gettempdir()
files = []
for i in os.listdir(dir_temp):
    if os.path.isdir(os.path.join(dir_temp,i)) and '_MEI' in i:
        files.append(i)
dir_temp = dir_temp + str(files[0])
dir_temp = os.path.join(dir_temp, str(files[0]))
dir_temp_locale = os.path.join(dir_temp, 'locales')
dir_temp_subprocess = os.path.join(dir_temp_subprocess, 'subprocess.exe')

print dir_temp
dir_temp = dir_temp.replace("\\", "\\\\")
print dir_temp
print dir_temp_locale
dir_temp_locale = dir_temp_locale.replace("\\", "\\\\")
print dir_temp_locale
dir_temp_supbprocess = dir_temp_subprocess.replace("\\", "\\\\")
print dir_temp_subprocess

...

settings = {'auto_zooming': '-2.5', 'locales_dir_path': dir_temp_locale, 'resources_dir_path': dir_temp, 'browser_subprocess_path': dir_temp_subprocess}

i had to do this because the name of the created folder on temp (_MEIxxxx) is always changing. And probably i will have problems in the future because if the App crashes, the _MEIxx folder will not be deleted and if i try to re-run the executable, this piece of code will have two _MEI folders and possibily will not work at all until someone clean the temp directory.

So, resuming... To bundle in onefile the app: - Paste the hook-cefpython3.py (available on the package) on Python27/envs/libs/site-package/Pyinstaller/hooks - Run Pyinstaller with --onefile options - Tell the cefpython code where locale, resource and subprocess are (locale_dir_path, resource_dir_path, browser_subprocess_path)

Ricardo Goncalves
  • 315
  • 3
  • 15

2 Answers2

2

I had a similar problem and found using the _MEIPASS environmental variable to be a more elegant solution.

import cefpython
import os
import sys

if hasattr(sys, '_MEIPASS'):
    # settings when packaged
    settings = {'locales_dir_path': os.path.join(sys._MEIPASS, 'locales'),
                'resources_dir_path': sys._MEIPASS,
                'browser_subprocess_path': os.path.join(sys._MEIPASS, 'subprocess.exe'),
                'log_file': os.path.join(sys._MEIPASS, 'debug.log')}
else:
    # settings when unpackaged
    settings = {}

cefPython.Initialize(settings=settings)
resident
  • 41
  • 4
1

I guess the errors you got are because your spec file didn't include all necessary CEF binary files. There is an official pyinstaller example that you can use and modify to use --onefile option: https://github.com/cztomczak/cefpython/blob/master/examples/pyinstaller/README-pyinstaller.md

Czarek Tomczak
  • 20,079
  • 5
  • 49
  • 56
  • well, i tried to run the example of pyinstaller and it gives an error: 'AssertionError: Missing cefpython3 Cython modules' when pyinstaller access the hook-cefpython3.py. Do you know what can be the cause? Thanks a lot for the help. – Ricardo Goncalves Jul 29 '18 at 22:55
  • It seems the contents of the cefpython3 package on your system was modified. There are expected multiple .pyd files there. Assertion is here: https://github.com/cztomczak/cefpython/blob/master/examples/pyinstaller/hook-cefpython3.py#L72 – Czarek Tomczak Jul 30 '18 at 03:06
  • @RicardoGoncalves Check what CEFPYTHON3_DIR is set to. – Czarek Tomczak Jul 30 '18 at 03:07
  • Thank you a lot! It was that. The CFPYTHON3_DIR was : c:\xxx\xxx\envs\inst_exe\scripts\Lib\site-packages\cefpython3, instead of c:\xxx\xxx\envs\inst_exe\Lib\site-packages\cefpython3. It works now. Now i will try to modify the .spec file to generate single executable. Again, thank you for the help and for the amazing package. – Ricardo Goncalves Jul 30 '18 at 09:46
  • I have edited the .spec file to generate one file. And although it is successfully generated, it does not run because of the same problem: Missing locale pak, cef.pak, cef_100.pak, etc.. basically, the locale folder and the .pak files. Here my .spec modified (only the part i changed, deletec Collect() and edited Exe()): exe = EXE(pyz, a.scripts, a.binaries, a.zipfiles, a.datas, name='cefapp', debug=DEBUG, strip=False, upx=False, console=DEBUG, icon="../resources/wxpython.ico"). Im still trying to do this on your pyinstaller example – Ricardo Goncalves Jul 30 '18 at 10:22
  • @RicardoGoncalves Check pyinstaller temp extracted directory when running app. See sys._MEIPASS set by pyinstaller. See if all binary files reside there. If Yes then try setting ApplicationSettings.locales_dir_path and resources_dir_path options manually. See docs for reference. – Czarek Tomczak Jul 30 '18 at 10:48
  • I tried more a couple of things. As this is a single executable, all the data and binary files are stored on a temp directory (_MIEPASS2) when I execute the cefapp.exe. The missing files are on that temp directory (locale and .pak files), so the package bundling is correct. Whats wrong is the directory is being used by the exe to use the binary data. So, i stored the locale folder and .pak files on the same directory of the .exe, and now no error appear. Anyway, it is not working properly because the window open, but dont show anything inside. So it is still not working... :/ – Ricardo Goncalves Jul 30 '18 at 11:01
  • i tried what you've said because i think thats the source of the problem. However, for somehow, the locales_dir_path and resources_dir_path is not working at all and i dont know where the code is trying to find the resources and locales.. Even if i run the code normally (When i say normally, it is without pyinstaller, just typying: 'python server.py') – Ricardo Goncalves Jul 30 '18 at 14:37
  • Edit: It was because of '\' instead of '\\'. I think its working now. I will post the solution. Thank you again! – Ricardo Goncalves Jul 30 '18 at 14:49
  • Now its, solved! I just posted the solution! Thank you! – Ricardo Goncalves Jul 31 '18 at 08:14
  • @RicardoGoncalves You should use the _MEIPASS2 or _MEIPASS env variables to detect the temp dir. These are set by pyinstaller and available in `os.environ`. App sometimes do not exit correctly, thus the temp directory won't be cleaned and your solution will fail. – Czarek Tomczak Jul 31 '18 at 12:20