1

I have a multiprocessing function that runs fine in Spyder like this:

if __name__ == '__main__':
    global results
    p = Pool(20)
    results = p.map(get_api_item, date_list)
    p.terminate()
    p.join()

When I run that .py file in an Execute Process task in SSIS, I get this error:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "E:\Python\Anaconda3\lib\multiprocessing\spawn.py", line 105, in spawn_ma
in
    exitcode = _main(fd)
  File "E:\Python\Anaconda3\lib\multiprocessing\spawn.py", line 114, in _main
    prepare(preparation_data)
  File "E:\Python\Anaconda3\lib\multiprocessing\spawn.py", line 225, in prepare
    _fixup_main_from_path(data['init_main_from_path'])
  File "E:\Python\Anaconda3\lib\multiprocessing\spawn.py", line 277, in _fixup_m
ain_from_path
    run_name="__mp_main__")
  File "E:\Python\Anaconda3\lib\runpy.py", line 263, in run_path
    pkg_name=pkg_name, script_name=fname)
  File "E:\Python\Anaconda3\lib\runpy.py", line 96, in _run_module_code
    mod_name, mod_spec, pkg_name, script_name)
  File "E:\Python\Anaconda3\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "<path to api>\api.py", line 29, in <module>
    result = pd.concat(results)
NameError: name 'results' is not defined

This is because the service (SSIS) is likely not calling it main. How do I remedy this?

OverflowingTheGlass
  • 2,324
  • 1
  • 27
  • 75
  • Could it possibly be due to single quotes vs double quotes around `'__main__'`? I know they're supposed to be mostly equivalent in Python, but maybe there's something weird going on there. Have you tried changing it? – digital.aaron Jan 31 '20 at 20:24

0 Answers0