Why does "NameError: name not defined" occur when executing Python file externally?

Question

I have multiprocessing code that runs a function to retrieve items from an API:

import os
import pandas as pd
from multiprocessing import Pool
from datetime import timedelta, date
from external_functions import get_api_item #separate .py file with a function

def date_range(start_date, end_date):
    for n in range(int ((end_date - start_date).days)):
        yield start_date + timedelta(n)

start_date = date(2020, 1, 29)
end_date = date(2020, 1, 30)
date_list = []

for single_date in date_range(start_date, end_date):
    date_list.append(single_date.strftime('%Y-%m-%d'))

if __name__ == '__main__':
    global results
    p = Pool(20)
    results = p.map(get_api_item, date_list)
    p.terminate()
    p.join()

result = pd.concat(results)

When I run this code from Jupyter Notebook, Spyder, etc., it works. When I execute it from an Execute Process task in SSIS, I get this error:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "E:\Python\Anaconda3\lib\multiprocessing\spawn.py", line 105, in spawn_ma
in
    exitcode = _main(fd)
  File "E:\Python\Anaconda3\lib\multiprocessing\spawn.py", line 114, in _main
    prepare(preparation_data)
  File "E:\Python\Anaconda3\lib\multiprocessing\spawn.py", line 225, in prepare
    _fixup_main_from_path(data['init_main_from_path'])
  File "E:\Python\Anaconda3\lib\multiprocessing\spawn.py", line 277, in _fixup_m
ain_from_path
    run_name="__mp_main__")
  File "E:\Python\Anaconda3\lib\runpy.py", line 263, in run_path
    pkg_name=pkg_name, script_name=fname)
  File "E:\Python\Anaconda3\lib\runpy.py", line 96, in _run_module_code
    mod_name, mod_spec, pkg_name, script_name)
  File "E:\Python\Anaconda3\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "<path to api>\api.py", line 29, in <module>
    result = pd.concat(results)
NameError: name 'results' is not defined

This tells me that the function itself is not actually running/completing to feed results to the next line. Why is this happening?

If I remove if __name__ == '__main__': and just run it directly, I get this error instead:

  File "E:\Python\Anaconda3\lib\multiprocessing\spawn.py", line 143, in get_prep
aration_data
    _check_not_importing_main()
  File "E:\Python\Anaconda3\lib\multiprocessing\spawn.py", line 136, in _check_n
ot_importing_main
    is not going to be frozen to produce an executable.''')
RuntimeError:
    An attempt has been made to start a new process before the
    current process has finished its bootstrapping phase.

    This probably means that you are not using fork to start your
    child processes and you have forgotten to use the proper idiom
    in the main module:

        if __name__ == '__main__':
            freeze_support()
            ...

    The "freeze_support()" line can be omitted if the program
    is not going to be frozen to produce an executable.

score 0 · Answer 1 · answered Jan 30 '20 at 20:51

0

the line if __name__ == '__main__' specifically is checking where the file was called from, if you're calling it directly the name does equal main, however if you're running it from within a service the name is based on whatever service called it. You get a name error because results is only defined inside of that if statement. (not a scope issue because of your use of global though)

I have been doing a little googling to see what the name of SSIS instances is but did not find anything.

for more on __name__=='__main__':

What does if __name__ == "__main__": do?

answered Jan 30 '20 at 20:51

EyveG

71
3

That makes sense. How would you put that in initial googling terms? Would it be the same name all the time? Would there be a way to exclude that, or is that not recommended? – OverflowingTheGlass Jan 30 '20 at 21:04
I unfortunately don't know how to fix it with the `if` statement but more details on how things are named can be found at: https://www.geeksforgeeks.org/__name__-special-variable-python/ at a guess maybe __name__==SSIS? – EyveG Jan 30 '20 at 21:18
I've just been googling '__name__', 'ssis __name__', and 'if __name__ == '__main__'' (all bold words are wrapped in __ but I don't know how to make the formatting show them) – EyveG Jan 30 '20 at 21:19

Why does "NameError: name not defined" occur when executing Python file externally?

1 Answers1