in script to update django db, I get ImportError depending on the path of script

Question

I want a python script that will update a django db. I have one that works fine when it is a top level of the django project directory, but fails with an ImportError when it is in a subdirectory.

Although my project is a more complicated than this, here is a pared down example of my project directory:

    myproj/
    ├── settings.py
    ├── manage.py
    ├── bin/
    ├── myapp/
    │   ├── __init__.py
    │   ├── models.py
    │   ├── templates/
    │   ├── urls.py
    │   ├── views.py
    └── myproj.db

The package myproj is in the python path, and inside settings.py, the INSTALLED_APPS tuple lists 'myapp'.

Now, if I have a simple python script like this

import os
os.environ['DJANGO_SETTINGS_MODULE'] = 'myproj.settings'

from django.contrib.auth.models import User

print User.objects.filter(username='foo').count()

and I name this test.py and put it at the top of myproj/ and then from inside myproj/ run python test.py everything works just fine (e.g. a 1 is printed if I have a user called 'foo').

If I simply move test.py into myproj/bin/ and from inside myproj/bin do python test.py I get a cascade of errors like this

    Traceback (most recent call last):
      File "test.py", line 6, in <module>
        print User.objects.filter(username='foo').count() 
      File "/usr/local/lib/python2.7/dist-packages/django/db/models/manager.py", line 155, in filter
        return self.get_query_set().filter(*args, **kwargs)
      File "/usr/local/lib/python2.7/dist-packages/django/db/models/query.py", line 669, in filter
        return self._filter_or_exclude(False, *args, **kwargs)
      File "/usr/local/lib/python2.7/dist-packages/django/db/models/query.py", line 687, in _filter_or_exclude
        clone.query.add_q(Q(*args, **kwargs))
      File "/usr/local/lib/python2.7/dist-packages/django/db/models/sql/query.py", line 1271, in add_q
        can_reuse=used_aliases, force_having=force_having)
      File "/usr/local/lib/python2.7/dist-packages/django/db/models/sql/query.py", line 1139, in add_filter
        process_extras=process_extras)
      File "/usr/local/lib/python2.7/dist-packages/django/db/models/sql/query.py", line 1325, in setup_joins
        field, model, direct, m2m = opts.get_field_by_name(name)
      File "/usr/local/lib/python2.7/dist-packages/django/db/models/options.py", line 351, in get_field_by_name
        cache = self.init_name_map()
      File "/usr/local/lib/python2.7/dist-packages/django/db/models/options.py", line 380, in init_name_map
        for f, model in self.get_all_related_m2m_objects_with_model():
      File "/usr/local/lib/python2.7/dist-packages/django/db/models/options.py", line 469, in get_all_related_m2m_objects_with_model
        cache = self._fill_related_many_to_many_cache()
      File "/usr/local/lib/python2.7/dist-packages/django/db/models/options.py", line 483, in _fill_related_many_to_many_cache
        for klass in get_models(only_installed=False):
      File "/usr/local/lib/python2.7/dist-packages/django/db/models/loading.py", line 197, in get_models
        self._populate()
      File "/usr/local/lib/python2.7/dist-packages/django/db/models/loading.py", line 72, in _populate
        self.load_app(app_name, True)
      File "/usr/local/lib/python2.7/dist-packages/django/db/models/loading.py", line 94, in load_app
        app_module = import_module(app_name)
      File "/usr/local/lib/python2.7/dist-packages/django/utils/importlib.py", line 35, in import_module
        __import__(name)
    ImportError: No module named myapp

I'm not sure what to make of these errors. They don't really make it clear what has gone wrong. Presumably django is using its own importing functions and presumably they are being affected by the relative position of the python script and settings.py file.

Any insights?

related: http://stackoverflow.com/questions/8047204/django-script-to-access-model-objects-without-using-manage-py-shell — Ciro Santilli OurBigBook.com, May 13 '16 at 14:10

score 0 · Accepted Answer · answered Oct 04 '13 at 02:47

0

The problem is this line: os.environ['DJANGO_SETTINGS_MODULE'] = 'myproj.settings'

When your script is outside myproj/, then your PYTHON_PATH starts outside myproj/, and hence myproj is a valid package.
When your script is inside myproj/, then your PYTHON_PATH starts inside myproj/, and hence myproj is no longer a valid package.

The fix would be to explicitly setup your PYTHON_PATH from inside the script.

HOWEVER

What you should do instead is turn your script into a management command so that you can run python manage.py myscript and be sure that Django has done all the necessary initialisation and checking before it runs your code.

answered Oct 04 '13 at 02:47

Thomas

11,757
4
41
57

Brilliant! I appended the sys.path inside the script and that solved it. As for management command, I did not think of that. Ultimately, I want to automatically run scripts like those I mentioned from git hooks or cron jobs, as opposed to from the command line. I'm not sure if running that using python manage.py is the right thing to do. I should look into that further. – mjandrews Oct 04 '13 at 08:25
by creating your script as a management command, you gain some advantages such as, automatically taking advantage of the currently activated settings (see `--settings=` cmd line argument to manage.py) and the ability to call that script from inside django in a standard way. It's generally considered best practice to make scripts into management commands. – Thomas Oct 06 '13 at 11:17

in script to update django db, I get ImportError depending on the path of script

1 Answers1