Trouble importing the correct module in python in GAE

Question

How do I explicitly tell Google App Engine (python) to import json from the python standard libraries?

Due to a poorly named file (that I can NOT change or rename at this time) I am having trouble importing json.

There is a json.py in the same directory as the file that I am working on. When I try:

import json

it imports the file in the same directory.

Is there a way I can do something along the lines of:

from ../ import json

to import the Native JSON library?

EDIT:

I have now even tried renaming the offending file and replacing all uses of this file. But I'm still not able to import the json standard lib through GAE.

Attached is the error log:

File "/Users/admin/Blah/dataaccess.py", line 13, in <module> 
   from classes import ZenDesk    
File "/Users/admin/Blah/classes/ZenDesk.py", line 10, in <module> 
   import json
File "/Users/admin/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/devappserver2/python/sandbox.py", line 892, in load_module
   raise ImportError('No module named %s' % fullname)
ImportError: No module named classes.json

Please Help google app engine should be looking for the standard library and not in the subdirectory classes

I just said that I can NOT rename the file due to other dependencies! — Ervin, May 09 '13 at 17:31
You'd have to juggle the `sys.path` variable so the local directory is not consulted first. Just renaming the module is easier, including fixing the dependencies. — Martijn Pieters, May 09 '13 at 17:32
Not really. My question is how do I explicitly tell python what to import. — Ervin, May 09 '13 at 17:48
You have no choice but to rename the `json.py` file since you have no access to the file system on GAE. — Burhan Khalid, May 09 '13 at 18:30
@BurhanKhalid I understand that I do not have access to the GAE file system. But how can I tell GAE to look for the python standard library first when I import? — Ervin, May 09 '13 at 18:45
Have yo managed to sort this out? Did you try importing from another directory? — , May 10 '13 at 07:42

Henry Keiter · Answer 1 · 2013-05-09T18:38:58.083

1

If you can't rename the module for some reason, you have to mess around with sys.path to put your standard lib in front of the current module. Make sure to fix it again after you do your import, though...

import sys
sys.path = [r'C:\Python27\Lib'] + sys.path
import json
sys.path = sys.path[1:]

Alternately, I'm pretty sure the imp module has functionality for this.

Let me be clear, though: it would be much better (as others have said) to just rename the module in the project directory (e.g. to my_json.py). You're not going to have to change that many dependencies.

Some Edits: If you cannot find an actual path, absolute or otherwise, to the standard lib, you have two further options besides renaming the thing. You can move the file you're working on to a different level in your package (so that it can't see the local json.py). For instance, like this:

/package
    __init__.py
    your_file.py # "import json" should now hit the std lib.
                 # To import the local copy, use "from app import json"
    app/
        __init__.py
        json.py
        other_thing.py # contains "import json" to get the local copy

If you don't want to move your file, then you can fake it by adding an extra file that does nothing but import the real json module:

/package
    __init__.py
    json_importer.py # contains nothing but "import json"
    app/
        __init__.py
        json.py
        your_file.py # "from package.json_importer import json"
        other_thing.py # contains "import json" to get the local copy

edited May 09 '13 at 18:38

answered May 09 '13 at 18:06

Henry Keiter

16,863
7
51
80

this is deploying to google app engine. So I am not able to specify a path to MY standard library. I need to specify that I want to import from python standard libraries. – Ervin May 09 '13 at 18:13
I agree it would be BETTER to rename the offending file. But that is NOT my question. My question is how do I explicitly tell python where to import from? in particular where does GAE allow me to import from? – Ervin May 09 '13 at 18:19
@tiger13cubed If you can get a relative path to the standard lib, you can put that at the front of `sys.path`. If you can't get any kind of path to the standard libraries at all, your only options are to rename the offending nonstandard module or to hack it by making a new module (e.g. `json_importer`) at a different level of the package so it can't see your local `json` module, importing the real `json` **in there**, and then in your real file you can do `from package.subpackage.json_importer import json`. :-/ – Henry Keiter May 09 '13 at 18:22
The files importing the local json are just using import! There is no hack import will look in the current directory first! – Ervin May 09 '13 at 18:22
a path to my local json library is useless when this is running on Google App Engine – Ervin May 09 '13 at 18:23
2

@tiger13cubed: where does *GAE* think the stdlib lives? What do you get if you `import string` and then `print string`? Alternatively, what is the value of `sys.path`? – DSM May 09 '13 at 18:26
@tiger13cubed There is **no way** to say "magically import the module I want" without telling Python what that module is. I guess you could try hunting down the directory of `__file__` within `sys.path`, removing it, importing `json`, and then adding the current dir back in at the same index... – Henry Keiter May 09 '13 at 18:29
@DSM that prints as but that was running on local. Will that change once it's deployed? – Ervin May 09 '13 at 18:30
@tiger13cubed: almost certainly (I doubt GAE's Mac-based). – DSM May 09 '13 at 18:31
I'm sure there is a from google._____.___ import json. Im just surprised there is no way to tell import to start looking in the parent directory! – Ervin May 09 '13 at 18:36
@tiger13cubed If you really only have to go to the parent directory, that means that you can use a relative path, so you can use `sys.path` hacks by adding `'..'` to the front of the path. However, it sounds more and more like you don't have access to the structure of the std lib, which means you need to rename or move files. – Henry Keiter May 09 '13 at 18:40
@HenryKeiter I've gone through and renamed pretty much every instance of json to json2. However when I import it says there is no module named classes.json. Classes is the name of the directory – Ervin May 09 '13 at 19:55
@tiger13cubed Well that's because you just renamed it to `json2`, no? If you were importing a file and do nothing but change the name and it no longer works, then you either didn't change all the names, or you were never importing that file in the first place. – Henry Keiter May 09 '13 at 20:09
@HenryKeiter I did change all the names! The only one that is still raising the exception is the file that I want to import the correct json. – Ervin May 09 '13 at 20:11
@tiger13cubed If your actual statement is now `import classes.json`, it's broken because that doesn't exist. If your statement is (correctly) `import json`, then I don't know how you could be getting an error message referencing `classes.json`. – Henry Keiter May 09 '13 at 20:16
@HenryKeiter My statement is import json. Is there anything in app engine that would be changing it? – Ervin May 09 '13 at 20:43
@tiger13cubed I have no idea why there would be, but I know almost nothing about how GAE handles any of this. If all else fails, you could copy the `json` library from your local Python install into your app directory and reference it from there, I guess, but that shouldn't be necessary. Does GAE not supply the json module from the core lib? Can you access other core modules? This is a little divested from the original question at this point; it might be best to just go hunt down some documentation on how GAE actually resolves Python imports. – Henry Keiter May 09 '13 at 20:57

score 0 · Answer 2 · answered May 09 '13 at 18:04

0

Try this perhaps?

import imp

json = imp.load_source('module.name', '/path/to/built-in/json/file.py')

answered May 09 '13 at 18:04

Andenthal

849
5
13

what exactly would '/path/to/built-in/json/file.py' be from google app engine? I can't use the local directory on my computer – Ervin May 09 '13 at 18:26

score 0 · Answer 3 · answered May 09 '13 at 18:53

0

Try this, although I'm not sure if it will work on GAE since its a PaaS platform:

import os
import sys

from distutils.sysconfig import get_python_lib

sys.path.insert(0,os.path.dirname(get_python_lib()))
import json
sys.path = sys.path[1:]

answered May 09 '13 at 18:53

Burhan Khalid

169,990
18
245
284

Then you have exhausted all options. Open your directory in an editor that supports multiple file find replace, and change your `json.py` to something else. I am not sure how involved or complicated the code base is because shadowing a built-in hasn't raised any issues yet, so I guess its not a bit find-replace job. – Burhan Khalid May 09 '13 at 19:00
yes yes but that is such a hack. The real question is how do you chose where to import from? – Ervin May 09 '13 at 19:02
Its not a hack, its the solution to the problem. What you are asking here are hacks around the actual solution; and to answer your other question, on how to choose where to import stuff, you use `PYTHONPATH` and `sys.path`. As you have thrown in GAE into the mix, those do not apply since Python running on GAE is running in a non-standard environment. – Burhan Khalid May 09 '13 at 19:07

Trouble importing the correct module in python in GAE

3 Answers3