Sphinx - file not found - sys.path issue

Question

I am trying to build documentation for my flask project, and I am experiencing issues with the path

My project structure is like:

myproject
    config
        all.py
        __init__.py
        logger.py
        logger.conf
    myproject
        models.py
        __init__.py
    en (english language docs folder)
        conf.py

logger.py includes a line

with open('logger.conf') as f: CONFIG = ast.literal_eval(f.read())

which reads the configuration from logger.conf

While "make html"

I receive many errors according to models:

/home/username/projects/fb/myproject/en/models/index.rst:7: WARNING: autodoc: failed to import class u'User' from module u'myproject.models'; the following exception was raised:
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/sphinx/ext/autodoc.py", line 326, in import_object
__import__(self.modname)
  File "/home/username/projects/fb/myproject/myproject/__init__.py", line 14, in <module>
from logger import flask_debug
  File "/home/username/projects/fb/myproject/logger.py", line 5, in <module>
with open('logger.conf') as f: CONFIG = ast.literal_eval(f.read())
IOError: [Errno 2] No such file or directory: 'logger.conf'

which is strange because conf.py includes the path: sys.path.insert(0, '/home/username/projects/fb/myproject/')

and when I print sys.path it shows that the path is there.

When I paste FULL PATH to the file logger.conf in logger.py it goes to another line simmilar to that and throws the same error for a different file.

Why Sphinx does not check the path files relatively to the sys.path?

Because it does not work for "./file" or "file". It started working only for "../file" - when I changed all the paths, but "destroyed" python working, as for python the path is broken.

score 4 · Accepted Answer · edited Jul 22 '16 at 11:52

4

It is the behaviour of open() that is the problem. Commands like open() and chdir() and so work from the directory you're now in, which is probably the directory where the makefile is.

To test it, add an print(os.listdir('.') above your call to open('logger.conf'), that'll show you the problem.

The solution? Use an absolute path. So, a little bit verbose, something like this:

import os

this_directory = os.path.dirname(__file__)
# __file__ is the absolute path to the current python file.
open(os.path.join(this_directory, 'logger.conf'))

Bonus points if you've turned it into a python package (="it has an setup.py"), in that case you can do:

import pkg_resources

open(pkg_resources.resource_filename('myproject.config', 'logger.conf'))

edited Jul 22 '16 at 11:52

Etaoin

149
2
7

answered Sep 04 '13 at 07:26

Reinout van Rees

13,486
2
36
68

Thank you, for your answer. That helped me a lot :). Do you maybe know if there is a possibility to use something like that in logger.conf file, as there are configurations like: 'flask_info':{ 'level': 'INFO', 'class': 'logging.handlers.TimedRotatingFileHandler', 'formatter': 'simple', 'filename': './log/flasklogs.log', 'when': 'midnight', 'interval': 1, 'backupCount': 30, }, For now I have to separate logger.conf files to generate and to run the app. I wonder if I can modify this as well and use only one file. – Archarachne Sep 07 '13 at 10:20
1

Do you read that conf file yourself? If so, nothing keeps you from treating a path in there as a relative path. Alternative is to look for a LOGDIR environment variable or so and os.path.join() with that. – Reinout van Rees Sep 07 '13 at 18:20

score 3 · Answer 2 · edited May 23 '17 at 12:14

I had a similar problem when generating sphinx documentation for some python code that was not written to be run in my computer, but in an embedded system instead. In that case, the existing code attempted to open a file that did not exist in my computer, and that made sphinx fail. In this case, I decided to change the code to verify the file existence first, and that allowed sphinx to pass over this logic without a problem.

if os.path.isfile(filename):
    # open file here
else:
    # handle error in a way that doesn't make sphinx crash
    print "ERROR: No such file: '%s'" % filename

For a moment, I tried mocking open(), but it turns out that sphinx does require open() to do its job.

Sphinx - file not found - sys.path issue

2 Answers2