what does the __file__ variable mean/do?

Question

import os

A = os.path.join(os.path.dirname(__file__), '..')

B = os.path.dirname(os.path.realpath(__file__))

C = os.path.abspath(os.path.dirname(__file__))

I usually just hard-wire these with the actual path. But there is a reason for these statements that determine path at runtime, and I would really like to understand the os.path module so that I can start using it.

Answer 1

When a module is loaded from a file in Python, __file__ is set to its path. You can then use that with other functions to find the directory that the file is located in.

Taking your examples one at a time:

A = os.path.join(os.path.dirname(__file__), '..')
# A is the parent directory of the directory where program resides.

B = os.path.dirname(os.path.realpath(__file__))
# B is the canonicalised (?) directory where the program resides.

C = os.path.abspath(os.path.dirname(__file__))
# C is the absolute path of the directory where the program resides.

You can see the various values returned from these here:

import os
print(__file__)
print(os.path.join(os.path.dirname(__file__), '..'))
print(os.path.dirname(os.path.realpath(__file__)))
print(os.path.abspath(os.path.dirname(__file__)))

and make sure you run it from different locations (such as ./text.py, ~/python/text.py and so forth) to see what difference that makes.

Answer 2

I just want to address some confusion first. __file__ is not a wildcard it is an attribute. Double underscore attributes and methods are considered to be "special" by convention and serve a special purpose.

http://docs.python.org/reference/datamodel.html shows many of the special methods and attributes, if not all of them.

In this case __file__ is an attribute of a module (a module object). In Python a .py file is a module. So import amodule will have an attribute of __file__ which means different things under difference circumstances.

Taken from the docs:

__file__ is the pathname of the file from which the module was loaded, if it was loaded from a file. The __file__ attribute is not present for C modules that are statically linked into the interpreter; for extension modules loaded dynamically from a shared library, it is the pathname of the shared library file.

In your case the module is accessing it's own __file__ attribute in the global namespace.

To see this in action try:

# file: test.py

print globals()
print __file__

And run:

python test.py

{'__builtins__': <module '__builtin__' (built-in)>, '__name__': '__main__', '__file__':
 'test_print__file__.py', '__doc__': None, '__package__': None}
test_print__file__.py

Answer 3

Per the documentation:

__file__ is the pathname of the file from which the module was loaded, if it was loaded from a file. The __file__ attribute is not present for C modules that are statically linked into the interpreter; for extension modules loaded dynamically from a shared library, it is the pathname of the shared library file.

and also:

__file__ is to be the “path” to the file unless the module is built-in (and thus listed in sys.builtin_module_names) in which case the attribute is not set.

Answer 4

Just going to add a quick note here (mostly answering the question's title rather than its description) about a change which can confuse some people. As of Python 3.4 there has been a slight change in how the __file__ behaves:

• It's set to the relative path of the module in which it's used, if that module is executed directly.
• It's set to the absolute path of the file otherwise.

Module __file__ attributes (and related values) should now always contain absolute paths by default, with the sole exception of __main__.__file__ when a script has been executed directly using a relative path. (Contributed by Brett Cannon in issue 18416.)

Example:

Calling module x directly and module y indirectly:

# x.py:
from pathlib import Path
import y
print(__file__)
print(Path(__file__))
print(Path(__file__).resolve())

# y.py:
from pathlib import Path
print(__file__)
print(Path(__file__))

Running python3 x.py will output:

/home/aderchox/mytest/y.py                                                                                                                       
/home/aderchox/mytest/y.py                                                                                                                       
x.py                                                                                                                                             
x.py                                                                                                                                             
/home/aderchox/mytest/x.py

Answer 5

Using __file__ combined with various os.path modules lets all paths be relative the current module's directory location. This allows your modules/projects to be portable to other machines.

In your project you do:

A = '/Users/myname/Projects/mydevproject/somefile.txt'

and then try to deploy it to your server with a deployments directory like /home/web/mydevproject/ then your code won't be able to find the paths correctly.

Answer 6

To add to aderchox's answer, the behaviour of the __file__ variable was again changed in Python 3.9, and now it's an absolute Path in all cases

Running the same example (but copying it here for self consistency)

# x.py:
from pathlib import Path
import y
print(__file__)
print(Path(__file__))
print(Path(__file__).resolve())

# y.py:
from pathlib import Path
print(__file__)
print(Path(__file__))

Now running x.py with two different versions of the interpreter

$ python3.8 x.py
/private/tmp/y.py
/private/tmp/y.py
x.py
x.py
/private/tmp/x.py

$ python3.9 x.py
/private/tmp/y.py
/private/tmp/y.py
/private/tmp/x.py
/private/tmp/x.py
/private/tmp/x.py

source: https://docs.python.org/3/whatsnew/3.9.html#other-language-changes

Answer 7

Necromancy engaged

[ONLY TESTED IN PYTHON 3.7]

I use:

os.sep.join(__file__.split(os.sep)[:-1]

And I prefer it over the other solutions I've seen provided. Some test code I wrote:

from timeit import Timer
from typing import List, Tuple

N = 100000
TIMEITOUT = [None]


def _p_timeit(stmt: str, setup: str = ""):
    tte = Timer(f"TIMEITOUT[0]={stmt}", setup=f"{setup};from __main__ import TIMEITOUT").timeit(N)
    print(f"{stmt:<54}|{tte:>17.10f}    [output]: \"{TIMEITOUT[0]}\"")
    return stmt, tte


def _p_header():
    print(f"Testing w/ number={N} iterations\n{'=' * 72}")
    print(f"{'Statement':<54}|{'Time':^17}\n{'-' * 72}")


def _p_compare(a: Tuple[str, float], b_l: List[Tuple[str, float]]):
    print(f"\n{'=' * 72}\nComparing {a[0]} to all other statements:\n{'-' * 72}")
    for b in b_l:
        d = (b[1]-a[1])/a[1]
        cmp = f"faster than" if d > 0 else f"slower than" if d < 0 else f"equally (t={a[1]}) as fast as"
        print(f"{a[0]} was {(abs(d)*100):.2f}% {cmp} {b[0]}")


_p_header()

my_suggestion = _p_timeit("os.sep.join(__file__.split(os.sep)[:-1])", setup="import os")
others = [_p_timeit("os.path.abspath(os.path.dirname(__file__))", setup="import os"),
          _p_timeit("Path(__file__).parent", setup="from pathlib import Path"),
          _p_timeit("Path(__file__).resolve().parent", setup="from pathlib import Path")]
_p_compare(my_suggestion, others)

Output:

Testing w/ number=100000 iterations
========================================================================
Statement                                             |      Time       
------------------------------------------------------------------------
os.sep.join(__file__.split(os.sep)[:-1])              |     0.0640765000    [output]: "C:\Users\abrewer\AppData\Local\Programs\Python\Python37\lib"
os.path.abspath(os.path.dirname(__file__))            |     0.6156060000    [output]: "C:\Users\abrewer\AppData\Local\Programs\Python\Python37\lib"
Path(__file__).parent                                 |     0.7117664000    [output]: "C:\Users\abrewer\AppData\Local\Programs\Python\Python37\lib"
Path(__file__).resolve().parent                       |     9.3563913000    [output]: "C:\Users\abrewer\AppData\Local\Programs\Python\Python37\Lib"

========================================================================
Comparing os.sep.join(__file__.split(os.sep)[:-1]) to all other statements:
------------------------------------------------------------------------
os.sep.join(__file__.split(os.sep)[:-1]) was 860.74% faster than os.path.abspath(os.path.dirname(__file__))
os.sep.join(__file__.split(os.sep)[:-1]) was 1010.81% faster than Path(__file__).parent
os.sep.join(__file__.split(os.sep)[:-1]) was 14501.91% faster than Path(__file__).resolve().parent

Keep in mind that if __file__ returns a string using any separator (ex: os.altsep) other than os.sep, the program will fail. I haven't had this affect me, but who knows these days...

You could always find the separator in the string first if need be, then use that instead of os.sep. The speed improvements would, as per time complexity standards (time to find the separator is essentially constant), remain.