Why are the variables inside my .env file resetting to the original file even after changing it from inside my Python script?

Question

I'm trying to implement a checkpoint mechanism for my original script with the following structure:

import os
import sys
from dotenv import load_dotenv

def main():
    load_dotenv()

    checkpoint = int(os.getenv('CHECKPOINT'))

    if checkpoint == 0:
        ...
        do something
        ...
        checkpoint += 1
        os.environ['CHECKPOINT'] = str(checkpoint)

    checkpoint = int(os.getenv('CHECKPOINT'))

    if checkpoint >= 1:
        ...
        do something
        ...
        checkpoint += 1
        os.environ['CHECKPOINT'] = str(checkpoint)

My .env file

CHECKPOINT=0

While this method is working inside a Windows machine (localhost), it is not working on my Ubuntu server.

That is, if my code fails somewhere on the checkpoint 2, I should be able to rerun it from that point only.

For example, if code fails at checkpoint=2:

Windows machine:

checkpoint=2

Ubuntu machine:

checkpoint=0

Also, I am using try and except to catch failures from the code, then exit system if there is any using:

sys.exit()

I can use a config.json to set these from time to time, but I really wanted to know why this is happening.

You want the `checkpoint` value be recorded in .env file or in environment variable when program exits? — balun, Dec 25 '21 at 09:12
I understand what you are trying to say here. I don't want my .env variables to only exits till the program is running but also after its killed. Like in Windows I'm still able to see the updated values after my program is killed. — tblaze, Dec 25 '21 at 09:29
Does the `dotenv` module promise to update the file? Unless that's documented behaviour, I wouldn't expect it. In any case, a [mcve] including precise instructions how you run it would help. That said, please read the descriptions of tags you apply: "linux" is for Linux-specific APIs, "dotenv" is for the Node.js module! — Ulrich Eckhardt, Dec 25 '21 at 10:17
I wrote >linux because this behaviour is specific to linux only while it was working for windows. As for the >dotenv sorry my mistake. — tblaze, Dec 25 '21 at 10:19
Already tried it. But the changes are made only until my program is alive. It resets to its original state after the process is killed. — tblaze, Dec 25 '21 at 10:22
https://docs.python.org/3/library/os.html did you see the part where it says " os.environ ... This mapping is captured the first time the os module is imported,"? — Allan Wind, Dec 25 '21 at 10:23
Is CHECKPOINT initially? Otherwise it will return None on Linux and int(None) is no good. — Allan Wind, Dec 25 '21 at 10:26
Yup it is initialized inside the __name__ == __main__ where it reads from the .evn file. — tblaze, Dec 25 '21 at 10:28
Do you run the program, set environment, then quit and expect the value to be the last value set? On linux environment is stored in process so that is not going to work. — Allan Wind, Dec 25 '21 at 10:35

Allan Wind · Accepted Answer · 2021-12-25T10:51:11.730

Here is a minimal working example:

import os


name = 'CHECKPOINT'


def getenv():
    return int(os.getenv(name) or 0)


def putenv(value):
    print(f'{name} (before): {getenv()}')
    os.environ[name] = str(value)
    print(f'{name} (after {value}): {getenv()}')
    return getenv()


checkpoint = getenv()
if checkpoint == 0:
    checkpoint = putenv(checkpoint + 1)
if checkpoint >= 1:
    checkpoint = putenv(checkpoint + 1)

and when you call it:

$ CHECKPOINT=1 python3 1.py 
CHECKPOINT (before): 1
CHECKPOINT (after setting 2): 2
$ echo -n $CHECKPOINT
$ python3 1.py 
CHECKPOINT (before): 0
CHECKPOINT (after setting 1): 1
CHECKPOINT (before): 1
CHECKPOINT (after setting 2): 2

The CHECKPOINT environment variable is not persisted as it's stored in the first process (python3 1.py) which exits.

Why are the variables inside my .env file resetting to the original file even after changing it from inside my Python script?

1 Answers1