60

I am trying to create a temporary file that I write in some lines from another file and then make some objects from the data. I am not sure how to find and open the temp file so I can read it. My code:

with tempfile.TemporaryFile() as tmp:
    lines = open(file1).readlines()
    tmp.writelines(lines[2:-1])

dependencyList = []

for line in tmp:
    groupId = textwrap.dedent(line.split(':')[0])
    artifactId = line.split(':')[1]
    version = line.split(':')[3]
    scope = str.strip(line.split(':')[4])
    dependencyObject = depenObj(groupId, artifactId, version, scope)
    dependencyList.append(dependencyObject)
tmp.close()

Essentially I just want to make a middleman temporary document to protect against accidentally overwriting a file.

fractalflame
  • 1,011
  • 1
  • 9
  • 20
  • I've never used temporary files, is there any reason you arn't using the standard `open()` `write` and `read` methods? – kpie Oct 11 '16 at 18:13
  • 1
    I want to protect against the possibility that the filename already exists and I could overwrite it – fractalflame Oct 11 '16 at 18:13
  • 1. have you considered simply piping the output from one script into the input of the second script? 2. Are you checking to be sure the temp file exists in the path that you're looking in? – erapert Oct 11 '16 at 18:14
  • 7
    You have a scope problem. The temporary file `tmp` only exists within the scope of the `with` loop which creates it. – Pierce Darragh Oct 11 '16 at 18:14
  • 1
    Please post your comment as an answer, Pierce Darragh, so I can mod it up. – erapert Oct 11 '16 at 18:15

4 Answers4

64

You've got a scope problem; the file tmp only exists within the scope of the with statement which creates it. Additionally, you'll need to use a NamedTemporaryFile if you want to access the file later outside of the initial with (this gives the OS the ability to access the file). Also, I'm not sure why you're trying to append to a temporary file... since it won't have existed before you instantiate it.

Try this:

import tempfile

tmp = tempfile.NamedTemporaryFile()

# Open the file for writing.
with open(tmp.name, 'w') as f:
    f.write(stuff) # where `stuff` is, y'know... stuff to write (a string)

...

# Open the file for reading.
with open(tmp.name) as f:
    for line in f:
        ... # more things here
Pierce Darragh
  • 2,072
  • 2
  • 16
  • 29
  • 14
    Also make sure that you add "f.seek(0)" after writing to a file if you are going to read from it without closing and reopening it. Otherwise you will be reading the end of the file which will give you wrong result. – jelde015 Nov 01 '17 at 15:01
  • 1
    I get `Permission denied` when trying to open the temp file – stefanbschneider Oct 30 '19 at 15:58
  • @CGFoX this indicates some issue with your filesystem, I think. Your OS has a default location for temporary files, and this is what Python uses for these. For example, on macOS 10.14 Mojave (and, I believe, other versions as well), the location is `/var/folders/`. If you're getting an error about permission being denied, you probably don't have write access to this location. – Pierce Darragh Oct 31 '19 at 20:01
  • `tempfile.NamedTemporaryFile()` has a default `delete=True`, so can be accessed outside scope of `with` or when closed. – Rahul Shenoy Jul 23 '21 at 20:28
  • 1
    You can use `with tempfile.NamedTemporaryFile() as f` directly, skipping `open()`. – ggorlen Oct 26 '22 at 00:50
58

As per the docs, the file is deleted when the TemporaryFile is closed and that happens when you exit the with clause. So... don't exit the with clause. Rewind the file and do your work in the with. 

with tempfile.TemporaryFile() as tmp:
    lines = open(file1).readlines()
    tmp.writelines(lines[2:-1])
    tmp.seek(0)

    for line in tmp:
        groupId = textwrap.dedent(line.split(':')[0])
        artifactId = line.split(':')[1]
        version = line.split(':')[3]
        scope = str.strip(line.split(':')[4])
        dependencyObject = depenObj(groupId, artifactId, version, scope)
        dependencyList.append(dependencyObject)
tdelaney
  • 73,364
  • 6
  • 83
  • 116
20

In case the file needs to be opened a second time, e.g. read by a different process this might cause trouble on Windows OS:

Whether the name can be used to open the file a second time, while the named temporary file is still open, varies across platforms (it can be so used on Unix; it cannot on Windows NT or later).

Hence a safe solution is to create a temporary directory instead and then manually create a file therein:

import os.path
import tempfile

with tempfile.TemporaryDirectory() as td:
    f_name = os.path.join(td, 'test')
    with open(f_name, 'w') as fh:
        fh.write('<content>')
    # Now the file is written and closed and can be used for reading.
a_guest
  • 34,165
  • 12
  • 64
  • 118
1

I just want to make a middleman temporary document to protect against accidentally overwriting a file.

Hopefully you're opening the file in read mode and avoiding arbitrary os.remove() calls, so in the common case there shouldn't be much chance of an accident.

That said, if the normal protections are off for a good reason, you can back up the file by creating a temp file and copying the original file's contents to it with shutil:

import shutil
import tempfile


orig_path = "hello.txt"

# make a sample file
with open(orig_path, "w") as f:
    f.write("hello world")

with tempfile.NamedTemporaryFile() as temp:
    shutil.copy2(orig_path, temp.name)

    # or optionally write to the file:
    #temp.write("hello world".encode("utf-8"))
    #temp.flush()
    #temp.seek(0)

    # temp will be available in this block
    print(temp.read().decode("utf-8")) # => hello world

# temp will be deleted after the block
ggorlen
  • 44,755
  • 7
  • 76
  • 106