Using Python tempfiles permanently

Question

I'm currently abusing tempfile a little bit by using it to generate unique names for permanent files. I'm using the following function to get a unique id:

def gen_id():
    tf = tempfile.mktemp()
    tfname = os.path.split(tf)[1]
    return tfname.strip(tempfile.gettempprefix())

Then I'm storing a file in a custom directory with a filename from that function. I use this function to give me more flexibility than the built-ins; with this function I can choose my own directory and remove the tmp prefix.

Since tempfiles are supposed to be "temporary files," are there any dangers to using their uniqueness for permanent files like this? Any reasons why my function would not be safe for generating unique ids?

EDIT: I got the idea to use tempfile for unique names from this SO answer.

The directory will *only* be populated with `tempfiles`, though. Do you mean `tempfile` will look at the directory before generating a name? — Luke Taylor, May 07 '16 at 20:42
Using the deprecated [`tempfiles.mktemp()`](https://docs.python.org/2/library/tempfile.html#tempfile.mktemp) function does not guarantee the uniqueness it sounds like you require. — martineau, May 07 '16 at 20:52

score 2 · Accepted Answer · answered May 07 '16 at 20:49

2

help(tempfile.mktemp) -> "This function is unsafe and should not be used. The file name refers to a file that did not exist at some point, but by the time you get around to creating it, someone else may have beaten you to the punch."

i.e. you could get a filename from this that has the same name as an existing file.

The replacement is tempfile.mkstemp() and it actually does create a file which you normally have to remove after use ... but you can give it a parameter for where your custom directory is and tell it to use no prefix, and let it create the files for you full stop. And it will check for existing files of the same name and make new names until it finds an unused name.

tempfile.mkstemp(suffix="", prefix=template, dir=None, text=False)

(The tempfile module is written in Python, you can see the code for it in \Lib\tempfile.py )

answered May 07 '16 at 20:49

TessellatingHeckler

27,511
4
48
87

I don't think it is deprecated. The comments in the `mktemp()` function explain what's wrong with it, and why it shouldn't be used, and say it's only still there for backwards compatibility with older code that was written to use it. The comments / help for `mkstemp` and `mkdtemp` say nothing like that, so I assume they're as current and valid as anything else - but I can't prove it. – TessellatingHeckler May 07 '16 at 20:55
OK. I see that now, deleted my other comment because I could just look in the docs to see that it wasn't. My one remaining question is: what does "There are no race conditions in the file’s creation" mean? – Luke Taylor May 07 '16 at 20:57
A race condition is a type of bug where two bits of code do the same thing at the same time, and it's luck which one works and which one crashes - a "race to the finish line". Here, two bits of code call mktemp and it could give the same random filename to both; they both try to open the file by that name, and one will luck out and work, the other will crash. Instead, to avoid the race, mkstemp doesn't return a filename until after it has successfully opened the file. Then there is no chance for both bits of code to get the same filename, no race condition, neither crashes. – TessellatingHeckler May 07 '16 at 21:56

score 0 · Answer 2 · edited May 23 '17 at 12:18

I highly suggest just using this comment from the same answer as the way of generating unique names.

No need to abuse mktemp for that (and it's deprecated anyway).

Keep in mind using mktemp guarantees the file name won't exist during the call but if you deleted all of your temp files and cache, or even immediately after the call, the same file (or in case of mktemp the same path) can be created twice.

Using a random choice has less chance in that case to cause collisions and has no downsides. You should however have a check for the small chance a collision occurs, in which case you should generate a new name.

Using Python tempfiles permanently

2 Answers2