I have a library that provides me with a TemporaryFile object which I would like to persist on the file system it was stored, without rewriting it: The file could be quite huge and performance is critical. I am working in an Linux/POSIX environment.
The tempfile documentation explains that the temporary file is created using the O_TMPFILE attribute. Doing this the file has no visible file name in the directory listing.
According to the Linux manpage about O_TMPFILE it is possible to use the C call linkat(fd, "", AT_FDCWD, "/path/for/file", AT_EMPTY_PATH); to persist the file.
How can I achieve this in Python?
(I am aware that python probably unlinks the file, once the file opener is closed. But unlinking, as the name suggest doesn't delete the file, just the record in the directory listing, so the second created listing should remain).
What I have tried
Unfortunately os.link, even so it uses linkstat in the background does not support to be fed with a file link.
import tempfile
import os
# Ensure we save tempfile to the correct block device (the same we want to create the hardlink at)
tempfile.tempdir = "/home/user/tmp"
file = tempfile.TemporaryFile()
# Add some content to the file, so it isn't empty
file.write(b"Das ist ein TEst")
os.link(file, "/home/user/tmp/test.txt")
# -> TypeError: link: src should be string, bytes or os.PathLike, not BufferedRandom
os.link(file.raw, "/home/user/tmp/test.txt")
# -> TypeError: TypeError: link: src should be string, bytes or os.PathLike, not FileIO
os.link(os.link(file.raw.fileno(), "/home/user/tmp/test.txt")
# -> TypeError: link: src should be string, bytes or os.PathLike, not int
I seems I need to call linkstat directly, but I don't know how.
Is this possible without compiling a Python C extension? Maybe using ctypes?
Background
Before people ask, why I do no simply use: NamedTemporaryFile(delete=False) ⇒ I don't have power over the creation of the temporary file.
It is created in Starlette during FileUploads and I want to directly use it without the hassle of creating my own stream file handler.
Starlette already handles the streaming and rendering very well, it just gives no possibility to change the details of the rendering behaviour.
The O_TMPFILE documentation states this option is used for two main reason, one of them:
Creating a file that is initially invisible, which is then populated with data and adjusted to have appropriate filesystem attributes (fchown(2), fchmod(2), fsetxattr(2), etc.) before being atomically linked into the filesystem in a fully formed state (using linkat(2) as described above).
Which is exactly what I want to achieve in my use case.
