Named memory-mapped files in Python?

Question

I'm using OpenCV to process some video data in a web service. Before calling OpenCV, the video is already loaded to a bytearray buffer, which I would like to pass to VideoCapture object:

# The following raises cv2.error because it can't convert '_io.BytesIO' to 'str' for 'filename'
cap = cv2.VideoCapture(buffer) 

Unfortunately, VideoCapture() expects a string filename, not a buffer. For now, I'm saving the bytearray to a temporary file, and pass its name to VideoCapture().

Questions:

• Is there a way to create named in-memory files in Python, so I can pacify OpenCV?
• Alternatively, is there another OpenCV API which does support buffers?

Answer 1

Note: POSIX-specific! As you haven't provided OS tag, I assume it's okay.

According to this answer (and this shm_overview manpage) there is /dev/shm always present on the system. That's a tmpfs mapped in a shared (not Python process memory) memory pool, as suggested here, but the plus is that you don't need to create it, so no funny inventing of:

• os.system("mount ...") or
• Popen(["mount", ...]) wrappers.

Simply use tempfile.NamedTemporaryFile() like this:

from tempfile import NamedTemporaryFile
with NamedTemporaryFile(dir="/dev/shm") as file:
    print(file.name)
    # /dev/shm/tmp2m86e0e0

which you could then feed into OpenCV's API wrapper. Alternatively, utilize pyfilesystem as a more extensive wrapper around that device/FS.

Also, multiprocessing.heap.Arena uses it too, so if it didn't work, there'd be much more trouble present. For Windows check this implementation which uses winapi.

For the size of /dev/shm:

• this is one of the size "specifications" I found,
• shm.h, shm_add_rss_swap(), newseg() from Linux source code may hold more details

Judging by sudo ipcs it's most likely the way you want to utilize when sharing stuff between processes if you don't use sockets, pipes or disk.

As it's POSIX, it should work on POSIX-compliant systems, thus also on MacOS(no) or Solaris, but I have no means to try it.

Answer 2

Partially to answer the question: there is no way I know of in python to create named file-like objects which point to memory: that's something for an operating system to do. There is a very easy way to do something very like creating named memory mapped files in most modern *nixs: save the file to /tmp. These days /tmp is almost always a ramdisk. But of course it might be zram (basically a compressed ramdisk) and you likely want to check that first. At any rate it's better than thrashing your disk or depending on os caching.

Incidentally making a dedicated ramdisk is as easy as mount -t tmpfs -o size=1G tmpfs /path/to/tmpfs or similarly with ramfs.

Looking into it I don't think you're going to have much luck with alternative apis either: the use of filenames goes right down to cap.cpp, where we have things like:

VideoCapture::VideoCapture(const String& filename, int apiPreference) : throwOnFail(false)
{
    CV_TRACE_FUNCTION();
    open(filename, apiPreference);
}

It seems the python bindings are just a thin layer on top of this. But I'm willing to be proven wrong!

References

https://github.com/opencv/opencv/blob/master/modules/videoio/src/cap.cpp#L72

Answer 3

If VideoCapture was a regular Python object, and it accepted "file-like objects" in addition to paths, you could feed it a "file-like object", and it could read from that.

Python's StringIO and BytesIO are file-like objects in memory. Something useful to remember ;)

OpenCV specifically expects a file system path there, so that's out of the question.

OpenCV is a library for computer vision. It's not a library for handling video files.

You should look into PyAV. It's a (proper!) wrapper for ffmpeg's libraries. You can feed data directly in there and it will decode. Here are some examples and here are its tests that demonstrate further functionality. Its documentation is thin because most usage is (or should have been...) documented by ffmpeg itself.

Answer 4

You might be able to get away with a named pipe. You can use os.mkfifo to create one, then use the multiprocess module to spawn a background process that feeds the video file into it. Note that mkfifo is not supported on Windows.

The most important limitation is that a pipe does not support seeking, so your video won't be seekable or rewindable either. And whether it actually works might depend on the video format and on the backend (gstreamer, v4l2, ...) that OpenCV is using.