How to create a new java.io.File in memory?

Question

How can I create new File (from java.io) in memory, not on the hard disk?

I am using the Java language. I don't want to save the file on the hard drive.

I'm faced with a bad API (java.util.jar.JarFile). It's expecting File file of String filename. I have no file (only byte[] content) and can create temporary file, but it's not beautiful solution. I need to validate the digest of a signed jar.

byte[] content = getContent();
File tempFile = File.createTempFile("tmp", ".tmp");
FileOutputStream fos = new FileOutputStream(tempFile);
fos.write(archiveContent);
JarFile jarFile = new JarFile(tempFile);
Manifest manifest = jarFile.getManifest();

Any examples of how to achieve getting manifest without creating a temporary file would be appreciated.

what's the point of creating a file if you don't want it on persistent memory? — Bhavik Shah, Jul 11 '13 at 13:47
@fge `/dev/null` is on the hard drive (even if it is only a device ;) — Uwe Plonus, Jul 11 '13 at 13:49
has nothing to do with java, but maybe you want a ramdisk? Though i can hardly believe what you are trying to do really needs that... — kutschkem, Jul 11 '13 at 13:49
@UwePlonus Unless you use a [`tmpfs`](http://en.wikipedia.org/wiki/Tmpfs) or a [`Ram Drive`](http://en.wikipedia.org/wiki/RAM_drive) — user000001, Jul 11 '13 at 13:49
@UwePlonus erm, no... Not with modern Linux systems at least (this is a tmpfs) — fge, Jul 11 '13 at 13:51
Also, what do you mean by "save" that stream in memory? If you have Java objects to work with, then the data is already in memory... — Andrzej Doyle, Jul 11 '13 at 14:01
@UwePlonus No, /dev/null isn't anywhere, let alone on the hard disk. It is just a perfect sink and a source of endless EOFs, implemented in software only. — user207421, Jul 14 '13 at 22:32
Similarly, it would be magical if there was a solution (although I know that there is not). I need to interact with an API that expects a File, but I'm interacting from a Google Appengine app, and we all know what GAE thinks of creating Files (hint: it hates it and it's granny) — ndtreviv, Jun 16 '16 at 09:20
Wow, the lack of imagination on StackOverflow is astounding. Someone might need an in-memory File simple because they are using an interface that requires a File. The interface should have been written for an InputStream, but it wasn't. A temp file just seems like overkill, so an in-memory java.io.File would be preferable. — Novaterata, Apr 17 '19 at 21:00
You are not 'faced with a bad API' at all. It's just that this is the only one you've found so far. Have a look at `java.util.jar.JarInputStream`. There is nothing about this problem that requires the contradiction in terms you have dreamed up. — user207421, Sep 05 '21 at 07:53

Andreas Fester · Answer 1 · 2013-07-11T14:05:55.647

97

How can I create new File (from java.io) in memory , not in the hard disk?

Maybe you are confusing File and Stream:

A File is an abstract representation of file and directory pathnames. Using a File object, you can access the file metadata in a file system, and perform some operations on files on this filesystem, like delete or create the file. But the File class does not provide methods to read and write the file contents.
To read and write from a file, you are using a Stream object, like FileInputStream or FileOutputStream. These streams can be created from a File object and then be used to read from and write to the file.

You can create a stream based on a byte buffer which resides in memory, by using a ByteArrayInputStream and a ByteArrayOutputStream to read from and write to a byte buffer in a similar way you read and write from a file. The byte array contains the "File's" content. You do not need a File object then.

Both the File... and the ByteArray... streams inherit from java.io.OutputStream and java.io.InputStream, respectively, so that you can use the common superclass to hide whether you are reading from a file or from a byte array.

edited Jul 11 '13 at 14:05

answered Jul 11 '13 at 13:53

Andreas Fester

36,091
7
95
123

131

I am not sure if this is what the original poster wanted, but one example when a "File in memory" would be useful is when you want to reuse an existing library which needs a java.io.File as an input parameter, but you actually want to use "files" stored in memory. In that case, I think that one solution would be to use Apache Commons VFS to instantiate a DefaultFileSystemManager and use it to create a RamFileProvider. I would have added this as a (more complete) proper answer, but it seems that this question has been closed some time ago... – Sorin Postelnicu Dec 18 '13 at 16:54
2

+1 I was looking for this kind of solution, to unit test come component without having to use an actual file – GClaramunt Dec 20 '13 at 14:10
3

If you don't want to create "an actual file" then you can try creating a temporary file. See `createTempFile` for `java.io.file`. This file can be set to auto delete when unit test exits. – Andrew-Dufresne Aug 16 '14 at 00:37
2

This doesn't answer the question. As @SorinPostelnicu well said a "File" in memory is something like a Filesystem in memory or so. – Felipe Sep 03 '14 at 03:09
1

Probably sums the OP's use case and a valuable method of reducing latency with expensive system disk i/o. – Edward J Beckett Dec 14 '15 at 01:30
4

OP is definitely NOT confusing files and streams. He thinks he wants to use jarFile to read a file and all he has is thebytes from some source (getContent()). – Zweibieren Oct 25 '18 at 21:20
Good explanation, which should be preferred in which case? – Ashish Lohia Nov 08 '18 at 04:44
@AshishLohia It depends - a `File` is the "name" of a file (its location), while a `Stream` is the data in the file. If you want to access the data in a file, you need a `Stream` (you can create `FileInputStream` and `FileOutputStream` from a `File` by passing the file to their constructors). If you want to handle file names and directories you use `File` (Note that since Java 7, there is also a `Paths` and `Files` class in the `java.nio.file` package) – Andreas Fester Nov 08 '18 at 13:16
Thanks Andreas, do we also need to consider the size of the data as well while choosing? Or it doesn't matter that much. – Ashish Lohia Nov 08 '18 at 14:29

Harald K · Accepted Answer · 2023-04-25T11:34:33.647

It is not possible to create a java.io.File that holds its content in (Java heap) memory *.

Instead, normally you would use a stream. To write to a stream, in memory, use:

OutputStream out = new ByteArrayOutputStream();
out.write(...);

But unfortunately, a stream can't be used as input for java.util.jar.JarFile, which as you mention can only use a File or a String containing the path to a valid JAR file. I believe using a temporary file like you currently do is the only option, unless you want to use a different API.

If you are okay using a different API, there is conveniently a class in the same package, named JarInputStream you can use. Simply wrap your archiveContent array in a ByteArrayInputStream, to read the contents of the JAR and extract the manifest:

byte[] content = getContent(); // from OP's code

try (JarInputStream stream = new JarInputStream(new ByteArrayInputStream(content))) {
     Manifest manifest = stream.getManifest();
}

*) It's obviously possible to create a full file-system that resides in memory, like a RAM-disk, but that would still be "on disk" (and not in Java heap memory) as far as the Java process is concerned.

This does not answer the question since ByteArrayOutputStream is not usable with an interface that requires a file object. — FableBlaze, Jan 17 '20 at 12:04
@FableBlaze Well, the answer to the question then, is “it’s not possible”. This answer tries to be pragmatic and tell what the user could do instead. Andreas’ answer is clearly more thorough and a better (attempt at an) answer. — Harald K, Jan 17 '20 at 12:11
"It's not possible" would be a valid answer in my opinion. However i think that an in-memory filesystem (mentioned in the comments of Andreas’ answer) could also be mentioned, because if you cant for some reason write to the hard-drive, then that could be a way around it. — FableBlaze, Jan 17 '20 at 12:19

score 11 · Answer 3 · answered May 21 '20 at 17:33

11

You could use an in-memory filesystem, such as Jimfs

Here's a usage example from their readme:

FileSystem fs = Jimfs.newFileSystem(Configuration.unix());
Path foo = fs.getPath("/foo");
Files.createDirectory(foo);

Path hello = foo.resolve("hello.txt"); // /foo/hello.txt
Files.write(hello, ImmutableList.of("hello world"), StandardCharsets.UTF_8);

answered May 21 '20 at 17:33

Tedward

193
3
6

3

Jimfs does not support `File` object – pixel Jun 02 '21 at 12:25

score 4 · Answer 4 · answered Sep 05 '21 at 06:00

4

I think temporary file can be another solution for that.

File tempFile = File.createTempFile(prefix, suffix, null);
FileOutputStream fos = new FileOutputStream(tempFile);
fos.write(byteArray);

There is a an answer about that here.

answered Sep 05 '21 at 06:00

mipasov

293
2
8

The OP wanted to create the file in memory and not to disk. – Hephaestus Mar 25 '22 at 05:05
1

@Hephaestus: Right, but that's not really doable, so this is the best alternative. – rjcarr Dec 20 '22 at 21:02
1

Not true. Our server code generates hundreds of files per second in memory for writing to S3. Putting this to disk first would a major bottleneck. Just use an InputStream as mentioned above. This is very doable. – Hephaestus Dec 21 '22 at 22:16

How to create a new java.io.File in memory?

4 Answers4

Linked