How to create a file with a given size in Linux?

Question

For testing purposes I have to generate a file of a certain size (to test an upload limit).

What is a command to create a file of a certain size on Linux?

large size http://stackoverflow.com/questions/257844/quickly-create-a-large-file-on-a-linux-system — Ciro Santilli OurBigBook.com, May 02 '16 at 10:40

score 239 · Accepted Answer · edited Jan 13 '21 at 12:45

239

For small files:

dd if=/dev/zero of=upload_test bs=file_size count=1

Where file_size is the size of your test file in bytes.

For big files:

dd if=/dev/zero of=upload_test bs=1M count=size_in_megabytes

edited Jan 13 '21 at 12:45

mit

11,083
11
50
74

answered Sep 26 '08 at 12:53

Ilya Kochetov

17,988
6
44
60

Oh, that might be more efficient than my approach because it does it all in one block. Good idea. – Paul Tomblin Sep 26 '08 at 13:02
12

Actually, using a huge blocksize will perform much worse once it gets very big, as it will allocate and read that amount into memory before writing. If this is somethig like bs=4GiB you'll probably end up swapping. – Brian Sep 29 '08 at 07:40
What about if you use a file as a template and copy its beginning (with `head -c file_size > desired_file`) or its ending (with `tail -c file_size > desired_file`)? Would it be faster? It would also be "randomish". – Carcamano Aug 01 '12 at 22:52
43

`dd` has a limit of 2^32 for its values, so to create a file larger than 4 GB, there's a trick: `dd if=/dev/zero of=test bs=1M count=`. – Dmytro Sirenko May 16 '13 at 05:53
2

See next answer for a better, quicker approach – elegant dice Oct 09 '14 at 01:23
3

@elegantdice it's not clear which answer is next, they can change their position – vladkras Jan 21 '16 at 12:20
1

@vladkras true... i meant jørgensen answer, truncate and fallocate – elegant dice Jan 24 '16 at 03:24
Since you specified `bs` to be `1M` wouldn't that cause the parameter `count` to actually work with MiB(Mebibytes), instead of MB(Megabytes)? – user2347921 Jul 28 '21 at 14:59
Beware when using `dd` that if you use certain devices (e.g. `/dev/random`) it can terminate early resulting in a smaller file. (See here for differences between `/dev/random` and `/dev/urandom`: https://unix.stackexchange.com/questions/324209/when-to-use-dev-random-vs-dev-urandom) – Pierz Oct 18 '21 at 14:58
@DmytroSirenko about `2^32` is it documented? and about `dd if=/dev/zero of=test bs=1M count=` is according your experience or from other source? Thanks. – Manuel Jordan Apr 01 '22 at 15:56

score 218 · Answer 2 · edited Aug 27 '15 at 16:37

218

Please, modern is easier, and faster. On Linux, (pick one)

truncate -s 10G foo
fallocate -l 5G bar

It needs to be stated that truncate on a file system supporting sparse files will create a sparse file and fallocate will not. A sparse file is one where the allocation units that make up the file are not actually allocated until used. The meta-data for the file will however take up some considerable space but likely no where near the actual size of the file. You should consult resources about sparse files for more information as there are advantages and disadvantages to this type of file. A non-sparse file has its blocks (allocation units) allocated ahead of time which means the space is reserved as far as the file system sees it. Also fallocate nor truncate will not set the contents of the file to a specified value like dd, instead the contents of a file allocated with fallocate or truncate may be any trash value that existed in the allocated units during creation and this behavior may or may not be desired. The dd is the slowest because it actually writes the value or chunk of data to the entire file stream as specified with it's command line options.

This behavior could potentially be different - depending on file system used and conformance of that file system to any standard or specification. Therefore it is advised that proper research is done to ensure that the appropriate method is used.

edited Aug 27 '15 at 16:37

kmcguire

874
10
12

answered Jan 03 '12 at 00:25

jørgensen

10,149
2
20
27

1

I tried `truncate`. It produced a zero-sized file using the syntax above. The 'man page' for `fallocate` says that the files it creates are full of empty `space` rather than data. It appears that it wouldn't be useful for some expected cases like "how long does it take to copy a 1G file". – Mark Stosberg Oct 23 '12 at 13:49
7

fallocate seems to work fine for me. It creates a file the right size. – Aater Suleman Nov 01 '12 at 16:47
6

This is the best answer to this question. The truncate/fallocate doesn't take long, because it doesn't write all the file's blocks. But, if you were to then upload the resultant file somewhere, it would read zeroes for the entire thing. – Mike Andrews Jun 26 '14 at 19:50
4

If you want to run this under OSX, then you'll need to do this: `brew install coreutils`. This will add a **g** in front of the command, so you have to run it like this: `gtruncate -s 10G foo`. Hope this helps! – DerekE Dec 13 '15 at 23:07
It seems that it does not work on a `NTFS` partition. – eloyesp Jun 22 '17 at 14:19
On my system (Ubuntu 16.04), `fallocate` fails with `fallocate: fallocate failed: Operation not supported` and leaves a 0 byte file. `truncate` fills with `^@` (`0x00`). – nivk Nov 04 '17 at 16:51
The disclaimer is important and has to be considered. I tried to use it together with df but that fails to count the files to the used part. – JackGrinningCat Feb 21 '19 at 18:06
Both `truncate` and `fallocate` create sparse files, where the `dd` command creates a pre-allocated file (unless dd is used in combination with the `seek=...` option). These files are faster to create (on supportted filesystems), but require more CPU and IO on heavy reading and/or writing. – royarisse Feb 07 '21 at 19:47
Tested on an ext4 file system: For `truncate` it seems that `df` does not care. For `fallocate` I did see an increase of used space (and decrease when I deleted the file). So **`fallocate`** does exactly what I was looking for. – msa Jun 06 '21 at 19:01

score 44 · Answer 3 · edited May 23 '17 at 12:10

44

Just to follow up Tom's post, you can use dd to create sparse files as well:

dd if=/dev/zero of=the_file bs=1 count=0 seek=12345

This will create a file with a "hole" in it on most unixes - the data won't actually be written to disk, or take up any space until something other than zero is written into it.

edited May 23 '17 at 12:10

Community

1
1

answered Sep 26 '08 at 13:11

Brian

116,865
28
107
112

Setting count=0 avoids having to subtract a byte from the file size. – andrewdotn Sep 28 '08 at 01:42
3

with `count=0`, `bs * seek` becomes the file size – Jayen Apr 11 '12 at 11:34

score 31 · Answer 4 · edited Mar 02 '14 at 16:39

31

Use this command:

dd if=$INPUT-FILE of=$OUTPUT-FILE bs=$BLOCK-SIZE count=$NUM-BLOCKS

To create a big (empty) file, set $INPUT-FILE=/dev/zero.
Total size of the file will be $BLOCK-SIZE * $NUM-BLOCKS.
New file created will be $OUTPUT-FILE.

edited Mar 02 '14 at 16:39

jaypal singh

74,723
23
102
147

answered Sep 26 '08 at 12:52

Grundlefleck

124,925
25
94
111

Why did you ask the question? – Henry B Sep 26 '08 at 12:55
17

I had to Google for the answer, so I put it here, so it could be discussed and kept up to date... you know, the point of the whole site? – Grundlefleck Sep 26 '08 at 13:32
4

I know people are voting @Grundlefleck down for XP whoring, but he does have a point - one of the ways to use this site as envisioned by Jeff and Joel is to put a question and answer for something you just discovered. – Paul Tomblin Sep 26 '08 at 16:55
4

Thanks Paul. Though I'm not so bothered about the points, I am bothered about things I find on Google which may be flawed in some way I'd never find out about unless I asked here. People should feel free to make my Q/A community owned if they think I'm whoring, *shrugs*. – Grundlefleck Sep 26 '08 at 17:17
3

To quote from the faq's "It's also perfectly fine to ask and answer your own programming question, but pretend you're on Jeopardy: phrase it in the form of a question." – Craig Angus Sep 27 '08 at 00:08
You might consider first checking if `$OUTPUT_FILE` already exists. Overwriting /etc/passwd with a 2GB file of zeros is discouraging ;-) – Boldewyn Dec 17 '09 at 12:08
@Boldewyn: it's a good point, but I think using the command responsibly is implied ;-p – Grundlefleck Dec 17 '09 at 17:07
@PaulTomblin come one. we are not in a game to get XP for doing something :) – BЈовић Sep 27 '13 at 12:40

score 27 · Answer 5 · answered Jan 02 '12 at 23:32

27

On OSX (and Solaris, apparently), the mkfile command is available as well:

mkfile 10g big_file

This makes a 10 GB file named "big_file". Found this approach here.

answered Jan 02 '12 at 23:32

steve

3,276
27
25

This is useful for situations such as OS X where the `truncate` and `fallocate` commands aren’t available. `dd` also works as described above although it’s `m` for megabytes, not `M`. – user535673 Jan 16 '15 at 15:20
Wrong: This creates a 10 **GiB** file (=~ 10,7 GB). – dessert Nov 14 '19 at 12:59

score 17 · Answer 6 · edited May 23 '17 at 11:55

You can do it programmatically:

#include <unistd.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <stdlib.h>

int main() {
    int fd = creat("/tmp/foo.txt", 0644);
    ftruncate(fd, SIZE_IN_BYTES);
    close(fd);
    return 0;
}

This approach is especially useful to subsequently mmap the file into memory.

use the following command to check that the file has the correct size:

# du -B1 --apparent-size /tmp/foo.txt

Be careful:

# du /tmp/foo.txt

will probably print 0 because it is allocated as Sparse file if supported by your filesystem.

see also: man 2 open and man 2 truncate

score 13 · Answer 7 · answered Sep 16 '13 at 16:50

Some of these answers have you using /dev/zero for the source of your data. If your testing network upload speeds, this may not be the best idea if your application is doing any compression, a file full of zeros compresses really well. Using this command to generate the file

 dd if=/dev/zero of=upload_test bs=10000 count=1

I could compress upload_test down to about 200 bytes. So you could put yourself in a situation where you think your uploading a 10KB file but it would actually be much less.

What I suggest is using /dev/urandom instead of /dev/zero. I couldn't compress the output of /dev/urandom very much at all.

My embedded system doesn't have `/dev/zero`, so `/dev/urandom` is good. — Fredrick Gauss, Aug 26 '14 at 08:40

score 12 · Answer 8 · answered Sep 26 '08 at 12:52

12

you could do:

[dsm@localhost:~]$ perl -e 'print "\0" x 100' > filename.ext

Where you replace 100 with the number of bytes you want written.

answered Sep 26 '08 at 12:52

dsm

10,263
1
38
72

And the actual filling byte as well. I need "\xff" and works fine. Thanks! :) – Ray Dec 19 '14 at 13:23

score 9 · Answer 9 · answered Sep 26 '08 at 12:52

9

dd if=/dev/zero of=my_file.txt count=12345

answered Sep 26 '08 at 12:52

Paul Tomblin

179,021
58
319
408

4

Remember that dd's default block size is 512 bytes, so this command will make a file 12345*512 bytes in size. – nobody Sep 28 '08 at 22:00

score 9 · Answer 10 · answered Jan 01 '20 at 16:05

Use fallocate if you don't want to wait for disk.

Example:

fallocate -l 100G BigFile

Usage:

Usage:
 fallocate [options] <filename>

Preallocate space to, or deallocate space from a file.

Options:
 -c, --collapse-range remove a range from the file
 -d, --dig-holes      detect zeroes and replace with holes
 -i, --insert-range   insert a hole at range, shifting existing data
 -l, --length <num>   length for range operations, in bytes
 -n, --keep-size      maintain the apparent size of the file
 -o, --offset <num>   offset for range operations, in bytes
 -p, --punch-hole     replace a range with a hole (implies -n)
 -z, --zero-range     zero and ensure allocation of a range
 -x, --posix          use posix_fallocate(3) instead of fallocate(2)
 -v, --verbose        verbose mode

 -h, --help           display this help
 -V, --version        display version

score 8 · Answer 11 · answered Jul 21 '16 at 12:55

8

This will generate 4 MB text file with random characters in current directory and its name "4mb.txt" You can change parameters to generate different sizes and names.

base64 /dev/urandom | head -c 4000000 > 4mb.txt

answered Jul 21 '16 at 12:55

Berkay92

552
6
21

score 6 · Answer 12 · answered Sep 27 '13 at 12:54

There are lots of answers, but none explained nicely what else can be done. Looking into man pages for dd, it is possible to better specify the size of a file.

This is going to create /tmp/zero_big_data_file.bin filled with zeros, that has size of 20 megabytes :

    dd if=/dev/zero of=/tmp/zero_big_data_file.bin  bs=1M count=20

This is going to create /tmp/zero_1000bytes_data_file.bin filled with zeros, that has size of 1000 bytes :

    dd if=/dev/zero of=/tmp/zero_1000bytes_data_file.bin  bs=1kB count=1

or

    dd if=/dev/zero of=/tmp/zero_1000bytes_data_file.bin  bs=1000 count=1

In all examples, bs is block size, and count is number of blocks
BLOCKS and BYTES may be followed by the following multiplicative suffixes: c =1, w =2, b =512, kB =1000, K =1024, MB =1000*1000, M =1024*1024, xM =M GB =1000*1000*1000, G =1024*1024*1024, and so on for T, P, E, Z, Y.

score 3 · Answer 13 · edited Oct 24 '12 at 18:52

3

As shell command:

< /dev/zero head -c 1048576 >  output

edited Oct 24 '12 at 18:52

Sven

69,403
10
107
109

answered Oct 24 '12 at 18:25

user1772090

31
1

score 0 · Answer 14 · answered Nov 26 '22 at 18:50

0

Kindly run below command for quickly creating larger file with certain size in linux

for i in {1..10};do fallocate -l 2G filename$i;done

explanation:-Above command will create 10 files with 10GB size in just few seconds.

answered Nov 26 '22 at 18:50

linux.cnf

519
6
7

How to create a file with a given size in Linux?

14 Answers14

Linked

Related