1

I am trying to work more efficiant with sparse files. I have read about the SEEK_HOLE functionallity in newer Linux Kernels. According to other people, this should be in Kernel version 3.1 and later.

However, as seen bellow, there must be something else to it as well. I am on kernel 3.2+ and working with sparse files are still slow.

Doing a "cp" or "tar" on this (completely empty) sparse file should take less than 1 sec.

Any idea what I am missing? How can I check if SEEK_HOLE is supported/activated?

They discuss the same thing here, but for some reason I cannot post comments there that it does not work:

Copying a 1TB sparse file

(root@r1)-(/nbd/test)# uname -a
Linux r1.exice.com 3.2.0-33-generic #52-Ubuntu SMP Thu Oct 18 16:29:15 UTC 2012 x86_64  x86_64 x86_64 GNU/Linux
(root@r1)-(/nbd/test)# dd if=/dev/zero of=10g.img seek=10G bs=1 count=1
1+0 records in
1+0 records out
1 byte (1 B) copied, 0.000271811 s, 3.7 kB/s
(root@r1)-(/nbd/test)# time cp 10g.img new.img

real    0m15.370s
user    0m1.544s
sys     0m13.405s
(root@r1)-(/nbd/test)# time bsdtar cvfz new.tar.gz 10g.img
a 10g.img

real    1m59.898s
user    1m43.938s
sys     0m15.769s
(root@r1)-(/nbd/test)# time tar cvfz new2.tar.gz 10g.img
10g.img

real    1m58.584s
user    1m51.275s
sys     0m30.382s
Community
  • 1
  • 1
Daniele Testa
  • 1,538
  • 3
  • 16
  • 34
  • Yes, I have also noticed this. I was using reiserfs and it refuses to work with it. Switching to ext4 makes it work if I am on 3.5+ kernel. Still does not work on my 3.2 kernel though. Still investigating. – Daniele Testa May 01 '13 at 19:39
  • Ok, seems like bsdtar 3.0.3 does NOT support this at all. Updating to 3.0.4 does work for ext4, but not reiserfs. Same goes for the other tools as well. "cp" version 8.5 does NOT work, but 8.13 do etc. – Daniele Testa May 01 '13 at 20:59
  • Possible duplicate of [Copying a 1TB sparse file](http://stackoverflow.com/questions/13252682/copying-a-1tb-sparse-file) – mvp Jan 31 '17 at 00:45

1 Answers1

2

Make sure you are using a tool that supports SEEK_HOLE.

bsdtar >=3.0.4

cp >=8.13

Also make sure you run a filesystem that supports it. Works fine with ext4. Does not work with reiserfs.

sra
  • 23,820
  • 7
  • 55
  • 89
Daniele Testa
  • 1,538
  • 3
  • 16
  • 34