91

I am trying to write a Unix script which will truncate/empty a file which is continuously being written/open by an application when it reaches say 3GB of space. I know that the below command would do it : 

cp /dev/null [filename]

But I am going to run this in an production environment automatically as a cron job - just posting here to see if you guys faced any issues while doing something similar to this.

Dan D.
  • 73,243
  • 15
  • 104
  • 123
peedee
  • 1,941
  • 2
  • 14
  • 14
  • These are prod. logs don't have value? I was fortunate to keep my production logs for a year+ (in bzip2 -9 fmt) and had a few small utilities that made it easy to extract job runtimes, records processed, and when loaded into a spreadsheet was easy to calculate and graph loading ratios and a bunch of stuff. We discovered a hdwr problem because of falling load-ratios. Yes, there are lovely monitoring tools out there, so it all depends on your situation. AS A SEPARATE OPTION, have you looked at `logrotate`, it might provide you some value (but be annoying with setting it up (just guessing)!). – shellter Nov 27 '12 at 02:27
  • 5
    Note that if the log file is opened with the O_APPEND flag, then truncation is effective. If it is not opened with the O_APPEND flag, then the program will continue writing at offset 3 GiB (the first time); the system will treat the first 3 GiB as all-bytes-zero (which compresses well), but the file will continue to grow. It all depends on the program writing the log. – Jonathan Leffler Dec 27 '12 at 01:00
  • Possible duplicate of [How to empty ("truncate") a file on linux that already exists and is protected in someway?](http://stackoverflow.com/questions/2423281/how-to-empty-truncate-a-file-on-linux-that-already-exists-and-is-protected-i) – Amit G Nov 16 '16 at 06:25

5 Answers5

126

Just to add another answer, 

: > filename

: is a no-op in bash (POSIX-compliant), so this essentially just opens the file for writing (which of course truncates the file) and then immediately closes it.

EDIT: as shellter commented, you don't actually need a command to go along with the redirection:

$ echo foo > foo.txt
$ cat foo.txt
foo
$ > foo.txt
$ cat foo.txt
$

A simple redirection all by itself will clear the file.

André Chalella
  • 13,788
  • 10
  • 54
  • 62
chepner
  • 497,756
  • 71
  • 530
  • 681
  • @shellter make that an answer. – antik Nov 27 '12 at 02:20
  • @antik : thanks, but see my comment to O.P. As a general rule, I'm against discarding production logs until they've been mined of any useful information. It's not clear if the O.P. is doing that (or, I'll admit, may not need to do it). Thanks and good luck to all. – shellter Nov 27 '12 at 02:33
  • @shellter - I *should* have known :) `exec > filename` was the first thing that popped into my head, and then I was fixated on the idea that there had to be some command to go with the redirection. – chepner Nov 27 '12 at 14:01
  • 8
    @shelter "$ > file" not POSIX and not portable. You should use the ":" no-op command before redirecting, as initially suggested. – Aaron Toponce Jul 02 '13 at 19:59
  • 4
    +plus for smiley face – Chris Suszyński Jul 08 '14 at 13:50
  • Thanks for pointing that out @cardil, maybe it will help me remember this! :>) – J.D. Nov 28 '15 at 16:52
  • 5
    As @AaronToponce poined out, "$ > file" is indeed not portable; on my Zsh setup it makes the shell hang (supposedly waiting for input). "$ : > file" works in Zsh just fine. – Linus Arver Apr 13 '16 at 22:52
  • @opert Not quite. The POSIX spec says "[i]f there is no command name, any redirections shall be performed in a subshell environment; it is unspecified whether this subshell environment is the same one as that used for a command substitution within the command." `zsh`, on the other hand, treats `> file` something like `cat > file` by default. Like most non-standard features of `zsh`, there are various options to modify the behavior. Setting `SH_NULLCMD` causes `zsh` to behave like the POSIX shell (`: > file`) instead. – chepner Apr 13 '16 at 23:03
  • @chepner Thank you for clarifying, although the POSIX language isn't crystal clear to me atm. I dug up the manpages and indeed Zsh treats `> file` like `cat > file`; see `man zshmisc` under the heading `REDIRECTIONS WITH NO COMMAND`. I read the POSIX spec you quoted to mean that the subshell environment is not defined precisely (the last phrase "within the command" hurts my brain). – Linus Arver Apr 14 '16 at 02:56
  • Seriously; that's why I pasted a direct quote rather than trying to paraphrase. I'm not sure what distinction is being drawn in that sentence. – chepner Apr 14 '16 at 03:40
  • Does it take a lock while truncating? In our case we have a java process continuously writing to the file and the shell command is expected to truncate the file. Will there be any contentions? – TechEnthusiast Sep 05 '18 at 10:42
  • 1
    @TechEnthusiast It depends on your expectation. The truncation itself is atomic, but you don't really know when it will happen relative to any particular write. You could, for example, wind up with a partial line at the beginning of the file. – chepner Sep 05 '18 at 11:03
  • @chepner Would there be a chance where the other process errors out while writing while truncate process is truncating. Are the operations serializable? – TechEnthusiast Sep 06 '18 at 03:08
  • 2
    The operations are serialized at the file system level, but writes are not necessarily atomic whereas truncations are. For example, if one process does `echo "foo bar"` and another processes truncates the file, you might have 3 operations `write "foo "`, `truncate`, and `write "bar"`. (With such a small write the odds of it being split up are small, but not zero. The truncation, however, is guaranteed to be a single, uninterruptible operation.) – chepner Sep 06 '18 at 12:43
68

I've used the following command on debian 

truncate -s 0 filename
Ricardo Marimon
  • 10,339
  • 9
  • 52
  • 59
  • @SteveClay: `sudo sh -c ': > filename'` will also truncate `filename` – Perleone Mar 29 '16 at 13:24
  • 2
    @SteveClay The OPs suggestion also works flawlessly with `sudo`: `sudo cp /dev/null filename`. Also `cp` is portable while `truncate` is modern Linux only. – Tino Jun 02 '16 at 14:33
12

That seems reasonable to me. Unix, of course, would let you do this about 50 different ways. Here are two examples:

echo -n "" >filename
cat /dev/null >filename
Don Branson
  • 13,631
  • 10
  • 59
  • 101
3

trunc filename

works on AIX flavor of UNIX

San
  • 3,933
  • 7
  • 34
  • 43
0

There is an other alternative:

dd count=0 of=filename

The advantages (mainly useful for embedded systems):

  • It does not need to spawn shell (bash, dash etc.) because no redirection
  • Compatible to busybox
Richard Buchmann
  • 33
  • 6