How to detect deleted file?

Question

Writing to a non-existent file does not produce an error in Go.

For example, here's a sample program writing to a file in a loop:

package main

import (
    "log"
    "os"
    "time"
)

func main() {

    f, err := os.OpenFile("mytest.log", os.O_APPEND|os.O_CREATE|os.O_WRONLY, 0644)
    if err != nil {
        log.Fatal(err)
    }

    for {

        n, err := f.WriteString("blah\n")
        if err != nil {
            log.Fatal(err)
        }
        log.Printf("wrote %d bytes\n", n)
        time.Sleep(2 * time.Second)
    }
}

While this is running, I issue rm mytest.log from the command line and observe that the program does not produce an error on the next call to WriteString(). (I tested on Linux, it may be different for other OS's)

Is there a way to detect if the file was deleted (other than doing a stat on the file before every write)? And presumably the bytes written are simply discarded by the operating system?

The bytes are not discarded. You can rewind to the beginning of the file and read it back. Creating and then immediately unlinking a file is a common pattern for dealing with temporary files. — Peter, Nov 07 '18 at 06:45
It is because you open a port using os.Openfile() and writing to that particular port no more error check is performed during the write operation. Because of that you won't get file not found error during the write operation. — ASHWIN RAJEEV, Nov 07 '18 at 07:54
On most Unix filesystems, deleting a file just removes the inode entry, leaving the underlying file in tact, until it is closed. This is a feature, which can be used to great advantage in some cases, but the TL;DR; is: The situation you're describing is intentional. — Jonathan Hall, Nov 07 '18 at 07:56
Perhaps you should explain why you care if the file gets deleted, so we can suggest appropriate countermeasures. As long as you don't Close() the file, all *os.File methods should JustWork(tm). Only once you start referring to the file by name will you get problems, but every time you do that you have to handle (expect, even!) errors anyway. — Peter, Nov 07 '18 at 07:56

datenwolf · Accepted Answer · 2018-11-07T10:28:28.683

While this is running, I issue rm mytest.log from the command line and observe that the program does not produce an error on the next call to WriteString()

Yes, that's exactly the behavior that's specified. Also the file hasn't been removed. The only thing that rm does remove is that particular path entry in the filesystem. A single file can have multiple paths, also called hardlinks.

The actual file is deleted only, when the last reference to it, either by filesystem entry (link) or by file descriptor (file open in a program) has been closed.

This particular behavior of the Unix file model was used for a long time to implement "unnamed" shared memory, by creating and opening a file in /dev/shm and then removing the filesystem entry – because this particular way of doing things introduces a race condition, for security sensitive applications new syscalls were introduced, that allow creating anonymous memory maps, and very recently Linux even got a function to create a file in a filesystem, without creating a path entry (open with O_TMPFILE flag).

On more recent versions of Linux you can even re-/create filesystem entries for files which last entry already was removed using the linkat syscall.

Update

The question is, do you really want to error out if the last filesystem entry vanishes? It's not a bad condition after all, you can safely write and read, without problems, just be aware, that once you close the last file descriptor to the file, it will be lost.

It is perfectly possible to detect if the last filesystem entry has been removed and abort file operations if so – however be aware, that such code might introduce it's very own share of problems, for example if the program expects to create a new filesystem entry, once everything has been written to the file properly, using linkat.

Anyway, what you can do, is fstat-ing the file (file.Stat in Go) and look at the number of hardlinks the file has. If that number drops to zero, all filesystem entries are gone. Actually getting that number is a little bit tricky in Go, it's described here Counting hard links to a file in Go

package main

import (
    "fmt"
    "log"
    "os"
    "syscall"
    "time"
)

func main() {
    fmt.Println("Test Operation")
    f, err := os.OpenFile("test.txt", os.O_CREATE|os.O_APPEND|os.O_WRONLY, 0644)
    if err != nil {
        log.Fatal(err)
    }

    for {

        n, err := f.WriteString("blah\n")
        if err != nil {
            log.Fatal(err)
        }
        log.Printf("wrote %d bytes\n", n)
        time.Sleep(2 * time.Second)
        stat, err := f.Stat()
        if err != nil{
            log.Fatal(err)
        }
        if sys := stat.Sys(); sys != nil {
            if stat, ok := sys.(*syscall.Stat_t); ok {
                nlink := uint64(stat.Nlink)
                if 0 == nlink {
                    log.Printf("All filesystem entries to original file removed, exiting")
                    break
                }
            }
        }
    }
}

How to detect deleted file?

1 Answers1

Update