How to write a safe rename in Go? (Or, how to write this Python in Go?)

Question

I've got the following code in Python:

    if not os.path.exists(src): sys.exit("Does not exist: %s" % src)
    if os.path.exists(dst): sys.exit("Already exists: %s" % dst)
    os.rename(src, dst)

From this question, I understand that there is no direct method to test if a file exists or doesn't exist.

What is the proper way to write the above in Go, including printing out the correct error strings?

Here is the closest I've gotten:

package main

import "fmt"
import "os"

func main() {
    src := "a"
    dst := "b"
    e := os.Rename(src, dst)
    if e != nil {
        fmt.Println(e.(*os.LinkError).Op)
        fmt.Println(e.(*os.LinkError).Old)
        fmt.Println(e.(*os.LinkError).New)
        fmt.Println(e.(*os.LinkError).Err)
    }
}

From the availability of information from the error, where it effectively doesn't tell you what the problem is without you parsing an English freeformat string, it seems to me that it is not possible to write the equivalent in Go.

I think you're misunderstanding the answers to the other question. http://stackoverflow.com/questions/12518876/how-to-check-if-a-file-exists-in-go. Also, doing this is always a race, so there is never a truly safe rename. — JimB, Sep 19 '14 at 18:12
The Go philosophy is to just **try** it and deal with an error if it happens. So, I can write `e := os.Rename(src, dst)` as the first line. But how do I then use `e` to detect whether the src does not exist, or that the destination already exists? — , Sep 19 '14 at 18:17
The Go philosophy has nothing to do with the semantics of a POSIX filesystem. You can certainly check if a file exists, and you can also rename over a file if you try. — JimB, Sep 19 '14 at 18:20
Folks -- I'd like to rename a file. If the source doesn't exist, print one result. If the destination already exists, print another result. Is this possible in Go? I do not see how given how errors are returned in Go. — , Sep 19 '14 at 18:35
@twotwotwo: except if you're trying to *prevent* replacing a file ;) — JimB, Sep 19 '14 at 18:35
@Ana: then what's wrong with Dewey's answer, which is the equivalent to your python. — JimB, Sep 19 '14 at 18:38

score 9 · Answer 1 · edited Jan 07 '20 at 16:39

The code you provide contains a race condition: Between you checking for dst to not exist and copying something into dst, a third party could have created the file dst, causing you to overwrite a file. Either remove the os.path.exists(dst) check because it cannot reliably detect if the target exists at the time you try to remove it, or employ the following algorithm instead:

Create a hardlink from src to dst. If a file named dst exists, the operation will fail and you can bail out. If src does not exist, the operation will fail, too.
Remove src.

The following code implements the two-step algorithm outlined above in Go.

import "os"

func renameAndCheck(src, dst string) error {
    err := os.Link(src, dst)
    if err != nil {
        return err
    }

    return os.Remove(src)
}

You can check for which reason the call to os.Link() failed:

If the error satisfies os.IsNotExist(), the call failed because src did not exist at the time os.Link() was called
If the error satisfies os.IsExist(), the call failed because dst exists at the time os.Link() is called
If the error satisfies os.IsPermission(), the call failed because you don't have sufficient permissions to create a hard link

As far as I know, other reasons (like the file system not supporting the creation of hard links or src and dst being on different file systems) cannot be tested portably.

I had just spent a good three minutes Googling when I saw this, afraid there was just no way to rename in POSIX without risking overwriting something. Yay! — twotwotwo, Sep 19 '14 at 19:11
@twotwotwo Actually, the algorithm above was standard before the `rename()` system call came along. `rename()` was introduced to avoid the race condition when you want to atomically overwrite one file with another. — fuz, Sep 19 '14 at 19:30

Simon Fox · Answer 2 · 2014-09-19T19:19:14.460

The translation if your Python code to Go is:

if _, err := os.Stat(src); err != nil {
    // The source does not exist or some other error accessing the source
    log.Fatal("source:", err)
}
if _, err := os.Stat(dst); !os.IsNotExists(dst) {
    // The destination exists or some other error accessing the destination
    log.Fatal("dest:", err)
}
if err := os.Rename(src, dst); err != nil {
    log.Fatal(err)
}

The three function call sequence is not safe (I am referring to both the original Python version and my replication of it here). The source can be removed or the destination can be created after the checks, but before the rename.

The safe way to move a file is OS dependent. On Windows, you can just call os.Rename(). On Windows, this function will fail if the destination exists or the source does not. On Posix systems, you should link and remove as described in another answer.

Is there some reference as to `os.Rename()` failing on Windows if the destination exists? I've counted on that in the past, and just now I've seen it merrily overwrite the destination on a current Windows 10. — entonio, Jun 08 '22 at 06:21

How to write a safe rename in Go? (Or, how to write this Python in Go?)

2 Answers2