How do you copy a file in Go?

Question

I have the following function to copy a file (io.Reader actually) to the destination string location. However, it seems only part of the file is actually copied resulting in a corrupt file. What am I doing wrong?

func CopyFile(in io.Reader, dst string) (err error) {

    // Does file already exist? Skip
    if _, err := os.Stat(dst); err == nil {
        return nil
    }

    err = nil

    out, err := os.Create(dst)
    if err != nil {
        fmt.Println("Error creating file", err)
        return
    }

    defer func() {
        cerr := out.Close()
        if err == nil {
            err = cerr
        }
    }()


    var bytes int64
    if bytes, err = io.Copy(out, in); err != nil {
        fmt.Println("io.Copy error")
        return
    }
    fmt.Println(bytes)

    err = out.Sync()
    return
}

I'm using this with the filepath.Walk(dir, visit) method to process files in a directory.

// Process each matching file on our walk down the filesystem
func visit(path string, f os.FileInfo, err error) error {

    if reader, err := os.Open(path); err == nil {
        defer reader.Close()

        // http://golang.org/pkg/os/#FileInfo
        statinfo, err := reader.Stat()

        if err != nil {
            fmt.Println(err)
            return nil
        }

        fmt.Println()
        fmt.Println(statinfo.Size())

        // Directory exists and is writable
        err = CopyFile(reader, "/tmp/foo/"+f.Name())

        if err != nil {
            fmt.Println(err)
        }

    } else {
        fmt.Println("Impossible to open the file:", err)
    }
}

The current closest question I could has an accepted answer that recommends using hard/soft links and doesn't abort if the file already exists.

`io.Copy` returns the number of bytes written. You should inspect that, maybe it'll help. — Ainar-G, May 21 '15 at 14:43
Also, your code works for me. Can you show how you use the function? — Ainar-G, May 21 '15 at 14:53
What are you providing to `in`? Are you seeing *any* errors? Does the bad file already exist (you return silently in that case)? — JimB, May 21 '15 at 15:12
@Ainar-G I updated the question with details of using `filepath.Walk()`. @JimB I'm not seeing any errors and the file does not exist before I create it. — Xeoncross, May 21 '15 at 15:13
BTW, your `os.Stat` check for existance followed by `os.Create` (which will truncate existing files) is a race; you can end up truncating and overwriting recently/simultaneously created files. It's better to use [`os.OpenFile`](https://golang.org/pkg/os/#OpenFile) with `os.O_EXCL` and test the error with [`os.IsExist`](https://golang.org/pkg/os/#IsExist) to safely detect existing files. — Dave C, May 21 '15 at 16:46
@DaveC I tried `out, err := os.OpenFile(dst, os.O_WRONLY|os.O_CREATE|os.O_EXCL, 0644)` instead of Stat()+Create() but it still gives corrupted files. [docs here](https://golang.org/pkg/os/#pkg-constants) — Xeoncross, May 21 '15 at 17:18
possible duplicate of [Simple way to copy a file in golang](http://stackoverflow.com/questions/21060945/simple-way-to-copy-a-file-in-golang) — Rick Smith, May 21 '15 at 17:35
@Xeoncross, sorry I didn't mean to imply using `O_EXCL` would fix the issue you asked about; just that as a "by the way" (BTW) your example has a separate issue with a race condition. — Dave C, May 21 '15 at 18:16
@RickSmith that question does not provide sufficient answers — Xeoncross, May 21 '15 at 18:18
@Xeoncross Gotcha, thanks for updating the question to explain the difference. — Rick Smith, May 21 '15 at 18:23
The folder you walk through, do all the files exist and complete in it before you start walking the directory? The code you posted should work. I'm leaning toward OS or file system (or hardware) errors. Can it be the files are modified during the run of your code? — icza, May 22 '15 at 05:26
Was this in a docker image? I am getting similar corruption on a Windows machine and can't work out the issue — Sentinel, Aug 29 '19 at 05:14
Does this answer your question? [How to read/write from/to a file using Go](https://stackoverflow.com/questions/1821811/how-to-read-write-from-to-a-file-using-go) — 030, Sep 05 '22 at 09:55

Apoorva Manjunath · Answer 1 · 2016-12-27T04:32:54.163

14

package main

import (
    "fmt"
    "io"
    "os"
)

func main() {
    srcFile, err := os.Open("test.txt")
    check(err)
    defer srcFile.Close()

    destFile, err := os.Create("test_copy.txt") // creates if file doesn't exist
    check(err)
    defer destFile.Close()

    _, err = io.Copy(destFile, srcFile) // check first var for number of bytes copied
    check(err)

    err = destFile.Sync()
    check(err)
}

func check(err error) {
    if err != nil {
        fmt.Println("Error : %s", err.Error())
        os.Exit(1)
    }
}

This code works for me. Do check the number of bytes copied with the return value from io.Copy.

edited Dec 27 '16 at 04:32

answered Feb 12 '16 at 02:03

Apoorva Manjunath

765
8
13

Just to be clear: One should always check err before using defer. – Matthias Loibl Nov 06 '16 at 18:32
1

Right. Made changes. – Apoorva Manjunath Dec 27 '16 at 04:34
io.Copy efficiently implements the copy operation by calling destFile.ReadFrom(srcFile). It's good practice to call io.Copy as shown in this answer because io.Copy does the "right thing" for any source and destination. – Apr 20 '21 at 15:43

score 0 · Answer 2 · answered Apr 20 '21 at 13:08

Another option is ReadFrom:

package main
import "os"

func copyFile(in, out string) (int64, error) {
   i, e := os.Open(in)
   if e != nil { return 0, e }
   defer i.Close()
   o, e := os.Create(out)
   if e != nil { return 0, e }
   defer o.Close()
   return o.ReadFrom(i)
}

func main() {
   _, e := copyFile("in.txt", "out.txt")
   if e != nil {
      panic(e)
   }
}

https://golang.org/pkg/os#File.ReadFrom

score -2 · Answer 3 · answered May 22 '15 at 22:24

-2

Easiest way to do a copy golang is - http://golang.org/pkg/io/#Copy

answered May 22 '15 at 22:24

Manan

93
6

How do you copy a file in Go?

3 Answers3