1

We are running a golang application in K8 environment, which at one point pushes generated file to an external SFTP server, configured as sftp://user:pass@host:22/upload/sftp2

The code that creates/opens a file on server is ...

    import (
        "github.com/pkg/sftp"
        "golang.org/x/crypto/ssh"
    )

    config := &ssh.ClientConfig{
        User: cred.RemoteUserName,
        Auth: []ssh.AuthMethod{
            ssh.Password(cred.RemotePassword),
        },
        Timeout:         time.Duration(30) * time.Second,
        HostKeyCallback: ssh.InsecureIgnoreHostKey(),
    }

    conn, err := ssh.Dial("tcp", addr, config)
    sftpConn, err := sftp.NewClient(conn)

    destFolder := ""
    if upload.DstFileDir != "." && upload.DstFileDir != "" {
        destFolder = upload.DstFileDir
    }
    dst := path.Join(destFolder, upload.DstFileName)
    log.Printf("Opening dest file: [%s]", dst)

    dstFile, err := sftpConn.OpenFile(dst, os.O_WRONLY|os.O_CREATE|os.O_TRUNC)

where upload.DstFileDir = upload/sftp2 extracted using regex from configured url above.

However we are stumbled upon a weird behaviour where we are getting EOF error at sftpConn.OpenFile statement ...

2021-04-06 00:39:18.459584 I | Opening dest file: [upload/sftp2/sample.001]
2021-04-06 00:39:18.461778 I | Failed uploading file sample.001 due to sftp create/open error [EOF]

... however if we hard code the foldername like below, the file gets uploaded successfully.

dst := path.Join("upload/sftp2", upload.DstFileName)

OR

destFolder := "upload/sftp2"
dst := path.Join(destFolder, upload.DstFileName)

2021-04-06 00:13:00.787883 I | Opening dest file: [upload/sftp2/sample.001]
2021-04-06 00:13:00.787883 I | File /home/user/sample.001 uploaded successfully sftp server.

  • If we debug the code in vscode, all works fine.
  • Apparently this behaviour is observed from running application inside a k8 POD.
  • If we write a standalone golang application just enough to upload a file, and execute the binary from same POD, it works fine.
  • Tried with direct string concatenation without path.Join with no luck.
  • Tried with absolute path, relative path with "./" or "/" or no / at start, with no luck.

Dependencies and their versions:

github.com/pkg/sftp v1.12.0 // indirect
golang.org/x/crypto v0.0.0-20201221181555-eec23a3978ad // indirect
golang.org/x/net v0.0.0-20210119194325-5f4716e94777 // indirect
golang.org/x/sys v0.0.0-20210119212857-b64e53b001e4 // indirect
golang version 1.16.1

Why a specific way of forming a string has an impact on opening a file on sftp server, when path.Join returns a string only?

mavaze
  • 1,187
  • 3
  • 9
  • 12
  • 1
    Try `%q` instead of `%s` in `log.Printf("Opening dest file: [%q]", dst)`. This will catch any hidden rogue characters in the path. – colm.anseo Apr 06 '21 at 19:44
  • Thanks @colm.anseo for suggestion. Yes indeed the problem was the string having invisible characters. The sftp url is AES encrypted string via YANG based CLI (tailf:aes-cfb-128-encrypted-string). With %q I could print the string as sftp://user:pass@host/upload/sftp2\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" and see the source of this issue. Why CLI produced these empty/control characters is of focus now. Your hint to lead in this direction was a time saver. Thank you very much. – mavaze Apr 18 '21 at 14:34

0 Answers0