3

I am copying a local directory into Kubernetes pod using kubectl cp command

kubectl cp test $POD:/tmp

It copies the test directory into Kubernetes pod /tmp directory.

Now I want to overwrite the test directory in the pod. I did not find any option to overwrite directory while copying using kubectl cp command.

Currently, I am deleting the test directory from the pod and then copy the directory. 

kubectl exec $POD -- sh -c 'rm -rf /tmp/test'
kubectl cp test $POD:/tmp

This is working fine, but in case any error comes while copying, existing directory from pod will also be deleted.

How can I overwrite the pod directory with a local directory without deleting the pod directory first?

Thanks in advance.

Kishore
  • 5,761
  • 5
  • 28
  • 53
  • 1
    Remember, all of this work will get undone as soon as the pod gets deleted; if a node fails or you have a HorizontalPodAutoscaler this can happen outside of your control. It'd be better to find a way to accomplish your end goals without relying on imperative commands like `kubectl cp`, whether that's updating the image or mounting persistent storage into the pod. – David Maze Feb 14 '20 at 12:43
  • 1
    I am copying the data in /tmp/test directory. And EFS is mounted on this test directory. So data will not delete when pod terminate. – Kishore Feb 14 '20 at 14:32

3 Answers3

10

Currently there is unfortunatelly no way to achieve your desired state with kubectl cp command.

If there are some undocumented features, please feel free to edit this answer and provide the solution, but currently there is no single place in documentation that could suggest the opposite.

Neither here nor in the context help of the kubectl command, available by running kubectl cp --help, there is no option mentioned that would modify the default operation of the kubectl cp command, which is basically a merge of the content of the already existing directory and copied one.

$ kubectl cp --help

Copy files and directories to and from containers.
    
Examples:
  # !!!Important Note!!!
  # Requires that the 'tar' binary is present in your container
  # image.  If 'tar' is not present, 'kubectl cp' will fail.
  #
  # For advanced use cases, such as symlinks, wildcard expansion or
  # file mode preservation, consider using 'kubectl exec'.
  
  # Copy /tmp/foo local file to /tmp/bar in a remote pod in namespace <some-namespace>
  tar cf - /tmp/foo | kubectl exec -i -n <some-namespace> <some-pod> -- tar xf - -C /tmp/bar
  
  # Copy /tmp/foo from a remote pod to /tmp/bar locally
  kubectl exec -n <some-namespace> <some-pod> -- tar cf - /tmp/foo | tar xf - -C /tmp/bar
  
  # Copy /tmp/foo_dir local directory to /tmp/bar_dir in a remote pod in the default namespace
  kubectl cp /tmp/foo_dir <some-pod>:/tmp/bar_dir
  
  # Copy /tmp/foo local file to /tmp/bar in a remote pod in a specific container
  kubectl cp /tmp/foo <some-pod>:/tmp/bar -c <specific-container>
  
  # Copy /tmp/foo local file to /tmp/bar in a remote pod in namespace <some-namespace>
  kubectl cp /tmp/foo <some-namespace>/<some-pod>:/tmp/bar
  
  # Copy /tmp/foo from a remote pod to /tmp/bar locally
  kubectl cp <some-namespace>/<some-pod>:/tmp/foo /tmp/bar

Options:
    -c, --container='':
    Container name. If omitted, use the kubectl.kubernetes.io/default-container annotation for selecting the
    container to be attached or the first container in the pod will be chosen

    --no-preserve=false:
    The copied file/directory's ownership and permissions will not be preserved in the container

    --retries=0:
    Set number of retries to complete a copy operation from a container. Specify 0 to disable or any negative
    value for infinite retrying. The default is 0 (no retry).

Usage:
  kubectl cp <file-spec-src> <file-spec-dest> [options]

Use "kubectl options" for a list of global command-line options (applies to all commands).

Basically the default behavior of kubectl cp command is a merge of the content of source and destination directory. Let's say we have local directory /tmp/test containing:

/tmp/test$ ls
different_file.txt

with single line of text "some content". If we copy our local /tmp/test directory to /tmp directory on our Pod, which already contains test folder with a different file, let's say testfile.txt, the content of both directories will be merged, so our destination /tmp/test will contain eventually:

/tmp/test# ls
different_file.txt  testfile.txt

If we change the content of our local different_file.txt to "yet another content" and run again the command:

kubectl cp /tmp/test pod-name:/tmp

it will only override the destination different_file.txt which is already present in the destination /tmp/test directory.

Currently there is no way to override this default behavior.

A.Casanova
  • 555
  • 4
  • 16
mario
  • 9,858
  • 1
  • 26
  • 42
  • Thanks for your explanation. Does it possible to add exception handling? e.g. when delete failed, execute the some other command. – Kishore Feb 16 '20 at 10:08
  • 1
    You can use simple **bash scripting** to execute second command only if the first one succeeds e.g.: `kubectl exec $POD -- sh -c 'rm -rf /tmp/test' && kubectl cp test $POD:/tmp`. If delete fails `kubectl cp` will never be executed. You can also do the opposite: `kubectl exec $POD -- sh -c 'rm -rf /tmp/test' || echo "directory couldn't be deleted"` - second comand will be executed only if the first one fails. `kubectl` comands are like other bash commands and have their exit status so they can be scripted pretty easily. – mario Feb 16 '20 at 11:46
  • Thanks, really great workaround. one more thing. There are many subdirectories inside test. Sometimes I am getting error - `rm: cannot remove '/tmp/test/__pycache__': Directory not empty`. But it deletes the remaining all directories and exit by error. And copy command not executes. This creates an issue that no data on k8s pod test, even not the old one. – Kishore Feb 16 '20 at 15:11
  • Do you get this message even when using `rm -rf /tmp/test` ? Basically `-r` flag tells `rm` to delete the directory with the whole content, no matter if it contains files or other directories (which also may contain some other directories or files). Are you sure you're not using simple `rm` without the required flag ? – mario Feb 16 '20 at 15:38
  • This happens in case some other process using that directory - https://unix.stackexchange.com/questions/506319/why-am-i-getting-directory-not-empty-with-rm-rf – Kishore Feb 16 '20 at 15:40
  • Look also at [this](https://askubuntu.com/questions/566474/why-do-i-get-directory-not-empty-when-i-try-to-remove-an-empty-directory) thread. It might be helpful. Second comment under the question explains it pretty well what might be the case. – mario Feb 16 '20 at 15:48
  • It is an issue with some other process running while deleting the __pycache__. When I retry the same command after 2 seconds, it works – Kishore Feb 16 '20 at 15:53
  • You can use `lsof +D directory_name` to list all processes that might be using it. – mario Feb 16 '20 at 15:53
0

It is very possible. I did it. Try this in PowerShell:

PS> $items = ls <dir1>

PS> foreach ($item in $items) { kubectl cp <dir1>/$item podname:var/opt/mount/<dir2>/$item }

PS> if ($? -eq $false) { kubectl cp <dir1>podname:var/opt/mount/<dir2>}

This is for persistent volume so it will exist until volume is destroyed.

The logic to this is:

  1. Get all items for the desired directory
  2. Try writing all items of that desired directory, if destination directory does not exist, then go to second statement in.
  3. Second part, is write desired directory as destination directory name
  4. Instead of running the second part again if you rerun command (which will place the desired directory inside destination directory), you instead write all items to destination directory
Ryan Harlich
  • 157
  • 1
  • 7
-4

Instead of doing "kubectl cp" everytime for your directory, mount your local directory to your pod using "volume mounts".

Anmol Agrawal
  • 814
  • 4
  • 6
  • This won't work well in a cluster environment, especially if nodes will be dynamically created and destroyed in response to load or other conditions. – David Maze Feb 14 '20 at 12:44
  • Yes. I agree. But he is trying `kubectl cp` which is also not a good approach for the same. Ultimately he needs to use volume and mount in pod. `hostPath` was just a suggestion in the next line. – Anmol Agrawal Feb 14 '20 at 14:03
  • In most cluster configuration using "volume mounts" will not work if you wan to mount source code of you workstation inside a pod. Copying the code is a more generic option and remains much more simple than using https://www.telepresence.io, which might be the more powerful technique. – Fabrice Jammes May 28 '20 at 06:12