7

I'd like to access and edit files in my Kubernetes PersistentVolume on my local computer (macOS), but I cannot understand where to find those files!

I'm pointing my hostPath to /tmp/wordpress-volume but I cannot find it anywhere. What is the hidden secret I'm missing

I'm using the following configuration on a docker-for-desktop cluster Version 2.0.0.2 (30215).

PersistentVolume

kind: PersistentVolume
metadata:
  name: wordpress-volume
spec:
  # ...
  hostPath:
    path: /tmp/wordpress-volume

PersistentVolumeClaim

kind: PersistentVolumeClaim
metadata:
  name: wordpress-volume-claim
# ...

Deployment

kind: Deployment
metadata:
  name: wordpress
# ...
spec:
  containers:
  - image: wordpress:4.8-apache
    # ...
    volumeMounts:
    - name: wordpress-volume
      mountPath: /var/www/html
  volumes:
  - name: wordpress-volume
    persistentVolumeClaim:
      claimName: wordpress-volume-claim
a.barbieri
  • 2,398
  • 3
  • 30
  • 58

4 Answers4

10

Thanks to @aman-tuladhar and some hours lost on the internet I've found out that you just need to make sure storageClassName is set for you PersistentVolume and PersistentVolumeClaim.

As per documentation if you want to avoid that Kubernetes dynamically generetes PersistentVolumes without considering the one you statically declared, you can just set a empty string " ".

In my case I've set storageClassName: manual.

PersistentVolume

kind: PersistentVolume
metadata:
  name: wordpress-volume
spec:
  # ...
  storageClassName: manual
  hostPath:
    path: /tmp/wordpress-volume

PersistentVolumeClaim

kind: PersistentVolumeClaim
metadata:
  name: wordpress-volume-claim
spec:
  storageClassName: manual
  # ...

This works out of the box with docker-for-desktop cluster (as long as mountPath is set to a absolute path).

References:

a.barbieri
  • 2,398
  • 3
  • 30
  • 58
  • This answer doesn't work in the latest version of Docker Desktop for macOS (v4.1.1 as on date). It ends up using the said folder from within the Docker VM. That said, the local-storage seems to be doing the same thing. – Rads Nov 23 '21 at 02:07
  • 2
    Correcting my partially incorrect comment. The solution works, however, does not work for any non-user owned directory. Docker Desktop silently mounts the folder from Docker VM if it is not a user owned directory chain (example: /tmp/xyz). However, works for /Users/$(whoami)/. It is extremely frustrating that this mechanism is not documented anywhere. – Rads Nov 23 '21 at 02:16
  • For anyone surprised they can’t find the docs on this: `manual` can be pretty much any string, e.g., `foo`, and this still works. As well you’ll probably want to [add some selectors](https://kubernetes.io/docs/concepts/storage/persistent-volumes/#selector) to ensure the PVC will only get attached to the intended PV. – andrewdotn Aug 15 '22 at 20:06
  • PS: the pvc spec `volumeName` only ‘works’ when specified at resource creation time if the pv spec `claimRef.name` and `claimRef.namespace` for the intended pv match. But if you leave it blank you can read the pv name from that field once it is bound. – andrewdotn Aug 15 '22 at 20:31
4

First this you need to remember is that Kubernetes is running on minikube cluster. minikube itself run on Virtual Machine. So that path won't be on you host machine, rather it is the path in Virtual Machine.

But with minikube we have easy way to do this. First you have to mount host directory to minikube.

(If you are using cloud providers you will have some way to create a storage. For GCE you have gcePersistentDisk)

minikube mount /path/to/dir/to/mount:/vm-mount-path

Now

kind: PersistentVolume
metadata:
  name: wordpress-volume
spec:
  # ...
  hostPath:
    path: /vm-mount-path

If you create this resource this should save file in your host machine.

Follow this minikube documentation for more detail

A0__oN
  • 8,740
  • 6
  • 40
  • 61
  • I was trying to use [docker-for-desktop](https://blog.docker.com/2018/07/kubernetes-is-now-available-in-docker-desktop-stable-channel/), but at this point I'll switch to *Minikube* which seems to have more documentation on the matter. – a.barbieri Feb 13 '19 at 20:56
  • Uh, do you remember where you saw it? I found out [this article](https://medium.com/@snowmiser/kubernetes-binding-persistentvolumes-and-persistentvolumeclaims-33323b907722) that says: "in order for the pvc to actually bind to the pv, you need to give both of them the same storageClassName in the yaml definition." – a.barbieri Feb 13 '19 at 21:15
  • I don't remember where but, if you are not using StorageClass Resource may be you need to add `spec.storageClassName: ""` in a yaml where you are creating PVC. It ensures to use existing PV instead of creating new one. – A0__oN Feb 13 '19 at 21:23
  • 1
    Yes, I've found another article talking about the same thing [here](https://medium.com/@xcoulon/storing-data-into-persistent-volumes-on-kubernetes-fb155da16666) – a.barbieri Feb 13 '19 at 21:24
3

In case of MacOS and Kubernetes inside Docker for Mac. How to find a real location of dir-based local volume into VM

1) Create a new PersistentVolume with unique path:

blablabla.yml:

kind: PersistentVolume
apiVersion: v1
metadata:
  name: blablabla
spec:
  storageClassName: manual
  capacity:
    storage: 1G
  accessModes:
    - ReadWriteMany
  hostPath:
    path: "/mnt/blablabla"

kubectl apply -f blablabla.yml

2) Log into VM:

screen ~/Library/Containers/com.docker.docker/Data/vms/0/tty
# then press Enter

3) Find your volume:

find / -name blablabla
/containers/services/docker/rootfs/mnt/blablabla # <= got it!
/containers/services/docker/tmp/upper/mnt/blablabla

4) Exit from screen: Ctrl-a k y, Detach from screen: Ctrl-a d

Note

Sometimes you have a chance to get a broken screen session, it seems like a garbaged stdout with messed symbols, stdin still works fine. In that case, try to terminate all screen sessions and reconnect to the first one. Or just restart you docker for mac.

a0s
  • 479
  • 4
  • 8
  • In case of the question path (/tmp/wordpress-volume) it could be find on /containers/services/docker/rootfs/tmp/wordpress-volume. Having a K8S IDE like Lens allows you to log into the VM/Node without using screen. – Shondeslitch Jan 31 '21 at 11:18
  • In latest macOS versions tty (from Step 2) will not work. Alternate is; docker run -it --privileged --pid=host justincormack/nsenter1 – abhinavroy23 Oct 01 '21 at 20:35
2

I create a PersistentVolume and use the storageClassName (local-storage in my example below). Make sure to replace the path (/Users/user/data-pv1) with an actual path on your Mac.

apiVersion: v1
kind: PersistentVolume
metadata:
  name: local-pv1
spec:
  capacity:
    storage: 30Gi
  accessModes:
  - ReadWriteOnce
  persistentVolumeReclaimPolicy: Retain
  storageClassName: local-storage
  local:
    path: /Users/user/data-pv1
  nodeAffinity:
    required:
      nodeSelectorTerms:
      - matchExpressions:
        - key: kubernetes.io/hostname
          operator: In
          values:
          - docker-desktop

If you'd like to access the hostPath, you need to use nsenter:

docker run -it --rm --privileged --pid=host alpine:edge nsenter -t 1 -m -u -n -i sh

or nsenter1

docker run -it --rm --privileged --pid=host justincormack/nsenter1

and navigate to the following directory:

/var/lib/k8s-pvs
h q
  • 1,168
  • 2
  • 10
  • 23
  • Thank you sir, this answer fixed it for me. Pay close attention to `/spec/local/path` Most examples use a hostPath/path setting, which only created the folder on the docker container host. – Michel Rugenbrink Oct 19 '21 at 14:00
  • Correcting my partially incorrect comment. The answer (and the answer marked as the accepted, works, however, does not work for any non-user owned directory. Docker Desktop silently mounts the folder from Docker VM if it is not a user owned directory chain (example: /tmp/xyz). However, works for /Users/$(whoami)/. It is extremely frustrating that this mechanism is not documented anywhere. – Rads Nov 23 '21 at 02:17