file that is shared between two different pods during deployment

Question

I have scenario when my cluster consist of two microservices.

In service A i have .CSV(15MB) file, which also is needed in service B.

I don't want to place this file two times in each repo.

During deployment of service A I want to place this .csv file in some kind of shared volume, that pod containing service B can consume and process it. Any ideas and best practices how to do it?

Best regards

At that relatively small size, if the data is immutable, build it into both images, and it not, retrieve it using something like an HTTP call at startup time. Trying to share files in a multi-node environment is always tricky, and Kubernetes doesn't really solve the various issues on its own. — David Maze, Mar 24 '23 at 11:00

score 1 · Answer 1 · answered Mar 24 '23 at 11:10

1

The easiest solution would be to build the file into the docker image.

answered Mar 24 '23 at 11:10

Jonas

121,568
97
310
388

score 1 · Answer 2 · answered Mar 24 '23 at 11:31

If you need both microservice that can read and write to a file, then you need a sharedStorage that supports the ReadWriteMany accessMode. There are a couple of options in Kubernetes like:

NFS
CephFS
Glusterfs

You could find more on this topic here.

Another solution would be using object storages like s3.

Harsh Manvar · Answer 3 · 2023-03-24T13:11:31.923

If you dont want to set it in the docker image, you can use the initcontainer to download when POD starts. You can set the file into the hostPath when service A starting.

When you say service A has file not sure if it's in the repo or it's part of Docker image build. If not part of the repo and it's stored at some central place like Bucket you can download it when service B starts with initcontainer.

Example downloading file and set & share to hostpath:

apiVersion: v1
kind: Pod
metadata:
  name: init-demo
spec:
  containers:
  - name: nginx
    image: nginx
    ports:
    - containerPort: 80
    volumeMounts:
    - name: save-data
      mountPath: /data-dir
  initContainers:
  - name: install
    image: ubuntu
    command: ['curl', 'download-link-path >', '/tmp/file.csv']
    volumeMounts:
    - name: save-data
      mountPath: "/data-dir"
  dnsPolicy: Default
  volumes:
  - name: save-data
    hostPath:
      path: /data-dir
      type: Directory

other option is to set the shared file system which multiple PODs can attach and use same time. ReadWriteMany or ReadOnlyMany option.

Minio

is very good option to use for above scenario.

file that is shared between two different pods during deployment

3 Answers3