7

I am creating a dockerfile in Azure Devops and is trying to mount Azure Blob Storage Container (which contains files) into the dockerfile. I understand that there is help on microsoft regarding volume but they are in yaml format which is not what I suitable for me as I am using Azure Devops Pipeline to build my Image. I truly appreciate your help. Thanks in advance

Currently my code looks like this

FROM ubuntu:18.04

# Update the repository sources list
RUN apt-get update
FROM python:3.7.5
RUN apt-get update
RUN DEBIAN_FRONTEND=noninteractive 
RUN apt-get install -y  build-essential python3-pip python3-dev postgresql-11 postgresql-contrib ufw sudo libssl-dev libffi-dev 
RUN apt-get install -y  libldap2-dev libsasl2-dev ldap-utils 
RUN apt-get install -y  systemd vim

# Install ODBC driver 17
RUN curl https://packages.microsoft.com/keys/microsoft.asc | apt-key add -
RUN curl https://packages.microsoft.com/config/ubuntu/18.04/prod.list > /etc/apt/sources.list.d/mssql-release.list
RUN apt-get update -y
RUN ACCEPT_EULA=Y apt-get install -y msodbcsql17 mssql-tools unixodbc-dev redis-server rabbitmq-server

# Install python libraries
RUN mkdir /code
WORKDIR /code
COPY requirements.txt /code
RUN pip install --upgrade pip && \
    pip install -r requirements.txt

RUN mkdir /home/workspace
WORKDIR /home/workspace

COPY ./ /home/workspace

# Mount
## *** REQUIRE HELP HERE ***##


RUN service postgresql start
CMD ["python", "home/workspace/python manage.py runserver 0:8000"]
masseyb
  • 3,745
  • 1
  • 17
  • 29
Evelyn Omo
  • 89
  • 1
  • 4
  • 1
    do you find how to do this? – Leo BL Oct 13 '20 at 11:28
  • By mount, do you mean mounting a volume? Because that's done at runtime, not build time. What exactly are you trying to do? – C.Nivs Mar 17 '21 at 16:40
  • `RUN service postgresql start` implies that you want a postgres service *and* a python service running in the same container. That's not how this will work, unfortunately. Your postgres service should be a separate container – C.Nivs Mar 17 '21 at 16:57
  • @risail Evelyn, Please, where are you deploying your application? To ACI, to App Service? – jccampanero Mar 18 '21 at 13:18
  • Im using AKS. Which I don't believe is relevant nor is what or how the services runs the question is specifically for mounting volumes in a docker file – risail Mar 18 '21 at 13:33
  • You don't mount volumes in a dockerfile. You mount a volume into a container. If you want to mount an Azure blob into AKS, I'm not sure that this is directly supported. The supported storage classes are listed [here](https://learn.microsoft.com/en-us/azure/aks/concepts-storage#storage-classes). What you might want is to set up a persistent volume claim with Azure files and use AzCopy to move your blob into the one mounted in AKS – C.Nivs Mar 18 '21 at 15:16
  • Thank you very much for the feedback @risail. Well, in a certain way it could be relevant because Azure provides you some options or not depending on where you deploy the container. Please, consider for instance [this question](https://stackoverflow.com/questions/64563312/unable-to-configure-https-endpoint-for-net-core-kestral-server-in-linux-contain) in which I provided some help: as you can see, if you use docker compose and App Service, Azure brings you the opportunity of mounting a Azure File or Blob Storage in your container.As you can see, the OP in the question uses Azure DevOps as well – jccampanero Mar 18 '21 at 18:07
  • Honesty, I do not if this functionality is supported in AKS. I will dig into it. – jccampanero Mar 18 '21 at 18:09
  • I think you guys are onto something It's better to do this in AKS. I bountied someone elses question which did not include the AKS component. – risail Mar 18 '21 at 18:25
  • @risail AKS and k8s are a very specific use case. Instead of trying to use the AZ CLI to copy content from the storage account to your container, it would be preferable to take advantage of the mechanisms that k8s provides and try to use volume mounts. I dig into the issue and came across [this excellent article](https://zimmergren.net/mount-an-azure-storage-file-share-to-deployments-in-azure-kubernetes-services-aks/). It expose in two companion posts a very similar use case to yours, but with SQLite. It is based in the definition of a volume mount build over an Azure File share ... see next – jccampanero Mar 18 '21 at 22:18
  • And the configuration of the necessary k8s secret based on the storage account access keys required to access the file share. I hope it helps. – jccampanero Mar 18 '21 at 22:20
  • 1
    @jccampanero that article is actually how I ended up doing it. – risail Mar 19 '21 at 13:20
  • Hi @risail. That is nice, I am happy to hear that you were able to find the solution. I you do not mind, I will include an answer summarizing the necessary information. – jccampanero Mar 19 '21 at 14:58

1 Answers1

1

The final runtime in which your container will be deployed can be of relevance because Azure provides you some options or not depending on where you actually deploy it.

Please, consider for instance this question in which I provided some help: as you can see, if you use docker compose and App Service, Azure brings you the opportunity of mounting a Azure File or Blob Storage in your container.

According to your comments you are using AKS. AKS and Kubernetes are a very specific use case. Instead of trying to use the AZ CLI to copy content from the storage account to your container, it would be preferable to take advantage of the mechanisms that Kubernetes provides and try to use volume mounts.

Please, consider for instance read this excellent article. The author exposes in two companion posts a very similar use case to yours, but with SQLite.

Basically, following the guidance provided in the abovementioned post, first, create a Kubernetes Secret with the Azure Storage connection data. You need to provide information about your storage account name and the value of one your access keys. The yaml configuration file will look similar to:

apiVersion: v1
kind: Secret
metadata:
  name: your-secret
  namespace: your-namespace
type: Opaque
data:
  azurestorageaccountname: Base64_Encoded_Value_Here
  azurestorageaccountkey: Base64_Encoded_Value_Here

Then, create the Kubernetes deployment and define the appropriate volume mounts. For example, assuming the name of your Azure Storage file share is your-file-share-name:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: deployment-azurestorage-test
  namespace: your-namespace
spec:  
  selector:
      matchLabels:
        app: your-app
  template:
    metadata:
      labels:
        app: your-app
    spec:
      containers:
      - name: your-app
        image: your-registry.azurecr.io/your-app:latest
        volumeMounts:
        - name: azurefileshare
          mountPath: /postgre-require-dmount-path
      imagePullSecrets:
      - name: your-pull-secret-if-you-have-one       
      volumes:
      - name: azurefileshare
        azureFile:
          secretName: your-secret
          shareName: your-file-share-name
          readOnly: false

The important things to note in the deployment are volumeMounts, used to specify the path in which the file share will be mounted inside the container, the appropriate for you PostgreSQL deployment, and volumes, with the azureFile entry, with information about the secret name created previously and the name of your actual Azure Storage File share.

jccampanero
  • 50,989
  • 3
  • 20
  • 49