6

I am trying to install a pip package from Azure Artifacts as part of a Docker image(with Docker@2 task) but whatever I try does not work.

It looks like my pip inside Docker cannot authenticate against Azure Artifacts whatever I try. Closest I got is with

RUN pip install keyring artifacts-keyring
ENV ARTIFACTS_KEYRING_NONINTERACTIVE_MODE true
RUN pip install <> --index-url https://pkgs.dev.azure.com/<>/_packaging/<>/pypi/simple/

but in my Azure devops, i keep getting 

ERROR: Could not find a version that satisfies the requirement <> (from versions: none)
ERROR: No matching distribution found for <>

Also - Azure documentation on this seems to very poor, if I switch ENV ARTIFACTS_KEYRING_NONINTERACTIVE_MODE false it prompts my Azure DevOps pipeline to authenticate intercatively which is not what I want.

How can I install a Python package published in Azure Artifacts as part of my Azure Pipeline Docker task automatically?

and_apo
  • 1,217
  • 3
  • 17
  • 41

2 Answers2

10

How can I install a Python package published in Azure Artifacts as part of my Azure Pipeline Docker task automatically?

We could use the PipAuthenticate task to populates the PIP_EXTRA_INDEX_URL environment variable:

It authenticates with your artifacts feed and per the docs, will store the location of a config file that can be used to connect in the PYPIRC_PATH environment variable.

Then pass it in the build arg:

arguments: --build-arg INDEX_URL=$(PIP_EXTRA_INDEX_URL)

You could check this document Consuming Azure Pipelines Python artifact feeds in Docker for some more details.

Hope this helps.

Leo Liu
  • 71,098
  • 10
  • 114
  • 135
  • 1
    Thanks - it does help a lot. It worked although for my use case I had to add onlyAddExtraIndex: true on PipAuthenticate@1 task. – and_apo Apr 09 '20 at 11:02
  • I think it is worth pointing out that, as has been discussed in the linked article, the secret ENV variable content of `INDEX_URL` is included in the built Docker image. Fortunately, the author does provide a solution to that. – sh0rtcircuit May 20 '20 at 14:30
  • Everytime the pipeline runs, I am getting a new value for the `PIP_EXTRA_INDEX_URL` - this results in the docker image layer being re-built (cannot take advantage of caching concept). Is there any solution to this? – variable Jun 01 '20 at 05:18
  • 3
    This is fine on Azure, but any suggestion on a smooth workflow to build this in a local dev environment? I can edit pip.conf and authenticate with Azure Artifacts on my host machine using the keyring. But when building the docker container locally, the docker build process cannot authenticate. – Leonard Feehan Jan 27 '21 at 18:06
7

To add to the accepted answer, here is a somewhat more complete code example:

azure-pipelines.yml

- task: PipAuthenticate@1
  inputs:
    artifactFeeds: 'my_artifacts_feed'
    # 'onlyAddExtraIndex' populates PIP_EXTRA_INDEX_URL env variable
    onlyAddExtraIndex: True

- task: Docker@2
  displayName: 'Build Docker Image'
  inputs:
    command: build
    dockerfile: $(dockerfilePath)
    tags: |
      $(tag)
    arguments: --build-arg INDEX_URL=$(PIP_EXTRA_INDEX_URL)

Dockerfile

FROM python:3.8-buster

# add an URL that PIP automatically searches (e.g., Azure Artifact Store URL)
ARG INDEX_URL
ENV PIP_EXTRA_INDEX_URL=$INDEX_URL

COPY requirements.txt
RUN pip install -r requirements.txt
sh0rtcircuit
  • 445
  • 3
  • 13
  • 2
    Note that, if you use `buildAndPush` command instead of `build`, --build-arg will be ignored and thus `PIP_EXTRA_INDEX_URL` will be empty and the pip install will fail – Simon Ndunda Jul 15 '20 at 19:04
  • 1
    This is fine on Azure, but any suggestion on a smooth workflow to build this in a local dev environment? I can edit pip.conf and authenticate with Azure Artifacts on my host machine using the keyring. But when building the docker container locally, the docker build process cannot authenticate. – Leonard Feehan Jan 27 '21 at 18:04
  • @LeonardFeehan you can pass manually the URL with you access token, that you generate via azure – Sonne Nov 01 '21 at 15:42