Azure DevOps build Dockerfile while projects have their own repository in Azure DevOps

Question

I'm currently dealing with a problem within my Azure DevOps pipeline where I'm trying to run and build a pipeline for repo 'RepoX', whereafter the Dockerfile tries to use the '.csproj' of repo 'RepoY'. However, 'RepoY' is added as 'existing project reference' to 'RepoX' within Visual Studio. When the dockerfile tries to use the .csproj of repo 'Y' I get a message that this file cannot be found. This is true, since both are located in different repositories, thus are separate projects. How can I make sure RepoX clones repo Y somehow or is able to use RepoY, so that the Dockerfile finds the .csproj of repo Y and builds it in order to push it to, for example Docker hub?

Projects overview

Azure Repo: RepoX
  ------ [RepoX directory]
  ------------ Controllers
  ------------ Properties
  ------------ appsettings.Development.json
  ------------ appsettings.json
  ------------ Program.cs
  ------------ RepoX.csproj
  ------ .dockerignore
  ------ .gitignore
  ------ azure-pipelines.yml
  ------ Dockerfile
  ------ RepoX.sln

Azure Repo: RepoY (Class Library)
  ------ [CoreClasses directory]
  ------------ ClassA
  ------------ ClassB

I tried to use checkout to retrieve the repoY in repoX, but somehow this is not persistent when the Dockerfile is trying to search the .csproj of repoY?

azure-pipelines.yml:

trigger:
- main

# resources:
# - repo: self <---------------- tried to turn this off as well, makes no difference...

variables:
  tag: '$(Build.BuildId)'

stages:
- stage: Build
  displayName: Build image
  jobs:
  - job: Build
    displayName: Build
    pool:
      vmImage: ubuntu-latest
    - task: Docker@2
      - checkout: self
        path: s/CheckOutFolder
      - checkout: git://ProjectA/RepoY
        path: s/CheckOutFolder/RepoX/RepoY
      - script: |
          ls
      displayName: Build an image
      inputs:
        command: build
        dockerfile: '$(Build.SourcesDirectory)/CheckOutFolder/Dockerfile'
        tags: |
          $(tag)

Dockerfile:

#See https://aka.ms/containerfastmode to understand how Visual Studio uses this Dockerfile to build your images for faster debugging.

FROM mcr.microsoft.com/dotnet/aspnet:6.0 AS base
WORKDIR /app
EXPOSE 80
EXPOSE 443

FROM mcr.microsoft.com/dotnet/sdk:6.0 AS build
WORKDIR /src

COPY ["RepoX/RepoX.csproj", "RepoX/"]
RUN dotnet restore "RepoX/RepoX.csproj"
#RUN ls ../home/vsts/work/1/s/                                  <----- says no such directory exists...
#RUN ls ../home/vsts/work/1/s/CheckOutFolder/RepoX/RepoY        <----- says no such directory exists...
#RUN ls RepoX                                   <----- does not contain files of RepoY, although I used checkout? I don't get it..

COPY ["RepoX/RepoY/RepoY.csproj", "RepoX/"]                 <----- says cannot find file...
RUN dotnet restore "RepoX/RepoY.csproj"

COPY . .

WORKDIR "/src/RepoX"
RUN dotnet build "RepoX.csproj" -c Release -o /app/build

FROM build AS publish
RUN dotnet publish "RepoX.csproj" -c Release -o /app/publish

FROM base AS final
WORKDIR /app
COPY --from=publish /app/publish .
ENTRYPOINT ["dotnet", "RepoX.dll"]

Can someone please help me with this?

Suki Ji-MSFT · Answer 1 · 2022-11-22T09:45:19.613

1

If RepoX and RepoY are in different project but in the same organization, you could use Inline syntax checkout to download those two repos like below:

steps:
- checkout: self
- checkout: git://MyProject/RepoY # Azure Repos Git repository in the same organization

Then, your source code is checked out into directories named after the repositories as a subfolder of s in (Agent.BuildDirectory). If (Agent.BuildDirectory) is C:\agent\_work\1 and your repositories are named RepoX and RepoY, your code is checked out to C:\agent\_work\1\s\RepoX and C:\agent\_work\1\s\RepoY. Please define the right directory for your docker file. Here is the reference document: Inline syntax checkout.

Update:

You could add a copy file task to copy the .csproj file from RepoY to the folder where the dockerfile is located or even deeper by using match patterns. Here is an YAML example like below.

- task: CopyFiles@2
  inputs:
    SourceFolder: 'C:\agent\_work\1\s\RepoY'
    Contents: '**.csproj'
    TargetFolder: 'C:\agent\_work\1\s\RepoX\...\'

edited Nov 22 '22 at 09:45

answered Nov 22 '22 at 03:32

Suki Ji-MSFT

603
2
5

The dockerfile would be in 'C:\agent\_work\1\s\RepoX' given by your example. How can the dockerfile copy from this path? I think this is not possible since the dockerfile can only dig deeper (go inside other folders where the dockerfile is located). I cannot go one directory up to copy something from 'C:\agent\_work\1\s\RepoY'? Or am I missing something here... If this is possible, can you please give me an example so I can see how this is achieved? – user20291437 Nov 22 '22 at 09:22
Hi @user20291437, I just update my answer you could use a copy file task to copy the file from RepoY to RepoX. – Suki Ji-MSFT Nov 22 '22 at 09:48
I have some questions, but your example really helps what I was maybe missing. However, I'm wondering if I only need the '.csproj' file, **because my RepoY obviously contains some classes**, but when executing your example only '.csproj' is being copied? Also my RepoY **depends on others repos RepoA and RepoB**. Is it needed to also do something with them, because only coping RepoY while that repo depends on other repos as well can still not make image that is built with the dockerfile not fully operational? – user20291437 Nov 22 '22 at 10:50
Can you also give an example for the dockerfile, because it seems to be that the copied 'csproj' is not present when doing **RUN ls** in **WORKDIR /src** – user20291437 Nov 22 '22 at 10:56
I really want to thank you Suki for your answer, because your examples were helpful and enlightened me very well. Although I couldn't exactly do it the way you described it and CopyFiles task wasn't even needed. Copying the files made it even worse with the way I set up my Dockerfile by copying ./ ./ into the /src folder. I decided to just stick with the checked out projects into the root folder. Now everything is working as intended and I'm able to build a Docker image. I will share my files to this question. – user20291437 Nov 22 '22 at 21:04

user20291437 · Accepted Answer · 2022-11-29T00:00:08.210

1

I finally was able to solve the problem by the following .yml and Dockerfile configuration:

azure-pipelines.yml

trigger:
- main
resources:
- repo: self
variables:
  tag: '$(Build.BuildId)'
stages:
- stage: Build
  displayName: Build image
  jobs:
  - job: Build
    displayName: Build
    pool:
      vmImage: ubuntu-latest
    steps:
    # Copy current RepoX and RepoY projects
    # to a new (root) folder
    - checkout: self
    # change <MyProject> including removing the open/closed angled brackets to your project name
    - checkout: git://<MyProject>/RepoY
    - task: Docker@2
      displayName: Build an image
      inputs:
        command: build
        dockerfile: 'RepoX/Dockerfile'
        useDefaultContext: false
        # /root folder (contains: RepoX and RepoY)
        # Should be different from where the Dockerfile is located, 
        # because the build context should be placed on a higher 
        # level of directory structure to be able to access 
        # 'RepoY' project folder.
        buildContext: $(Build.Repository.LocalPath)
        tags: |
          $(tag)

Dockerfile

#See https://aka.ms/containerfastmode to understand how Visual Studio uses this Dockerfile to build your images for faster debugging.
FROM mcr.microsoft.com/dotnet/aspnet:6.0 AS base

WORKDIR /app
EXPOSE 80
EXPOSE 443

FROM mcr.microsoft.com/dotnet/sdk:6.0 AS build
WORKDIR /src

# Copy RepoX and RepoY projects that are checked out in azure-pipeline.yml to /src
COPY . .

# Not needed to copy RepoX project to a new directory.. just restore the project.
#COPY ["RepoX/RepoX/RepoX.csproj", "RepoX/"]
RUN dotnet restore "RepoX/RepoX/RepoX.csproj"

WORKDIR "/src/RepoX/RepoX"
RUN dotnet build "RepoX.csproj" -c Release -o /app/build

FROM build AS publish
RUN dotnet publish "RepoX.csproj" -c Release -o /app/publish

FROM base AS final
WORKDIR /app
COPY --from=publish /app/publish .
ENTRYPOINT ["dotnet", "RepoX.dll"]

References:

edited Nov 29 '22 at 00:00

answered Nov 22 '22 at 21:19

user20291437

99
1
8

You should remember if you have used 'Add project reference' to your solution by adding an existing project to your main project. Let's say you had some class libraries that you added as a project reference to your main project. Make sure to always check out those other repositories from the same Azure DevOps project (in my case) and put the build context path outside your Dockerfile to be able to access those repositories outside your main project. Hopefully, this might help you guys as well. – user20291437 Nov 22 '22 at 21:26
Initial reason that I did not create those class libraries within the main project was that I had logic that had to be shared across multiple other projects. Therefore, I created a core class library that was being used by various main projects. – user20291437 Nov 22 '22 at 21:27
Another mis-usage of Google was that I was asking the wrong question on Google... Finally, I realized that it had to do something with the 'build context' as I figured this out by googling a lot. In the end of my journey for solving this problem, I asked the following question on Google: "dockerfile build context parent directory". As simple as it sounds, if you do not know what to search for it could be a rabbit hole... – user20291437 Nov 22 '22 at 21:32
Hi @user20291437, as it resolved now, you could [mark your answer](https://meta.stackexchange.com/questions/5234/how-does-accepting-an-answer-work/5235#5235) to benefit those who with the similar issue. – Suki Ji-MSFT Nov 23 '22 at 01:21
Thank you for sharing @user20291437. I assume - based on your response - that this is how your Dockerfile accesses a GitHub repository checked out by the pipeline and activated using 'RUN dotnet restore'? I am trying to est. a GH private repo into a Dockerfile, and thought I could refer to GH repo as a resource w/Docker@2 - e.g., via buildContext or other. Not been able to find any concrete example outside this one you presented. Any feedback appreciated! – gto406 Nov 28 '22 at 18:23
1

Yes indeed @gto406. By copying everything to /src through COPY . . I'm able to restore my referenced projects from RepoY in RepoX. However, as I'm showing in my example the Dockerfile then should not be in the same directory of your '.csproj' file but outside that directory. On the other hand, by checking out 'self' and 'RepoY' they are being copied to the current directory. Then I'm able to copy them through COPY . . and I can act on this through the Dockerfile. Also both projects RepoX and RepoY exists within the same Azure DevOps project and are private repos as well. – user20291437 Nov 29 '22 at 00:04
If you wanna do this through your Dockerfie instead of the pipeline, what you could do is to use the git copy command, something like this: https://dunglas.dev/2022/08/securely-access-private-git-repositories-and-composer-packages-in-docker-builds/ or this https://stackoverflow.com/a/29464430/20291437 – user20291437 Nov 29 '22 at 00:08
1

I agree @user20291437. I thought more on this 'approach' and it makes more sense to have the pipeline do the 'git' clone/extract operation(s). The Dockerfile/image is then focused exclusively on establishing the environment under which the checked-out git repo runs (tests, etc.). Assuming Dockerfile (as you noted) is not in the same place as .csproj. 8^) – gto406 Dec 02 '22 at 17:10

Azure DevOps build Dockerfile while projects have their own repository in Azure DevOps

2 Answers2