8

A sample repo with the directory structure of what I'm working on is on GitHub here. To run the GitHub Action, you just need to go to the Action tab of the repo and run the Action manually.


I have a custom GitHub Action I've written as well with python as the base image in the Docker container but want the python version to be an input for the GitHub Action. In order to do so, I am creating a second intermediate Docker container to run with the python version input argument.

The problem I'm running into is I don't have access to the original repo's files that is calling the GitHub Action. For example, say the repo is called python-sample-project and has folder structure:

python-sample-project
│   main.py
│   file1.py    
│
└───folder1
│   │   file2.py

I see main.py, file1.py, and folder1/file2.py in entrypoint.sh. However, in docker-action/entrypoint.sh I only see the linux folder structure and the entrypoint.sh file copied over in docker-action/Dockerfile.

In the Alpine example I'm using, the action entrypoint.sh script looks like this:

#!/bin/sh -l
ALPINE_VERSION=$1
cd /docker-action
docker build -t docker-action --build-arg alpine_version="$ALPINE_VERSION" . && docker run docker-action

In docker-action/ I have a Dockerfile and entrypoint.sh script that should run for the inner container with the dynamic version of Alpine (or Python)

The docker-action/Dockerfile is as follows:

# Container image that runs your code
ARG alpine_version
FROM alpine:${alpine_version}

# Copies your code file from your action repository to the filesystem path `/` of the container
COPY entrypoint.sh /entrypoint.sh

RUN ["chmod", "+x", "/entrypoint.sh"]

# Code file to execute when the docker container starts up (`entrypoint.sh`)
ENTRYPOINT ["/entrypoint.sh"]

In the docker-action/entrypoint I run ls but I do not see the repository files.

Is it possible to access the main.py, file1.py, and folder1/file2.py in entrypoint.sh in the docker-action/entrypoint.sh?

sytech
  • 29,298
  • 3
  • 45
  • 86
Alex F
  • 2,086
  • 4
  • 29
  • 67
  • I'm not sure I understand the problem statement. Can you provide the error output as well as the expected output? Also, the repository you linked does not contain anything related to Python and the action in there appears to be successful. Please be very specific about the problem. – sytech Oct 23 '21 at 18:37
  • The problem seems to be in your `Dockerfile`, however, you have not included its contents. – Danny Varod Oct 24 '21 at 10:37

1 Answers1

10

There's generally two ways to get files from your repository available to a docker container you build and run. You either (1) add the files to the image when you build it or (2) mount the files into the container when you run it. There are some other ways, like specifying volumes, but that's probably out of scope for this case.

The Dockerfile docker-action/Dockerfile does not copy any files except for the entrypoint.sh script. Your entrypoint.sh also does not provide any mount points when running the container. Hence, the outcome you observe is the expected outcome based on these facts.

In order to resolve this, you must either (1) add COPY/ADD statements to your Dockerfile to copy files into the image (and set appropriate build context) OR (2) mount the files into the container when it runs by adding -v /source-path:/container-path to the docker run command in your entrypoint.sh.

See references:

Though, this approach of building another container just to get a user-provided python version is a highly questionable practice for GitHub Actions and should probably be avoided. Consider leaning on the setup-python action instead.

The docker-in-docker problem

Nevertheless, if you continue this route and want to go about mounting the directory, you'll have to keep in mind that, when invoking docker from within a docker action on GitHub, the filesystem in the mount specification refers to the filesystem of the docker host, not the filesystem of the container.

It works on my machine?!

Counter to what you might experience running docker on a local system for example, this does not work in GitHub -- the working directory is not mounted:

docker run -v $(pwd):/opt/workspace \
           --workdir /opt/workspace \ 
           --entrypoint /bin/ls \
           my-container "-R"

This doesn't work either:

docker run -v $GITHUB_WORKSPACE:$GITHUB_WORKSPACE \
           --workdir $GITHUB_WORKSPACE \ 
           --entrypoint /bin/ls \
           my-container "-R"

This kind of thing would work perfectly fine if you tried it on a system running docker locally. What gives?

Dealing with the devil (daemon)

In Actions, the starting working directory where files are checked out into $GITHUB_WORKSPACE. In docker actions, that's /github/workspace. The workspace files populate into the workspace when your action runs by the Actions runner mounting the workspace from the host where the docker daemon is running.

You can see that in the command run when your action starts:

/usr/bin/docker run --name f884202608aa2bfab75b6b7e1f87b3cd153444_f687df --label f88420 --workdir /github/workspace --rm -e INPUT_ALPINE-VERSION -e HOME -e GITHUB_JOB -e GITHUB_REF -e GITHUB_SHA -e GITHUB_REPOSITORY -e GITHUB_REPOSITORY_OWNER -e GITHUB_RUN_ID -e GITHUB_RUN_NUMBER -e GITHUB_RETENTION_DAYS -e GITHUB_RUN_ATTEMPT -e GITHUB_ACTOR -e GITHUB_WORKFLOW -e GITHUB_HEAD_REF -e GITHUB_BASE_REF -e GITHUB_EVENT_NAME -e GITHUB_SERVER_URL -e GITHUB_API_URL -e GITHUB_GRAPHQL_URL -e GITHUB_WORKSPACE -e GITHUB_ACTION -e GITHUB_EVENT_PATH -e GITHUB_ACTION_REPOSITORY -e GITHUB_ACTION_REF -e GITHUB_PATH -e GITHUB_ENV -e RUNNER_OS -e RUNNER_NAME -e RUNNER_TOOL_CACHE -e RUNNER_TEMP -e RUNNER_WORKSPACE -e ACTIONS_RUNTIME_URL -e ACTIONS_RUNTIME_TOKEN -e ACTIONS_CACHE_URL -e GITHUB_ACTIONS=true -e CI=true -v "/var/run/docker.sock":"/var/run/docker.sock" -v "/home/runner/work/_temp/_github_home":"/github/home" -v "/home/runner/work/_temp/_github_workflow":"/github/workflow" -v "/home/runner/work/_temp/_runner_file_commands":"/github/file_commands" -v "/home/runner/work/my-repo/my-repo":"/github/workspace" f88420:2608aa2bfab75b6b7e1f87b3cd153444  "3.9.5"

The important bits are this:

-v "/home/runner/work/my-repo/my-repo":"/github/workspace" 
-v "/var/run/docker.sock":"/var/run/docker.sock"

/home/runner/work/my-repo/my-repo is the path on the host, where the repository files are. As mentioned, that first line is what gets it mounted into /github/workspace in your action container when it gets run.

The second line is mounting the docker socket from the host to the action container. This means any time you call docker within your action, you're actually talking to the docker daemon outside of your container. This is important because that means when you use the -v argument inside your action, the arguments need to reflect directories that exist outside of the container.

So, what you would actually need to do instead is this:

docker run -v /home/runner/work/my-repo/my-repo:/opt/workspace \
           --workdir /opt/workspace \ 
           --entrypoint /bin/ls \
           my-container "-R"

Becoming useful to others

And that works. If you only use it for the project itself. However, you have (among others) a remaining problem if you want this action to be consumable by other projects. How do you know where the workspace is on the host? This path will change for each repository, after all. GitHub does not guarantee these paths, either. They may be different on different platforms, or your action may be running on a self-hosted runner.

So how do you content with that problem? There is no inbuilt environment variable that points to this directory you need specifically, unfortunately. However, by relying on implementation detail, you might be able to get away with using the $RUNNER_WORKSPACE variable, which will point, in this case to /home/runner/work/your-project. This is not the same place as the origin of $GITHUB_WORKSPACE but it's close. You can use the GITHUB_REPOSITORY variable to build the path, though this isn't guaranteed to always be the case afaik:

PROJECT_NAME="$(basename ${GITHUB_REPOSITORY})"
WORKSPACE="${RUNNER_WORKSPACE}/${PROJECT_NAME}"

You also have some other things to fix like the working directory form which you build.

TL;DR

You need to mount files in the container when you run it. In GitHub, you're running docker-in-docker, so paths you need to use to mount files work different, so you need to find the correct paths to pass to docker when called from within your action container.

A minimally working solution for the example project you linked is this entrypoint.sh in the root of the repo looks like this:

#!/usr/bin/env sh
ALPINE_VERSION=$1

docker build -t docker-action \
             -f ./docker-action/Dockerfile \
             --build-arg alpine_version="$ALPINE_VERSION" \
             ./docker-action

PROJECT_NAME="$(basename ${GITHUB_REPOSITORY})"
WORKSPACE="${RUNNER_WORKSPACE}/${PROJECT_NAME}"

docker run --workdir=$GITHUB_WORKSPACE \
           -v $WORKSPACE:$GITHUB_WORKSPACE \
           docker-action "$@"

There are probably further concerns with your action, depending on what it does, like making available all the default and user-defined environment variables for the action to the 'inner' container, if that's important.

So, is this possible? Sure. Is it reasonable just to get a dynamic version of alpine/python? I don't think so. There's probably better ways of accomplishing what you want to do, like using setup-python, but that sounds like a different question.

sytech
  • 29,298
  • 3
  • 45
  • 86
  • thanks for the feedback. How have you handled different versions of python for a GitHub action? – Alex F Oct 25 '21 at 11:55
  • Using GitHub's first party [setup-python](https://github.com/actions/setup-python) action is probably the most reasonable way. You can incorporate this requirement in to your action several ways... A few ideas on how you might proceed: (1) expect users to use `setup-python` in a prior step (similar to how we expect them to use checkout) (2) use a composite action (3) publish multiple tags of your action using different versions (e.g. `myaction@v3-python3.9`). Depends partly on _why_ you need the python version specified. But leaning on `setup-python` is almost certainly the right way to go. – sytech Oct 25 '21 at 17:01
  • The other difficulty you'll have with the docker-in-docker approach is that mounting files is not straight forward because the mount points are specified on the daemon host filesystem, not the filesystem of the container your action in which your action is running. You can address that easy enough, but it's pretty fragile. Then there's additional work to do in order to make the 'inner' container have all the `GITHUB_` or user-defined variables made available to it, if the action needs them. Maybe consider a non-docker action to start with. I'll add a minimally working solution to this, FWIW. – sytech Oct 25 '21 at 17:09
  • I appreciate all the feedback. I haven't had a chance to work through your solution on my own machine yet, but I highly doubt anyone will give me a thorough of an answer as you have. Thank you for taking the time. – Alex F Oct 26 '21 at 13:26