How do you push to a gitlab repo using a gitlab-ci job?

Question

I am new to GitLab CI/CD jobs, but I'm trying to set up a Python script that when pushed to GitLab, triggers the CI/CD job to run it, and call an internal function that pushes to GitLab again provided that certain criteria are met. So, for example, suppose I have the following:

def hasFileInDirectory():
    # checks if the current directory has at least 1 other file in it
    if (1 or more files exist):
        print 'Great! You have enough files!';
    else:
        print 'Oh no! You need more files! Let me create one!';
        createFile('missingFile'+str(random.randint(0,1000000)+'.txt');
        os.system('git add -A');
        os.system('git commit -m "Automatically added new file..."');
        os.system('git push origin HEAD:master --force');

This function seems to run perfectly fine if I run it myself from the command line, however, it seems to not be able to run in the GitLab CI/CD job. The output I am getting is:

remote: You are not allowed to upload code.
fatal: unable to access 'https://gitlab-ci-token:xxxxx@gitlab.com/path_to/my_repository.git/': The requested URL returned error: 403

This error occurs when I call git push so I was wondering what I could do to fix this. I would really appreciate any help!

Possible duplicate of [Cannot push from gitlab-ci.yml](https://stackoverflow.com/questions/46472250/cannot-push-from-gitlab-ci-yml) — Clive Makamara, Jul 23 '18 at 08:08

score 21 · Answer 1 · edited Nov 09 '22 at 22:11

21

A GitLab CI runner cannot yet push to a repo: there is a proposal in progress here.

In the meantime, you can use an SSH URL, with:

An SSH private key is defined as a secret variable through the Settings > CI/CD Pipelines web interface in GitLab, and
the public part of the SSH key is stored as a deployment key Settings > Repository > Deploy Keys section of the same web UI.

Or, as mentioned here, you can use a “personal access token” in Settings of your profile.

I created a token with scope api and configure in my pipeline.
Open the project in gitlab console, go to Settings > CI/CD > Secret variables, create a variable with value the key (generated in profile).
I replace “${CI_JOB_TOKEN}” to my variable “${VAR01}”.

With a gitlab-ci.yml

script:

   - url_host=`git remote get-url origin | sed -e "s/https:\/\/gitlab-ci-token:.*@//g"`
   - git remote set-url origin "https://gitlab-ci-token:${CI_TAG_UPLOAD_TOKEN}@${url_host}"

CI_TAG_UPLOAD_TOKEN is the Secret variable

edited Nov 09 '22 at 22:11

rubicks

4,912
1
30
41

answered Jul 22 '18 at 14:23

VonC

1,262,500
529
4,410
5,250

1

Both solutions have drawbacks. "ssh url" can't be used in some environments, where only http access is possible. And "personal access token" is personal - when CI job wants to change something in repository - it should be done not from some predefined user account, but from the one, who started a job. – Ezh Mar 12 '19 at 09:18
@Ezh So using a deploy token (https://docs.gitlab.com/ce/user/project/deploy_tokens/) instead of a user token? That should work. and match your criteria. – VonC Mar 12 '19 at 11:58
1

@VonC deploy tokens can not be used to write to the repository. Use can use them to read the repository or read/write the registry. – Martin Žid Jul 17 '20 at 07:13
@MartinŽid OK, no deploy token then. – VonC Jul 17 '20 at 07:29

jtheoof · Answer 2 · 2019-03-12T16:35:09.907

The command we ended up using was:

git tag my-new-tag
git push --repo=git@YOUR_REPO_URL:YOUR_GROUP/YOUR_PROJECT.git --tags

By default we were getting:

remote: You are not allowed to upload code.
fatal: unable to access 'https://gitlab-ci-token:xxxxxxxxxxxxxxxxxxxx@YOUR_REPO_URL:YOUR_FORK/YOUR_PROJECT.git/': The requested URL returned error: 403`

The reason is because the CI runner executes git commands using the HTTPS protocol with a token that does not support push as stated by @VonC.

We have configured our runner to share a volume to the /root/.ssh directory. So using the git protocol and a proper ssh configuration we are able to push git commands using the Gitlab CI runner.

Edit

The gitlab runner is exectuded as follows (I removed useless parameters for the purpose of clarity)

docker exec -it gitlab-runner.service gitlab-runner register \
  --non-interactive \
  --name `hostname` \
  --url "some-gitlab-url" \
  ...
  --executor "docker" \
  --limit "1" \
  --docker-image "debian" \
  --docker-volumes "/var/run/docker.sock:/var/run/docker.sock" \
  --docker-volumes "/fs1/runner/builds:/builds" \
  --docker-volumes "/fs1/runner/cache:/cache" \
  --docker-volumes "/fs1/runner/profile:/root" \

So the /root directory is shared across all our runners. Therefore once configured properly with /root/.ssh/ proper keys and those keys have the right to push to gitlab, it will work as described above.

Could you please detail the part "onfigured our runner to share a volume to the /root/.ssh directory". Was that in the ".gitlab-ci.yml'? — Anas Tiour, Feb 24 '19 at 15:22
I've edited my response to give a little bit more details. Hopes this helps. — jtheoof, Mar 12 '19 at 16:35
@jtheoof Are you saying that /root is already mounted across runners or that you had to change the way `docker exec` is launched (If so, then how)? The problem is that when using `gitlab-machines`, the runners are actually spawned as needed when the pipeline is started, so how would I configure the `/root/.ssh` then? — Frak, Dec 22 '19 at 05:35
btw I do get `403` error code despite my token has all permissions possible. there's absolutely no workaround to push via https? — im_infamous, Nov 16 '20 at 12:08

How do you push to a gitlab repo using a gitlab-ci job?

2 Answers2

Edit

Linked

Related