0

The files I create seem to be deleted after the script is completed. If I print the existing files in the python script they show, but aren't to be found in the repository. Is there any option I have to set for the files to stay?

This is the part of the gitlab-ci.yml responsible for running the python script:

retrieve-tags:
  stage: build
  script:
  - pip3 install python-gitlab
  - pip3 install markdown
  - python3 retrieve_tags.py 
  image: python
  artifacts:
    paths: 
      - .*

This is the part of the python script that should create files, but doesn't really:

path = CENSORED FOR REASONS 
file_name = "test.txt"
    with open(os.path.join(path,file_name), 'w') as fp:
        fp.write("TAG_CONTENTS")
        fp.close()
    
print(os.listdir(path))

I expect this to create files in the gitlab repository, in which the cicd pipeline is running and to view created files in the repository after the script ran.

Anthon
  • 69,918
  • 32
  • 186
  • 246
MaximusG
  • 11
  • Welcome to SO. Your code will throw an error if `with` is indeed indented relative to `file_name`, make sure you copy-paste real working minimal code (that often helps you find the solution yourself). Don't put tags like [tag:ascii], [tag:utf-8] or [tag:yaml] on a question when they apply but are not relevant to the problem. – Anthon Feb 02 '23 at 14:35

2 Answers2

1

Gitlab pipeline jobs are run on ephemeral virtual machines. Any changes you make to your repository are lost unless you push or upload those changes somewhere.

You can use normal Git commands to add and push your changes. For example:

git add .
git commit -m "My changes"
git push -o ci.skip "https://${GITLAB_USER_NAME}:${GITLAB_TOKEN}@${CI_REPOSITORY_URL#*@}"

GITLAB_TOKEN can be one of the token types on this page which has write_repository permissions.

Inspiration found on this answer.

D Malan
  • 10,272
  • 3
  • 25
  • 50
0

Your project is cloned to the runner. So the changes you make are only visible inside the runner. To Push back the changes to the repository, you have to commit and push them, just like on your local machine.

 - git config user.name "YourName"
 - git config user.email "YourName@email.com"
 - git add . # be careful with this and ensure your .gitignore is correct.
 - git commit -m "Update from pipeline"
 - git push

This workflow has the risk of merge conflicts when your Pipeline runs for a long time.

An alternative to adding these files to the repository is to create Job Artifacts as you already did. But they are not added to the repo, but can be viewed on the right side of a Job Execution. And I wouldn't add all your files, as this can quickly fill up your disk space. You should just add the generated files.

Uli Sotschok
  • 1,206
  • 1
  • 9
  • 19