How to push existing file onto gitlab repository using python

Question

is there a way to push existing file onto gitlab project repository in python like the git commit and git push commands, instead of creating a new file?

I'm currently using python-gitlab package and I think it only supports files.create which creates a new file using supplied string contents. This would result in slightly different file content in my case.

I'm looking for a way to push the file in python onto the repo untouched, can anyone help?

VonC · Accepted Answer · 2019-07-08T12:40:38.263

4

The Dec. 2013 0.5 version of gitlab/python-gitlab does mention:

Project: add methods for create/update/delete files (commit ba39e88)

So there should be a way to update an existing file, instead of creating a new one.

def update_file(self, path, branch, content, message):
    url = "/projects/%s/repository/files" % self.id
    url += "?file_path=%s&branch_name=%s&content=%s&commit_message=%s" % \
        (path, branch, content, message)
    r = self.gitlab.rawPut(url)
    if r.status_code != 200:
        raise GitlabUpdateError

In May 2016, for the 0.13 version, the file_* methods were deprecated in favor of the files manager.

warnings.warn("`update_file` is deprecated, "
                      "use `files.update()` instead",
                      DeprecationWarning)

That was documented in 0.15, Aug. 2016.
See docs/gl_objects/projects.rst

Update a file.
The entire content must be uploaded, as plain text or as base64 encoded text:

f.content = 'new content'
f.save(branch='master', commit_message='Update testfile')

# or for binary data
# Note: decode() is required with python 3 for data serialization. You can omit
# it with python 2
f.content = base64.b64encode(open('image.png').read()).decode()
f.save(branch='master', commit_message='Update testfile', encoding='base64')

What I am looking for is to push an "existing local file" to an empty GitLab project repository

To create a new file:

f = project.files.create({'file_path': 'testfile.txt',
                          'branch': 'master',
                          'content': file_content,
                          'author_email': 'test@example.com',
                          'author_name': 'yourname',
                          'encoding': 'text',
                          'commit_message': 'Create testfile'})

You can check the differences between a file created on GitLab (and cloned) with your own local file with

git diff --no-index --color --ws-error-highlight=new,old

I mentioned it in 2015 for better whitespace detection.

The OP Linightz confirms in the comments:

The file after created by python-gitlab misses a space (0x0D) on every line ending.
So I guess you're right.
However I tried to add core.autocrlf setting or add newline='' in my file open statement or read in binary and decode using different encoding, none of above worked.

I decided to just use shell command in python to push the file to avoid all these troubles, t

edited Jul 08 '19 at 12:40

answered Jul 06 '19 at 09:58

VonC

1,262,500
529
4,410
5,250

thanks for the input but this method still takes a string as content to update an "existing file on repo". What i'm looking for is to push an "existing local file" to an empty gitlab project repository. – Linightz Jul 06 '19 at 10:11
@Linightz That would be the create method you mention. Why this "would result in slightly different file content"? – VonC Jul 06 '19 at 11:08
I have no idea, but the file size after uploaded to gitlab using create method will be slightly different even tho the content seems to be the same. It's actually a special config file used for a specific program, the program won't take the file after uploaded but only before. – Linightz Jul 06 '19 at 18:57
@Linightz Maybe the eol (end of lines) have changes. Do you have a `core.autocrlf` set to true locally? – VonC Jul 06 '19 at 18:58
yes that might be it, but isn't `core.autocrlf` a setting for `git` command? or does it affect `python-gitlab` create too? I tried to use `git` to push the file and the file stayed intact as the file size didn't change. – Linightz Jul 07 '19 at 10:38
@Linightz Yes, it is a local setting, and maybe GitLab has one on its side which would explain the difference. Any chance you can somehow download and compare the create file from GitLab with the one you have locally? – VonC Jul 07 '19 at 10:42
yes I can download and compare the two but like i said, i couldn't see any difference other than the file size. Do you have a way to check the line ending in windows? much appreciated. – Linightz Jul 07 '19 at 14:42
@Linightz Yes: https://stackoverflow.com/q/3920650/6309, that is `git diff -R` or `git diff --color --ws-error-highlight=new,old`. If not, try an hexadecimal comparison (http://www.flexhex.com/docs/howtos/file-comparison.phtml) – VonC Jul 07 '19 at 20:35
I have no idea how to compare local file to the file created by `python-gitlab` using `git diff` but I did use the hex comparison to find the differences - the file after created by `python-gitlab` misses a space (0x0D) on every line ending. So i guess you're right. However i tried to add `core.autocrlf` setting or add `newline=''` in my file open statement or read in binary and decode using different encoding, none of above worked. Do you have any other suggestion? thank you. – Linightz Jul 08 '19 at 02:52
@Linightz Note: two compare any two files outside a Git repo: `git diff --no-index <--other options> -- file1 file2` – VonC Jul 08 '19 at 04:21
@Linightz Try again with `git config --global core.autocrlf false`, and a file created with a string without CRLF. Create it on GitLab, then clone and compare. – VonC Jul 08 '19 at 04:25
I decided to just use shell command in python to push the file to avoid all these troubles, thanks anyway! – Linightz Jul 08 '19 at 12:30
@Linightz OK. I have edited the answer to include the whitespace diff, for other readers to benefit from. – VonC Jul 08 '19 at 12:41

How to push existing file onto gitlab repository using python

1 Answers1