0

I am trying to automate data download from github. Assume the file in github is in a zip folder. Is there a way to write a python code to download the entire zip folder?

I have used the request method but it does not work. The code can run without error but the file is clearly smaller than the original file and I cannot open it by double click with error message file is invalide

import requests

URL = 'https://github.com/XYZ/file.zip'
r = requests.get(url)
# open method to open a file on your system and write the contents    
with open("file.zip", "wb") as code:
    code.write(r.content)
Gavin
  • 1,411
  • 5
  • 18
  • 31
  • 1
    Look there: https://2.python-requests.org/en/master/ – olinox14 Mar 16 '20 at 14:49
  • Since it is on github, why don't you call git commands from Python? Like you can set that github as your remote, then you can `git pull`, `git checkout`, etc. In fact there are even libraries like [GitPython](https://gitpython.readthedocs.io/en/stable/tutorial.html) that allow you to run git commands using python directly – Cory Kramer Mar 16 '20 at 14:49
  • 2
    I'm with @Craicerjack on this. Pending proof that there is *absolutely no way*, I'm going to say: just try and report back when you get stuck. – Jongware Mar 16 '20 at 14:50
  • 2
    [How to ask a good question](https://stackoverflow.com/help/how-to-ask) – Craicerjack Mar 16 '20 at 14:50
  • 1
    Please be more specific. Are you downloading the GitHub project as a zip file? Does the GitHub project have tags/releases/downloadable links? Is there a zip file versioned into the git repo? Answers vary depending on what kind of scenario you are encountering. – Tin Nguyen Mar 16 '20 at 14:51
  • 1
    I have added the code that I have tested – Gavin Mar 16 '20 at 15:41
  • What is the issue, exactly? Have you tried anything, done any research? Stack Overflow is not a free code writing service, nor is it meant to provide personalized guides and tutorials. See: [ask], [help/on-topic], https://meta.stackoverflow.com/questions/261592/how-much-research-effort-is-expected-of-stack-overflow-users. – AMC Mar 16 '20 at 21:13
  • Does this answer your question? [How to create a zip archive of a directory in Python?](https://stackoverflow.com/questions/1855095/how-to-create-a-zip-archive-of-a-directory-in-python) – Craicerjack Mar 17 '20 at 09:56
  • Or this question - https://stackoverflow.com/questions/9419162/download-returned-zip-file-from-url. Your issue is specifically because you are trying to work with `zip files` – Craicerjack Mar 17 '20 at 09:58

1 Answers1

1

one way to workaround is to run a shell command in Python to clone the file then unzip on local using Python code

from subprocess import check_output

cmd = '!git clone https://github.com/XYI/file.git C:/Users/UserName/Desktop/test'
check_output(cmd, shell=True).decode()
Gavin
  • 1,411
  • 5
  • 18
  • 31