How can I download a single raw file from a private github repo using the command line?

Question

On the CI server, I want to fetch a config file that we maintain on Github so it can be shared between many jobs. I'm trying to get this file via curl, but these approaches both fail (I get a 404):

# As advised by the oAuth docs
curl -H 'Authorization: token the_token' -L -o setup.sh https://raw.github.com/org/repo/file

# The url of the raw file after clicking to view it
curl -L https://raw.github.com/org/repo/file?login=username&token=the_token 

Answer 1

The previous answers don't work (or don't work anymore).

You can use the V3 API to get a raw file like this (you'll need an OAuth token):

curl -H 'Authorization: token INSERTACCESSTOKENHERE' \
  -H 'Accept: application/vnd.github.v3.raw' \
  -O \
  -L https://api.github.com/repos/owner/repo/contents/path

All of this has to go on one line. The -O option saves the file in the current directory. You can use -o filename to specify a different filename.

To get the OAuth token follow the instructions here:

• https://help.github.com/articles/creating-an-access-token-for-command-line-use

I've written this up as a gist as well:

• https://gist.github.com/madrobby/9476733

EDIT: API references for the solution are as follows:

• https://developer.github.com/v3/#authentication
• https://developer.github.com/v3/media/#request-specific-version
• https://developer.github.com/v3/repos/contents/#get-contents

Answer 2

Alternatively, you can use a github "personal access token" (https://github.com/settings/tokens):

TOKEN=...
curl -s https://$TOKEN@raw.githubusercontent.com/<user or organization>/<repo name>/<branch>/<path to file>/<file_name>

Example:

$ curl -s https://1bacnotmyrealtoken123beefbea@raw.githubusercontent.com/concourse/concourse/master/README.md
....

Answer 3

I know this is an old question, but none of the solutions proposed above worked for me. Perhaps the API has changed since then.

This worked:

curl -H 'Authorization: token [insert your token here]' -o output.txt https://raw.githubusercontent.com/[organization]/[repo]/[branch]/[path to file]

Answer 4

A simpler solution is to use Official GitHub CLI gh.

1. First you must be logged in :

gh auth login

For me, this command is not required since I'm already logged in.

2. Then we need API URL targeting the file to download. And call gh to download it:

API_URL=https://api.github.com/repos/owner/repo/contents/path/file.ext
gh api $API_URL -H "Accept: application/vnd.github.raw" > file.ext

An real example is maybe better. Here it is to download install_linux.md from gh cli:

API_URL=https://api.github.com/repos/cli/cli/contents/docs/install_linux.md
gh api $API_URL -H "Accept: application/vnd.github.raw" > install_linux.md

In API_URL:

• User owner is cli
• Repository name repo is cli too
• Path to file (path/file.ext) is docs/install_linux.md

Answer 5

I was struggling with this for a few minutes until I realized all that is needed is to wrap the url in quotes to escape the ampersand.

curl "https://raw.github.com/org/repo/file?login=username&token=the_token"

That worked for me in my private repo.

Answer 6

Or, if you don't have a token:

curl --user [your_user] 'https://raw.github.com/path/to/file.config' > file.config

It will ask you for your password.

UPDATE: To those having 404s, you may have to create an SSH key for your machine. Refer to the SSH and GPG Keys in your Settings.

This works if you tap the Raw button on the respective file (It's a button on the top right, available after you tap the file you want to DL). Copy the URL for that raw resource. If you attempt to DL a resource that is not hosted through raw.github.com (it may have changed to raw.githubusercontent.com), it won't work.

I successfully DL'ed a resource from a personal repository, as well as non-personal one, using macOS as of Jul 14 2023 using this method.

Answer 7

I've got a token from an app installation.

Previously, you could use the query ?access_token=MY_TOKEN, but that has been deprecated and eventually removed in September 2021.

In their docs on Authenticating with GitHub Apps they're saying you can clone a repo with the access token and the username x-access-token in the URL.

This also seems to work to download a raw file (the ghs_... is the token):

$> curl "https://x-access-token:ghs_4qgGKx4skAcaF3bAb3scrTkN4@raw.githubusercontent.com/Octocat/codertocat/main/README.md"

Answer 8

1. in browser open your github repo: click on file
2. open Developer Tools in browser: select Network tab
3. in browser github: click on Download button
4. close pop-up
5. in browser dev tools: right click on list that has file_name?token=ABAHQCAT6KG...

6. select copy -> copy link address

   url is in format:

   https://raw.githubusercontent.com/<USERNAME>/<PATH>/<FILENAME>?token=ABAHQCAT6KGHYHMG2SLCDT243PH4I

7. in terminal:

   wget -O myFilename https://raw.githubusercontent.com/<USERNAME>/<PATH>/<FILENAME>?token=ABAHQCAT6KGHYHMG2SLCDT243PH4I

Link is valid only for limited time or you can create your token: GitHub article

Answer 9

I ran into an authentication error when the url was redirected to Amazon S3:

Only one auth mechanism allowed; only the X-Amz-Algorithm query parameter...

Changing from the Authorization: token X header to the ?access_token=<token> query param worked for me.

Answer 10

Surprisingly none of the answers didn't worked for me until I found a workaround.

You can use personal access token https://github.com/settings/tokens as answered by @thomasfuchs

Note : while creating a token, you must check the admin permissions. See the related issue

https://github.com/octokit/octokit.net/issues/1812

Answer 11

I was able to get it to work for github enterprise, thanks for the suggestions above. Had to take all your suggestions and try and finally i was able to make it work. These are the steps i followed to get it to work.

1. Create personal token, followed these steps:

https://docs.github.com/en/github/authenticating-to-github/creating-a-personal-access-token

2. Make sure you have minimum following permissions for the token:

   • repo (Select all under repo)
   • admin:org -> read:org (select "read:org" under "admin:org")

3. Use the following curl command to get the content:

curl -H "Authorization: token [yourPersonalToken]" -H "Accept: application/vnd.github.v3.raw" -o [filePath]-content.json -L https://github.[company].com/api/v3/repos/[ORG]/[REPO_NAME]/contents/[PATH_TO_FILE]/content.json?ref=[BRANCH_NAME]

Where->

 [yourPersonalToken] is the token you created.
 [filePath] is a path where you want to save the downloaded copy.
 [company] is the name of company which hosted the github enterprise.
 [ORG] is the github organization is which repo is created.
 [REPO_NAME] is the name of the repository.
 [PATH_TO_FILE] is the path where file is located.
 [BRANCH_NAME] is the name of the branch you want to use, e.g. master, develop etc.

Example:

curl -H "Authorization: token 5a86ecda9ff927baaa66fad2af5bee8" -H "Accept: application/vnd.github.v3.raw" -o C:\Downloads\manifest.json -L https://github.example.com/api/v3/repos/cms/cms_one/contents/app/data/manifest.json?ref=master

Answer 12

We had to download files from private GitHub repos fairly often and hacky shell scripts weren't quite cutting it, so we created fetch, which is an open source, cross-platform tool that makes it easy to download source files and release assets from a git tag, commit, or branch of public and private GitHub repos.

For example, to download the file baz from version 0.1.3 of a private GitHub repo to /tmp, you would do the following:

GITHUB_OAUTH_TOKEN="your token"
fetch --repo="https://github.com/foo/bar" --tag="0.1.3" --source-path="/baz" /tmp

Answer 13

Just an addition to the accepted answer, If you are using Github Enterprise url is slightly different:

curl -H 'Authorization: token [your token]' \
-H 'Accept: application/vnd.github.v3.raw' \
-L https://[your domain]/api/v3/repos/[owner]/[repo-name]/contents/[path of file]

Answer 14

curl -H 'Authorization: token YOUR_TOKEN' \
  -H 'Accept: application/vnd.github.v4.raw' \
  -O \
  -L https://api.github.com/repos/INSERT_OWNER_HERE/INSERT_REPO_HERE/contents/PATH/TO/FILE

So if the url of the raw file (when logged in) is

https://raw.githubusercontent.com/mr_coder/my_repo_name/master/my_script


Then 
  -L https://api.github.com/repos/INSERT_OWNER_HERE/INSERT_REPO_HERE/contents/PATH/TO/FILE
becomes
  -L https://api.github.com/repos/mr_coder/my_repo_name/contents/my_script

Note: We have API v4

Answer 15

For GitHub Enterprise and API v3, my bash solution looked like this (includes TOKEN cleanup / privacy):

TOKEN=yourTokenHere; history -d $((HISTCMD-1)) > /dev/null

curl -H "Authorization: token $TOKEN" \
  -H 'Accept: application/vnd.github.v3.raw' \
  -o file.ext \
  -L http://github.company.com/api/v3/repos/[org]/[repo]/contents/path/file.ext?ref=[branch]

unset TOKEN

Answer 16

I think it is a little bit dangerous and not good way to issue a personal access token which can access all repositories even just for download a single file from my private repository.

How to -

I would love to recommend using url with token for single file. Don't worry. The token string will generated by github automatically. You can get this url on your source code page.

1. Go to source code page what you want to download via curl or wget, etc
2. Find 'raw' button and click it.
3. New page opened and just copy the url. This url look like below:
   (https://raw.githubusercontent.com/USERNAME/REPONAME/BRANCHNAME/FILENAME?token=TOKENSTRING).
4. You can download a file using this url

Answer 17

A lot of the answers here either suggest to use curl and supply your own Authorization: Bearer **** header, or use the GitHub CLI (gh) to get a pre-authenticated download URL and then using curl with that URL.

However, gh supports custom headers so you can accomplish this using gh with a single request:

gh api "/repos/:owner/:repo/contents/README.md" -H "Accept: application/vnd.github.raw" > README.md

Answer 18

I tried a simple trick to open a GitHub private .iypnb file in Pycharm as well as Colab and it worked well for me.

1. get raw text for your .ipynb file by pressing Raw button this will open some text like this.

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": []
}]
}

2. open notepad/text editor on os(eg. windows) copy all the text to a new notepad file .

3. save notepad as name.ipynb instead of name.txt and make save as file type All Files(.) instead of Text Documents (*.txt)

4. finally open file in your IDE or colab.

Answer 19

Below should work fine. A "raw" before your branch name (master in this case).

curl -L -O https://github.com/your/repo/raw/master/fetch_file.sh

Answer 20

You can do this with a raw link.

curl -O https://raw.githubusercontent.com/owner/repo/branchname/path/to/file