403 error from Github.js getSha for files above ~1MB in size

Question

I'm getting a 403 error when I try to use Github.js to getSha (and subsequently getBlob) from files larger than ~1MB. Is there a limit to the file size? The code is below:

var gh = new GitHub({
    username: username,
    password: password
});

// get the repo from github
var repo = gh.getRepo('some-username','name-of-repo');

// get promise
repo.getSha('some-branch', 'some-file.json').then(function(result){

  // pass the sha onto getBlob
  return repo.getBlob(result.data.sha);

}).then(function(result){

  do_something_with_blob(result.data);

});

The GitHub API says that it supports blobs up to 100MB in size and I could not find anything about filesize limits in the Github.js docs. Also, the files are from a private Github repo.

Bertrand Martel · Accepted Answer · 2017-12-18T18:43:06.800

It throws a 403 Forbidden error because it uses Github GET contents API which gives results for file not exceding 1Mo. For instance the following will throw 403 :

https://api.github.com/repos/bertrandmartel/w230st-osx/contents/CLOVER/tools/Shell64.efi?ref=master

Using this method using GET tree API, you can get the file sha without downloading the whole file and then use repo.getBlob (which uses Get blob API for file not exceding 100Mo).

The following example will get the tree for the parent folder of the specified file (for a file exceding 1Mo) with the GET trees api, filter the specific file by name and then request blob data :

const accessToken = 'YOUR_ACCESS_TOKEN';

const gh = new GitHub({
  token: accessToken
});

const username = 'bertrandmartel';
const repoName = 'w230st-osx';
const branchName = 'master';
const filePath = 'CLOVER/tools/Shell64.efi'

var fileName = filePath.split(/(\\|\/)/g).pop();
var fileParent = filePath.substr(0, filePath.lastIndexOf("/"));

var repo = gh.getRepo(username, repoName);

fetch('https://api.github.com/repos/' +
  username + '/' +
  repoName + '/git/trees/' +
  encodeURI(branchName + ':' + fileParent), {
    headers: {
      "Authorization": "token " + accessToken
    }
  }).then(function(response) {
  return response.json();
}).then(function(content) {
  var file = content.tree.filter(entry => entry.path === fileName);

  if (file.length > 0) {
    console.log("get blob for sha " + file[0].sha);
    //now get the blob
    repo.getBlob(file[0].sha).then(function(response) {
      console.log("response size : " + response.data.length);
    });
  } else {
    console.log("file " + fileName + " not found");
  }
});

Thanks it works for public repos but I can not seem to get the tree (using fetch) for private repos where I'm getting a 404 error. I'm guessing that is because fetch is not passing any authentication token along. Is there a way to pass the token using Fetch or get the tree using a method in the gh object that already has the token? — Nick Fernandez, Dec 18 '17 at 16:36
@NickFernandez I've updated the answer with addition of Authorization header in fetch options — Bertrand Martel, Dec 18 '17 at 18:43
Note: the Get Content API now (May 2022) [supports raw files up to 100MB](https://stackoverflow.com/a/72108664/6309) — VonC, May 04 '22 at 06:29

403 error from Github.js getSha for files above ~1MB in size

1 Answers1