1

While trying to establish a list of incoming GitHub commits I've stumbled accross the GitHub rate api limits, of 60 calls per hour. As explained in this answer, one can get the lists of branches with an API call using:

https://api.github.com/repos/{username}/{repo-name}/branches

However, that triggers the rate limit for the average GitHub organisation/user. So I thought I'd try a different approach, using RSS/atom format. However, as that same answer explains, the atom format/rss feed seems to depend on the user having a list of all branches in a repository. This question asks for an overview of all commits in a repository, yet instead it is given an answer for all commits in the default branch of the repository. And this question receives a working answer that triggers the rate limit, as it relies on at least 1 API call per repository.

Hence, I would like to ask: How could one get a list of all branches of a GitHub user, using at most 1 GitHub API call?

Note, using atom views would be perfectly fine, however, I have not found an atom view like: https://github.com/:owner/:repo/commits.atom or https://github.com/:owner/:repo/branches.atom that displays all branches in a repository. I would strongly prefer a solution that does not rely on a third party like: https://rsshub.app/github/repos/yanglr as I imagine, they too will at some point start rate-limiting.

My current approach is to scrape the source code of https://github.com/:user/:repo/branches using bash. However, I imagine there might exist a more efficient solution to this.

MWE

Thanks to the comments, I was ble to find a bash MWE to perform a GraphQL query using terminal. It is given in this answer, where bearer is not a variable, it is the means of identification and the ...... should be your personal GitHub Access token. I am currently looking into how to get the repositories beyond the 1st hundred. Then I'll look at how to get the branches of those repositories.

Attempt I

The following query yields a json with the repositories and first 4 branches in each repository of a user!

name:examplequery.gql.

query {
  repositoryOwner(login: "somegithubuser") {
    repositories(first: 40) {
      edges {
        node {
          nameWithOwner
          refs(
            refPrefix: "refs/heads/"
            orderBy: { direction: DESC, field: TAG_COMMIT_DATE }
            first: 4
          ) {
            edges {
              node {
                ... on Ref {
                  name
                }
              }
            }
          }
        }
      }
    }
  }
}

Next, a bash script is made that runs the query:

#!/usr/bin/env bash
# Runs graphql query on GitHub. Execute with:
# ./run_graphql_query.sh examplequery1.gql

GITHUB_PERSONAL_ACCESS_TOKEN_GLOBAL="your_github_personal_access_token"

if [ $# -ne 1 ]; then
    echo "usage of this script is incorrect."
    exit 1
fi

if [ ! -f $1 ];then
    echo "usage of this script is incorrect."
    exit 1
fi

# Form query JSON
QUERY=$(jq -n \
           --arg q "$(cat $1 | tr -d '\n')" \
           '{ query: $q }')


curl -s -X POST \
  -H "Content-Type: application/json" \
  -H "Authorization: bearer $GITHUB_PERSONAL_ACCESS_TOKEN_GLOBAL" \
  --data "$QUERY" \
  https://api.github.com/graphql

It can be ran with:

./run_graphql_query.sh examplequery1.gql

There are two more issues to resolve before I can answer the question. How I can iterate over all repositories instead of only the first 100. How I can parse the json into a list of branches per repository.

a.t.
  • 2,002
  • 3
  • 26
  • 66
  • 1
    Did you try using the GraphQL API? – jonrsharpe Feb 28 '22 at 15:59
  • 1
    My guess would be to use the graph API so get all of that in a single go. But I'm not versed well enough in the query language to tell you how to do it. https://docs.github.com/en/graphql/overview/explorer and https://docs.github.com/en/graphql/overview/about-the-graphql-api – jessehouwing Feb 28 '22 at 15:59
  • @jonrsharpe and @jessehouwing, thank you for the suggestion, I have not yet looked at that. It seems to be what I am looking for, since the amount of data I want is small yet specific, compared to what is typically returned in a single API call like: `https://github.com/:owner/:repo/commits.atom`. I will look into it! – a.t. Feb 28 '22 at 16:33

0 Answers0