Querying all commits in a single repository with the GitHub GraphQL API v4

Question

I'm trying to query all of the commits to a specified repository on GitHub via GitHub's GraphQL API v4.

I only want to pull the dates they were committed at, in order to estimate the total time that was contributed to that repository (something along the lines of git-hours)

Here's my initial query: (note: you can try to run it in the Explorer)

{
  repository(owner: "facebook", name: "react") {
    object(expression: "master") {
      ... on Commit {
        history {
          nodes {
            committedDate
          }
        }
      }
    }
  }
}

Unfortunately it returns only the latest 100 commits, because of the API's resource limitations:

Node Limit

To pass schema validation, all GraphQL API v4 calls must meet these standards:

• Clients must supply a first or last argument on any connection.
• Values of first and last must be within 1-100.
• Individual calls cannot request more than 500,000 total nodes.

So since I'm not supplying a first or last argument, the API assumes I'm querying for history(first: 100). And I can't query more than 100 nodes in a single connection.

However, the total node limit being much higher (500,000), I should be able to query commits in groups of 100 until I have all of them.

I was able to query the latest 200 commits using this query:

{
  repository(owner: "facebook", name: "react") {
    object(expression: "master") {
      ... on Commit {
        total: history {
          totalCount
        }
        first100: history(first: 100) {
          edges {
            cursor
            node {
              committedDate
            }
          }
        }
        second100: history(after: "700f17be6752a13a8ead86458e343d2d637ee3ee 99") {
          edges {
            cursor
            node {
              committedDate
            }
          }
        }
      }
    }
  }
}

However I had to manually enter the cursor String that I'm passing in the second connection: second100: history(after: "cursor-string") {}.

How can I recursively run this connection until I have a query for all the committedDates of commits in a repository?

Answer 1

Although there could be a way of recursively querying all commits on a repo, I couldn't find a working solution.

Here's my solution

My need was:

I only want to pull the dates they were committed at, in order to estimate the total time that was contributed to that repository (something along the lines of git-hours)

Since I couldn't query the full commit history, I had to make the assumption that the contributed time over the latest 100 commits is the same as over any 100 commits.

Querying data from the GitHub GraphQL API

• the commit history's totalCount
• the committedDate of the latest 100 commits

{
  repository(owner: "facebook", name: "react") {
    object(expression: "master") {
      ... on Commit {
        history {
          totalCount
          nodes {
            committedDate
          }
        }
      }
    }
  }
}

Run today, the query returns:

{
  "data": {
    "repository": {
      "object": {
        "history": {
          "totalCount": 10807,
          "nodes": [
            {
              "committedDate": "2019-04-04T01:15:33Z"
            },
            {
              "committedDate": "2019-04-03T22:07:09Z"
            },
            {
              "committedDate": "2019-04-03T20:21:27Z"
            },
            // 97 other committed dates
          ]
        }
      }
    }
  }
}

Estimating total contributed time

I estimated the time contributed in the latest 100 commits using an algorith similar to the one explained on git-hours's README.

Then I scaled it to the totalCount:

const timeContributedTotal = timeContributedLatest100 * totalCount / 100;

I estimated that 13152 hours were put on Twitter's Bootstrap as of today, where git-hours estimated 9959 hours 7 months ago. Doesn't sound too bad.

As for React, I get a total 15097 hours, or 629 days.

The estimate is very rough, but it's as close as I could get to what I needed. Feel free to comment or answer if you see any possible improvement.