25

I'm trying to search for some piece of code using the GitHub API V3 given only the keyword, not limiting by user, organization, or repository.

For example, if I want to search for all pieces of code that contain the keyword "addClass", the results would be https://github.com/search?q=addClass&type=Code&ref=searchresults without using GitHub API.

But how can I do the same thing through GitHub API? I tried https://api.github.com/search/code?q=addClass It says "Must include at least one user, organization, or repository". How can I fix this?

xpt
  • 20,363
  • 37
  • 127
  • 216
user3724417
  • 373
  • 1
  • 3
  • 6

3 Answers3

24

You can do a code search without specifying a user/org/repo if you authenticate.

First, generate a personal access token for use for this purpose, from your Profile on GitHub's website: Settings -> Developer Settings -> Personal Access Token -> Generate New Token (you can leave all access options unticked, since you're just using to make web requests)

Now, your original GET request will work and return results, if you append the token to it:

https://api.github.com/search/code?q=addClass&access_token=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

UPDATE: OCT 2021 As pointed out by a comment below, passing the token in via a query parameter (like above) is deprecated. You must now add it as an Authorization header.

e.g.

curl --location --request GET 'https://api.github.com/search/code?q=addClass +in:file +language:csharp' \
--header 'Authorization: Token xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'

or in Python:

import requests

url = "https://api.github.com/search/code?q=addClass +in:file +language:csharp"

headers = {
  'Authorization': 'Token xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'
}

response = requests.request("GET", url, headers=headers)

print(response.text)
Mark Z.
  • 2,127
  • 16
  • 33
  • 1
    Oh, I only saw your solution ***after*** I've commented above. Now I think yours should be the accepted answer. upvoting. cheers. – xpt Aug 28 '20 at 15:11
  • Yes. Same as above comment. I just saw the selected answer and closed the tab. Saw this answer after couple weeks. This should be the accepted answer – Arshad May 30 '21 at 07:35
  • 1
    `{ "message": "Must specify access token via Authorization header. https://developer.github.com/changes/2020-02-10-deprecating-auth-through-query-param", "documentation_url": "https://docs.github.com/v3/#oauth2-token-sent-in-a-header" }` – Luk Aron Oct 20 '21 at 06:36
  • @LukAron indeed this is true - I have updated the answer, thanks. – Mark Z. Oct 21 '21 at 18:03
15

2020: As detailed in Mark Z.'s answer, using an authentication (Authorization': 'Token xxxx') allows for a code search.

get /search/code

You can use:

  • either a dedicated command-line tool like feinoujc/gh-search-cli

    ghs code --extension js "import _ from 'lodash'"

  • or the official GitHub CLI gh, (after a gh auth login) as show in issue 5117:

    gh api --method=GET "search/code?q=filename:test+extension:yaml+org:new-org"

    Or even:

    gh api --method=GET search/code -f q='filename:test extension:yaml org:new-org' \
       --jq '.items[] | [.repository.full_name,.path,.sha] | @tsv'

    That would get a line-based, tab-separated list of fields in this order: repo name, file path, git sha. (see gh help formatting)

2014 (original answer): That seems related to the new restriction "New Validation Rule for Beta Code Search API" (Oct. 2013)

In order to support the expected volume of requests, we’re applying a new validation rule to the Code Search API. Starting today, you will need to scope your code queries to a specific set of users, organizations, or repositories.

So, the example of the API search code mentions now:

Suppose you want to find the definition of the addClass function inside jQuery. Your query would look something like this:

https://api.github.com/search/code?q=addClass+in:file+language:js+repo:jquery/jquery

VonC
  • 1,262,500
  • 529
  • 4,410
  • 5,250
  • so the accepted answer already prove you are wrong, as long as authenticated, you could do the search without scoping to anything – Luk Aron Oct 20 '21 at 08:04
12

While Gihub does not currently support code search without repo, user, or organization (see VonC's answer), codesearch does index some sources from Github via the codesearch API, albeit with an API less fully featured out than Github's.

For example, to search for wget invocations indexed from Github, call

curl "https://searchcode.com/api/codesearch_I/?q=wget&src=2"

The optional src parameter picks the code source (e.g., Github, BitBucket) that should be searched, and you can find its integer value for a source by altering the parameters of faceted search in the codesearch UI. The current value of src for Github is 2.

You can verify that the returned results from the above example come from github.com by viewing the the repo property of results items.

Andrew Head
  • 163
  • 3
  • 8
  • This should be the answer as the accepted is not the correct answer, as the OP was obviously in need for search using ***only the keyword***, not limited by user, organization, or repository. – xpt Dec 30 '18 at 19:50
  • @xpt - no, the OP asked how he can do the same thing through Github API - and the answer addresses that spot on, by explaining well, it's just not possible, but here's what you can do with it... furthermore, this answer is pointing to an unofficial project maintained by one person. Its results pale in comparison to Github's. e.g. if you search "import tensorflow" you'll get 24 hits on searchcode, but 3M+ on Github. – Mark Z. Aug 28 '20 at 07:46
  • When I stressed ***"only the keyword"***, I meant not limiting by user, organization, or repository, i.e., the accepted answer only says it's just not possible. For the _"here's what you can do"_ part, the search is _again_ limited by repository. So it is actually not been able to provide a solution for OP. This, albeit an unofficial project maintained by one person, can at least satisfy what the OP wants. That's what I meant when saying this should be the accepted answer. Full disclosure, I have no association with that project or that person at all. I came to know that site only from here. – xpt Aug 28 '20 at 15:06