0

I am updating a sourcegraph's projectQuery configuration for gitlab but gitlab's (for the version I have) max count per page is 100 therefore, I can not get all the projects in a single query because I have more than 100 projects although I can add additional queries to fetch multiple projects but this I have to monitor and keep track of if the new projects are added that does not seems ntutive.

project query could look like this

"projectQuery": [
        "../v4/projects?per_page=100&page=1",
        "../v4/projects?per_page=100&page=2",
        ...
        ...
    ]

how can I improve this, a script could be better or extracts all the projects and add them to projects's key in the config? I am asking here because this looks a very generic question

DDStackoverflow
  • 505
  • 1
  • 11
  • 26
  • See also https://stackoverflow.com/questions/48829255/gitlab-count-total-number-of-issues – Aleksey Tsalolikhin May 07 '20 at 18:12
  • According to GitLab API documentation (https://docs.gitlab.com/ee/api/#offset-based-pagination) there is no way to get more than 100 records per page; in other words, it's a hard max. – Aleksey Tsalolikhin May 07 '20 at 18:14
  • @AlekseyTsalolikhin, thanks. `--head` can help us but again it would be multiple queries, so I was thinking to write a bash script to get all the projects and feed it to `projects` key. – DDStackoverflow May 07 '20 at 18:20
  • Yep. It would be nice if there was a toolkit for the GitLab API, like octokib.rb for the GitHub API (see https://developer.github.com/v3/guides/traversing-with-pagination/#consuming-the-information) -- I don't there is one yet, at least I couldn't find one! – Aleksey Tsalolikhin May 07 '20 at 19:42
  • 1
    Maybe you could use python-gitlab (https://python-gitlab.readthedocs.io/en/stable/cli.html)? `gitlab project list` – Aleksey Tsalolikhin May 07 '20 at 20:36
  • thank you for all the options :) – DDStackoverflow May 07 '20 at 21:13

1 Answers1

1

Try using python-gitlab

From reading the documentation, it seems that

gitlab project list --all

ought to return a list of all projects.

You can install it with:

pip install python-gitlab
Aleksey Tsalolikhin
  • 1,518
  • 7
  • 14