How to get the output of Buildkite job run through the Buildkite REST API

Question

I have a Buildkite pipeline: sample which I use to run Unit Tests on my Application: sampleapp

Within the Buildkite application, I have set up a job - Unit Test which actually runs the unit tests through a script: unittest.sh

The output of this script (in the form of Logs) is available in the Buildkite UI If I open the builds Page: https://www.buildkite.com/org/sample/builds/buildNumber and click on the Unit Test Section. The output is in the form of the Script run logs and at the end it specifies the result of the unit runs in following format:

[16:17:02]: ▸ Test Succeeded
+--------------------+-----+
|       Test Results       |
+--------------------+-----+
| Number of tests    | 773 |
| Number of failures | 0   |
+--------------------+-----+

Now, I have a client which calls the BuildKite REST API through the endpoint https://api.buildkite.com/v2/org/sampleapp/pipelines/sample/builds which returns a list of all the builds in JSON format. It has a lot of data for each build for example (a subset of the data is) :

[
  {
    "id": "xyz",
    "url": "xyz",
    "number": 1,
    "creator": {...},
    "pipeline:" {...},
    "jobs": [
      {
        "id": "abc",
        "type": "script",
        "name": "Unit Test",
        "command": "sh .buildkite/unittest.sh",
        "log_url": "https://api.buildkite.com/v2/org/sampleapp/pipelines/sample/builds/1/jobs/12oidna-3e1n-f3fsa2/log",
        "raw_log_url": "https://api.buildkite.com/v2/org/sampleapp/pipelines/sample/builds/1/jobs/12oidna-3e1n-f3fsa2/log.txt",
        "artifacts_url": "https://api.buildkite.com/v2/org/sampleapp/pipelines/sample/builds/1/jobs/12oidna-3e1n-f3fsa2/artifacts"
      }
    ]
  }
]

But, this JSON response has no information about the test results (i.e. number of tests, passed tests etc) which was reported in the logs of the unittest.sh script.

The client needs to also access the builds results somehow while calling the REST API.

One way would be for the client to first get the builds JSON response and the from that response for each build, fetch the logs (as specified in the log_url part of the JSON response) and then parse them (somehow?) to extract the unit test pass/fail results.

But, this will take a long time as there are 30 builds per response and the log files are quite large so it takes some time to download and parse a single log file.

Another challenge here would be to handle scenarios where the unit test run itself had some problems and the test result numbers were not reported at the end of file, but some error was reported.

I am wondering if there is a better way to do this? Maybe by using web-hooks to parse the data on Buildkite itself and then somehow send it to a receiving endpoint on the client.

But, I have no idea on where to start and how to do that.. Can someone please help me in doing that or at least point me in the right direction ?

score 0 · Answer 1 · answered Aug 25 '20 at 10:45

You can access the log output for any job via the REST API:

https://buildkite.com/docs/apis/rest-api/jobs#get-a-jobs-log-output

Most test runners have the ability to also output results as json or xml file. If you can find such an option then you could store your test results more richly as an artifact:

https://buildkite.com/docs/pipelines/artifacts

You could even use this file to uploads the test result summary as an annotation:

https://buildkite.com/docs/agent/v3/cli-annotate

How to get the output of Buildkite job run through the Buildkite REST API

1 Answers1