Commit a file to Git during continuous integration/continuous delivery process in VSTS using VSTS API

Question

We have multiple release environments configured in VSTS which use different databases. Each database in these environments contain specific metadata which control how the application works. This metadata is regularly configured by a administration tool to the user.

We want to backup this metadata and check it into Git on every release so we have it somewhere just in case of data loss or the needs arises for comparing previous versions. We have a utility that extracts this metadata from the database which generates an XML file, so I'll run this utility and then somehow put it into Git using a task in VSTS. If it is possible to do this, we also plan to export our VSTS release definitions into a JSON and check it into Git as well.

How can I check in a file to a repository in the release definition phase?

Since you solved your problem by REST API, you can add an answer and accept it. — Marina Liu, Mar 07 '18 at 07:46

score 2 · Accepted Answer · answered Mar 07 '18 at 21:52

After some research and a few hours later of testing in Postman/VSTS - I was able to write a powershell script that sends a API request to VSTS and commits a file to a repository.

I referenced the Create a push documentation here. I needed to determine the correct values for the following properties of the JSON request:

oldObjectId - Which is not quite documented anywhere. The value needed here can be returned by calling list refs API. I believe this is similar to git show HEAD
changeType
contentType

Here is an example to retrieve the latest objectId to use as the oldObjectId value:

Invoke-RestMethod -Uri "https://{accountName}.visualstudio.com/DefaultCollection/{project}/_apis/git/repositories/{repositoryId}/refs?filter=heads%2Fmaster&api-version=4.1-preview" -Method Get -Headers $header

My next task was to prepare the newContent property. Since I was trying to upload an XML file, rawText was out of the question because of all the special characters. The only other contentType allowed is base64Encoded.

I had to use Powershell's Get-Content cmdlet with the -Raw switch enabled in order to successfully base64 encode the content. I learned that from another StackOverflow post I made earlier when I was having problems with the formatting of the XML.

$xmlData = Get-Content -Path $xmlFile -Raw
$xmlDataEncoded = [System.Convert]::ToBase64String([System.Text.Encoding]::UTF8.GetBytes($xmlData))

Then, in order to determine if I was adding a new file or modifying an existing file, I needed to add logic to the script to check for the existence of a file and set changeType to add or edit. See below:

$xmlDataRepo = Invoke-RestMethod -Uri "{accountName}.visualstudio.com/DefaultCollection/{project/_apis/git/repositories/{project}/items/?recursionLevel=OneLevel&api-version=4.1-preview" -Method Get -Headers $header
ForEach-Object -InputObject $xmlDataRepo.value -Process {if ($_.path -eq $repoXmlFile) {$changeType = "edit"} else {$changeType = "add"}}

Finally, now that I have collected all of the information and encoded my XML content. I need to build the request body and call the API. Here is the body of the request:

$body = @"
{
    "refUpdates": [
    {
        "name": "refs/heads/master",
        "oldObjectId": "$objectId"
    }
    ],
    "commits": [
    {
        "comment": "Data backed up",
        "changes": [
        {
            "changeType": "$changeType",
            "item": {
                "path": "$repoXmlFile"
            },
            "newContent": {
                "content": "$xmlDataEncoded",
                "contentType": "base64Encoded"
            }
        }
        ]
    }
    ]
}
"@

And finally, the cherry on top:

Invoke-RestMethod -Uri "https://{accountName}.visualstudio.com/DefaultCollection/{project/_apis/git/repositories/{project}/pushes?api-version=4.1-preview" -Method Post -Headers $header -Body $body -ContentType application/json

Hope that helps

score 1 · Answer 2 · answered Mar 01 '18 at 20:55

While I generally think it's a bad idea to publish to the source repo that also triggers your build, it's relatively easy.

You can invoke git.exe as part of the build. You may need to tweak the repository settings to clone the whole repository and not just a shallow clone. You can pass in the OAuth token of the build job if you enable access to it:

You can see how VSTS itself adds the authorization headers:

git remote add origin https://jessehouwing.visualstudio.com/
git config --get-all http.https://jessehouwing.visualstudio.com/_git/Torpydo.extraheader
git -c http.extraheader="AUTHORIZATION: bearer $(System.AccessToken)" fetch --tags --prune --progress --no-recurse-submodules origin

Alternatives

Every build in VSTS has a concept of "Artefacts". You can use the Copy and publish build artefact task to capture data and to make sure they're not lost.

Build history

If you like your builds to be in source control, be sure to look into th new YAML build support:

https://learn.microsoft.com/en-us/vsts/build-release/actions/build-yaml

Thanks - I poked around the VSTS API documentation and found that they have a API call that might do something similar. https://www.visualstudio.com/en-us/docs/integrate/api/git/pushes#update-a-file This API call looks like I could send an updated file. Have you heard about the API calls to add/update/remove files from a repository? — Anthony Klotz, Mar 01 '18 at 21:31
That will work as well. I still recommend using build artefacts instead of pushing back to git. — jessehouwing, Mar 02 '18 at 10:23

Commit a file to Git during continuous integration/continuous delivery process in VSTS using VSTS API

2 Answers2

Alternatives

Build history