Interaction with TeamCity using service messages and REST api

This article treats about interactions with TeamCity – about controlling its flow or getting useful information. All interaction examples will in form of python snippets – for service messages it will be simple prints and for REST – ‘requests’ library.

Why do care?

In ideal situation TeamCity will do everything for us – fail/pass build, print intuitive message, count tests for us, even give possibility to create charts based on standard data. If your case hits this ideal situation then you’re lucky. But usually this built-in TeamCity intelligence is not enough, and we have to tell it how to behave when specific condition is fulfilled.

Let’s start

Service message is just a string in a format like:

##teamcity[<messageName> 'value']

Which needs to be printed into build log like e.g.:

• bash

echo "##teamcity[<messageName> 'value']"

• python

print "##teamcity[<messageName> 'value']"

• java

System.out.println("##teamcity[<messageName> 'value']");

The simplest case when we would like to interfere with TeamCity behavior is to fail a build. One may ask why, when TeamCity can do it by itself? That is true – but when we want to have more meaningful reason why it failed rather than simple:

We have to tell TeamCity to do so by printing service message:

##teamcity[buildProblem description='foobar']

Where ‘foobar’ is an actual failure message:

In case of builds with success status we can also customize build label using:

##teamcity[buildStatus text='{build.status.text} and some extra]

Where {build.status.text} is an actual label, so

##teamcity[buildStatus text='{build.status.text} and stuff']

Will set something like this:

Tests, tests, tests

TeamCity counts tests number and displays it in an overview label. Standard test libraries like junit or unittest have its own support and there is no need to handle anything unless you want to customize them by introducing your own pass/fail criteria or create ‘fake’ tests. By ‘fake’ tests I understand part of functionality which we would like to be treated as test in TeamCity manner in order to have it’s statistics or history (i.e. starting HTTP service and run tests on it –  treat service startup as test with pass/fail criteria based on startup time).

There are 2 basic messages for controlling test flow – starting and finishing and additional one for failing test. So printing:

##teamcity[testStarted name='foobar test']
##teamcity[testFinished name='foobar test' duration='800']

will start and finish a test in TeamCity what will result in adding new test to Tests tab:

Adding fail message in the middle:

##teamcity[testStarted name='foobar test']
##teamcity[testFailed name='foobar test' message='Error message']
##teamcity[testFinished name='foobar test' duration='800']

Will produce failed test in TeamCity with Error message:

Statistics and charts

TeamCity provides some standard charts in Statistics tab like success rate or build duration:

But wat is more important it also gives possibility to create your own charts based on statistics provided. Provided how? The answer is again – service messages.  Every piece of data with a structure of key – float/int value can be put to TeamCity with following message:

##teamcity[buildStatisticValue key='<key_name>' value='<value>']

Such reported pair is visible in ‘Parameter/Reported statistics value’ tab and based on it new chart can be created:

Get rather than put

In the first part of this article we have focused on sending data to TeamCity, while the second part is about retrieving data with REST API. As mentioned at the begging all examples will be based on Python library called requests together with shutil and xml.

Give me those artifacts

Getting artifacts from other builds is one of the most useful TeamCity API features. Output of one build can be used as input to another without any middleman. To download artifact we would need, besides obvious TeamCity URL, so called build_type_id and build_number. Build_type_id is the one given in configuration (Custom_build_type):

While build_number is incrementing number of particular build (319):

The URL for downloading artifacts from #319 build of Custom_build_type will be like:

http://<team_city_url>/repository/downloadAll/Custom_build_type/319

and the python example, how to handle it:

import shutil
import requests

USER = 'tc_user'
PASS = 'tc_password'


def get_artifacts(team_city_url, build_type, build_number):
    request = '%s/repository/downloadAll/%s/%s' % (team_city_url, build_type, build_number)
    target_file = 'example.zip'

    response = requests.get(request, auth=(USER, PASS), stream=True)

    with open(target_file, 'wb') as out_file:
        response.raw.decode_content = True
        shutil.copyfileobj(response.raw, out_file)


get_artifacts('http://teamcity.jlabs.com', 'Custom_build_type', '319')

Instead of passing particular build number we can use one of three key-words like .lastFinished, .lastSuccessful or .lastPinned that are pretty self-explaining or a build id which is unique numeric value identifying build – buildId:id. Below are two examples demonstrating such possibilities:

http://<team_city_url>/repository/downloadAll/Custom_build_type/.lastSuccessful

http://<team_city_url>/repository/downloadAll/Custom_build_type/28872198:id

How to retrieve useful data?

Besides downloading artifacts, there many reason for getting information about build itself – its number, status, any parameter or even log (or artifact file) content without downloading it. Assumption is – we know exact build id where we would like to retrieve data from. When this information is unknown we have to find one.  In order to do so we need to know so called locator which is nothing more than query parameter limiting search results among all build in TeamCity. I would like to present just few of them – for full list refer to documentation.

• id=<build_id> – already mentioned unique build id for exact match
• buildType=<build_configuration_id> – all builds within build configuration
• status=<SUCCESS/FAILURE> – all build with specified status
• sinceDate=<date> – all builds started after given time e.g. 20181212T203000%2B0100 for Dec 12 2018 20:30:00, GMT+1
• personal:<true/false/any> – include private builds
• count=<number> – limit gathered build list to specified number

 The data possible to get can be:

• artifacts/content/<path_to file> – to get artifacts file content (also with unpacking archives: archive.zip!/file_in_archive.txt)
• downloadBuildLog.html – to read build log
• app/rest/builds/ – to get builds list

 And so:

http://<team_city_url>/guestAuth/downloadBuildLog.html?buildId=28904541 will get build log content from build with given id;

http://<team_city_url>/guestAuth/app/rest/builds/id:28872198/artifacts/content/results.zip!/log.txt will get log.txt content from results.zip archive from given build;

http://<team_city_url>/guestAuth/app/rest/builds?buildType=Custom_build_type&personal=any will get list of builds from Custom_build_type (personal and non-personal) as XML;

Below example shows how to print list of build_id’s based on locator from last URL example.

import shutil
import xml.etree.ElementTree as et
import requests

USER = 'tc_user'
PASS = 'tc_password'


def get_all_builds(team_city_url, build_type):
    request = '%s/httpAuth/app/rest/builds?buildType=%s&personal=any' % \
              (team_city_url, build_type)

    response = requests.get(request, auth=(USER, PASS))

    parsed = et.fromstring(response.content)

    builds = list(el for el in parsed.findall('build'))

    return [build.get('id') for build in builds]


print get_all_builds('http://teamcity.jlabs.com', 'Custom_build_type')

Summary

Topic of this article is to broad to cover all possibilities – I just wanted to present few options in order to give you some clue how does it works. Detailed interaction can be found in TeamCity documentation.