The easy way to get started with bi-directional contract testing
May 30, 2022
What’s this article about?
Up until now, the Pact contract testing framework was characterized by having the consuming party in charge of generating contracts and, therefore, initiating the workflow. This approach is known as Consumer-Driven Contract Testing (CDCT).
In distributed systems, where communications between applications are key, consumers are usually the weakest link since they are at the mercy of the changes made in the APIs. That’s why the aforementioned workflow has clear advantages — it ensures from the outset that there aren’t incompatibilities between applications and that the provider cannot make changes that affect the consumers’ operations.
However, implementing this approach can be tricky in certain circumstances. For example:
- If the consuming party lacks technical detail about the provider’s implementation
- If the communication between teams isn’t fluid
- If the provider doesn’t verify the consumer-generated contracts
- If the workflow and organization are highly centered on the provider and the consumers have to adapt to the changes being made
Bi-directional contract testing (BDCT) was born to help provide a better solution for those situations. In this article, we’ll look at how to use it when only a few tests have been run in Postman to test the provider side.
Why is this interesting?
There’s no doubt that Consumer-Driven Contract Testing has some major advantages. Generally, it’s the best option for environments where there is good coordination between teams. However, that isn’t always the case.
On the other hand, there are situations where the provider has already generated an Open API specification or already developed a perfectly valid API test suite, meaning introducing contract verification into its code may be redundant.
In those cases, the bi-directional contract testing approach provides teams with a lot of versatility, allowing them to opt for the most convenient workflows.
It should also be noted that the two methodologies are not mutually exclusive. For instance, the same provider can use different techniques with their consumers.
The main advantage of implementation with Pactflow is that introducing contract testing into projects is relatively quick and easy. Primarily, that’s because Pactflow handles the compatibility check between the systems and the two parties just have to publish their communication schemes.
Where can you apply it?
Common use cases
In all likelihood, it makes most sense to use bi-directional contract tracing in projects that are already underway and where automated integration tests are available or the API is in OpenAPI format. In these cases, starting with a consumer-driven approach would take a lot of work and may not even be necessary.
Another common scenario where you’d want to use this is when companies or teams are organized around the provider, forcing the consumers to adapt to changes in API. In these cases, it can be difficult to make the necessary changes to workflows or API source code to enable contract verification. Here, you would want to limit yourself to just adding the specific publishing actions in Pactflow, which will allow you to verify that the systems are compatible.
To learn about more use cases for bi-directional contract testing, check out this website.
Where not to use
We can’t say 100% that this isn’t applicable for certain cases. Although if the communication between teams is fluid and the development of systems is underway, the consumer-driven approach might be your best bet since it favors early error detection and synchronization between teams.
How does it work?
Bi-directional contract tracing is a type of static contract tracing where two contracts — one representing the expectations of the consumer and the other representing the provider’s capabilities — are compared to make sure that they are compatible.
Teams generate a consumer contract from a mocking tool (like Pact or Wiremock) and the API providers verify a provider contract (an OAS) using a functional API testing tool (like Postman). Pactflow then statically compares the contracts, all the way down to the field, to guarantee that they are still compatible.
Bi-directional contract tracing offers the possibility for each team to keep using their current testing tools. Generating a consumer contract and an Open API specification for the provider is enough because PactFlow will automatically run the checks and communicate the results.
This simplifies adoption and reduces the time it takes to implement contract testing in workflows.
An Example
Here, we’ll explain the process to follow for introducing bi-directional contract tracing between two systems. In this example, we’re taking into account that we already have a collection of Postman tests for the provider.
Prerequisites
All the code for this example can be found in this repository.
Pactflow Account
To upload the contracts and have them correctly validated, we will need to have a Pactflow account.
Pactflow API Token
Once you have your account, generate an API Token to be able to send requests to Pactflow via HTTP. For that:
- Go to Settings
- Click on the API Tokens side menu
- Generate a Read/write token (CI)
Environment variables
Finally, you have to export both variables from the Pactflow environment to use them in your scripts:
- PACT_BROKER_TOKEN: with the token you just generated
- PACT_BROKER_BASE_URL: with the domain name that was generated when you created the Pactflow account. For example: https://testdemo.pactflow.io
Software requirements
For this example, you’d need to have:
Steps to follow on the provider's side
From the provider’s side, the service specification in OpenAPI format must be published in Pactflow.
The steps to follow are shown in the image below:
Generating the Open API (OAS) specification
In our case, we have a Postman collection that contains requests that test the provider’s API. Through it, we’ll generate our OAS.
To do so, we’ll need to use a tool that does the conversion for us. In this case, we’ll use postman2openapi — whose binary we’ve included in the repository — together with the following script:
#!/bin/bash
echo -e "\\n\033[34m ==========> Convert Postman collection into OAS <========== \n\033[0m"
./bin/postman2openapi ./provider/students.postman_collection.json > ./provider/oas.yaml
and then we’ll run the following command from the project base directory:
./scripts/postmanToOpenAPI.sh
As a result of running the script, we’ll have the oas.yaml file with the API specification in OpenAPI format that will be saved in the provider folder.
Publishing the provider contract in Pactflow
Once we’ve generated our OAS, we’ll publish it in Pactflow using the Pact CLI. For that, we have the following script:
#!/bin/bash
if [ "${1}" == "" ]; then
echo "Please specify PACTICIPANT. Example: ./publish.sh provider-postman"
echo "By default, version will be picked from last commit id. If you want to define a specific version please use: ./publish.sh provider-postman [VERSION]"
exit 1
fi
PACTICIPANT="${1}"
VERSION="${2}"
if [ "$VERSION" == "" ]; then
VERSION=$(git rev-parse HEAD)
fi
OAS=$(< provider/oas.yaml base64)
if [ "$OAS" == "" ]; then
OAS=$(< oas.yaml base64)
fi
BRANCH=$(git name-rev --name-only HEAD)
echo -e "\\n\033[34m ==========> Uploading OAS Provider to Pactflow <==========\n\033[0m"
curl \
--fail-with-body \
-X PUT \
-H "Authorization: Bearer ${PACT_BROKER_TOKEN}" \
-H "Content-Type: application/json" \
"${PACT_BROKER_BASE_URL}/contracts/provider/${PACTICIPANT}/version/${VERSION}" \
-d '{
"content": "'"$OAS"'",
"contractType": "oas",
"contentType": "application/yaml",
"verificationResults": {
"success": true,
"content": "'"$OAS"'",
"contentType": "application/json",
"verifier": "postman"
}
}' || exit 1
echo -e "\\n\033[34m ==========> Tagging ""$PACTICIPANT"" contract with current BRANCH <========== \n\033[0m"
docker run --rm \
-e PACT_BROKER_BASE_URL \
-e PACT_BROKER_TOKEN \
pactfoundation/pact-cli:latest \
broker create-version-tag \
--pacticipant "${PACTICIPANT}" \
--version "${VERSION}" \
--tag "${BRANCH}"
and run the following command from the project base directory:
./scripts/publish-provider.sh provider
If the execution is successful, we’ll have a contract published in Pactflow. It should look similar to the image below:
Checks the feasibility of the provider deployment (can-i-deploy)
With the command can-i-deploy, we’ll get an immediate response about whether it’s safe to release a version of an application into a specific environment.
To do so, just like in the previous step, we’ll use the Pact CLI together with the following script:
#!/bin/bash
echo
if [ "${1}" == "" ] || [ "${2}" == "" ]; then
echo "Please specify PACTICIPANT and ENVIRONMENT. Example: ./can-i-deploy.sh provider-poc production"
exit 1
fi
PACTICIPANT="${1}"
ENVIRONMENT="${2}"
echo -e "\\n\033[34m ==========> Check viability of the ""$PACTICIPANT"" deployment (can-i-deploy) <==========\n\033[0m"
docker run --rm \
--platform=linux/amd64 \
-e PACT_BROKER_BASE_URL \
-e PACT_BROKER_TOKEN \
pactfoundation/pact-cli:latest \
broker can-i-deploy \
--pacticipant "$PACTICIPANT" \
--to-environment "$ENVIRONMENT" \
--latest
To run it, we’ll use the following command from our project’s base directory:
./scripts/can-i-deploy.sh provider production
In our execution case, the provider is the PACTICIPANT and production is the ENVIRONMENT.
Versioning the provider's deployment
If the can-i-deploy returns a successful result, we can release our application and notify Pactflow of the launch… following the golden rule of tagging.
Pact recommends setting the branch property when publishing pacts and verifying results, as well as using record-deployment or record-release when deploying/versioning.
To carry out the deployment versioning, we’ll use the Pact CLI alongside the following script:
#!/bin/bash
echo
if [ "${1}" == "" ] || [ "${2}" == "" ]; then
echo "Please specify PACTICIPANT, ENVIRONMENT. Example: ./record-deployment.sh provider-postman production"
echo "By default, version will be picked from last commit id. If you want to define a specific version please use: ./record-deployment.sh provider-postman production [VERSION]"
exit 1
fi
PACTICIPANT="${1}"
ENVIRONMENT="${2}"
VERSION="${3}"
if [ "$VERSION" == "" ];
then
VERSION=$(git rev-parse HEAD)
fi
echo -e "\\n\033[34m ==========> Record deployment of new ""$PACTICIPANT"" version in a new environment <==========\n\033[0m"
docker run --rm \
--platform=linux/amd64 \
-e PACT_BROKER_BASE_URL \
-e PACT_BROKER_TOKEN \
pactfoundation/pact-cli:latest \
broker record-deployment \
--pacticipant "$PACTICIPANT" \
--environment "$ENVIRONMENT" \
--version "$VERSION"
If we wanted to version the release, we’d have to change the record-deployment script for record-release in the command that launches the docker with Pact CLI.
And to run it, we’ll launch the following command from the project’s base directory:
./scripts/record-deployment.sh provider production
After running that script in Pactflow, you should see the following:
Steps to follow on the consumer side
You can see the steps that Pactflow describes for the consumer side in the image below:
If you aren’t going to use Pact, you have to choose a tool that allows you to extract the mock information or use a pre-existing adapter to convert it to a pact (contract) file.
Several tools come with options that allow you to serialize your mocks to a file. Others require an introspection to be run through their APIs. Keep this in mind before you start.
Writing consumer tests and generating contract
Once the tool has been chosen — in our case, we’re going to use Pact – it’s time to implement the tests on the consumer side. Make sure to take the API behavior that the system expects into account to ensure that you have the coverage that you need.
The consumer contract tests are implemented in the ConsumerContractTest.java file. To generate the contract, you have to run the following command from the project base directory:
cd consumer && mvn verify
As a result of the above command, you’ll get the consumer’s contract, which will be hosted in the consumer/target/pacts directory.
Publishing the consumer contract in Pactflow
To publish a contract, there are several options. In our case, we’re using the PactCLI through a docker image, alongside the following script:
#!/bin/bash
echo
if [ "${1}" == "" ]; then
echo "Last git commit id will be used as contract version, if you want to define a specific version please use: ./publish-consumer.sh [VERSION]"
echo
fi
VERSION="${1}"
if [ "$VERSION" == "" ]; then
VERSION=$(git rev-parse HEAD)
fi
BRANCH=$(git name-rev --name-only HEAD)
echo -e "\\n\033[34m ==========> Publish Consumer contract <==========\n\033[0m"
docker run --rm \
-w /pacts-container \
-v "${PWD}"/consumer/target/pacts:/pacts-container \
-e PACT_BROKER_BASE_URL \
-e PACT_BROKER_TOKEN \
pactfoundation/pact-cli:latest \
publish \
. \
--consumer-app-version "$VERSION" \
--branch="$BRANCH" \
--tag="$BRANCH"
And to execute it, we’ll launch the following command from the project base directory:
./scripts/publish-consumer.sh
If the execution is successful, we’ll have the following contract published in Pactflow:
Check the feasibility of the consumer deployment (can-i-deploy)
With the can-i-deploy command, we’ll get an immediate response about whether it’s safe to release a version of the application into a specific environment.
To check, we’ll use the same script that we used in the provider’s step, but this time the PACTICIPANT is the consumer and the ENVIRONMENT is production.
./scripts/can-i-deploy.sh consumer production
Versioning consumer deployment
Just like in the case of the provider, if the can-i-deploy returns a successful response, we can deploy our application and notify Pactflow of the deployment following the golden rule of tagging.
We’ll use the same script we used for the provider step, but this time PACTICIPANT is the consumer and the ENVIRONMENT is production.
./scripts/record-deployment.sh consumer production
As a result of executing that script in Pactflow, we get the following:
Pipeline
To integrate all the steps described above in a Jenkins pipeline, we’ve created a Jankinsfile with different stages in which we’ll call each of the scripts that we’ve used in the different steps:
#!/usr/bin/env groovy
/* groovylint-disable-next-line CompileStatic */
pipeline {
agent any
environment {
PACT_BROKER_BASE_URL = '<pact_broker_url>'
PACT_BROKER_TOKEN = '<pact_broker_token>'
}
stages {
stage('Get project') {
steps {
/* groovylint-disable-next-line LineLength */
git branch: 'main', url: 'git@gitlab.sngular.com:sngulartech/bi-direccional_contract_testing.git'
}
}
stage('Generate OAS') {
steps {
sh './scripts/postmanToOpenAPI.sh'
}
}
stage('Publish provider contract to Pactlow') {
steps {
sh './scripts/publish-provider.sh provider-postman'
}
}
stage('Can-i-deploy provider') {
steps {
sh './scripts/can-i-deploy.sh provider-postman production'
}
}
stage('Record deployment provider') {
steps {
sh './scripts/record-deployment.sh provider-postman production'
}
}
stage('Generate Consumer contract') {
steps {
dir('consumer') {
sh 'mvn verify'
}
}
}
stage('Publish consumer contract to Pactlow') {
steps {
sh './scripts/publish-consumer.sh'
}
}
stage('Can-i-deploy consumer') {
steps {
sh './scripts/can-i-deploy.sh consumer production'
}
}
stage('Record deployment consumer') {
steps {
sh './scripts/record-deployment.sh consumer production'
}
}
}
}
You also have other configuration options. For example, you could have two pipelines (one for the provider steps and another for the consumer) with webhooks that notify each party when a change is made to a contract. For more information on setting this up, check out this article.
Our conclusions
This new Pactflow functionality does some heavy lifting when it comes to facilitating the inclusion of contract testing within development flows. It provides excellent versatility and makes it easy to transition teams to this work model.
It makes the entry curve to bi-directional contract testing quite easy and allows us to re-use most of the work already done in test automation tasks. Overall, we believe that it’s easy to start implementing in new projects as well as in ones that are already underway.
Code repositories
All of the code from this example can be found in this repository.
References