How to Test a Serverless API Using AWS Serverless Gradle Test Runner

Every project team sooner or later asks the question — how to test APIs? Do we need to write API tests? How can we write them? There are a lot of questions on this topic. Moreover, the answers for them are also varied and depend on each particular case.

Nowadays, a lot of solutions are using the microservices approach. They have a set of small APIs that communicate with each other. They get data from a database or other resources, apply some business logic on top of it, and return as an API response. So, you want to be sure that each of such APIs is working correctly, and they work well together.

It sounds like a complex task, but let’s see how to elaborate the testing approach for your project.

API Testing Strategies

We want to highlight two main testing strategies that can be considered for your API testing. As you know, the main problem here is not to write tests, it is rather how to deal with all the API’s dependencies? We see here two options: mock all external API dependencies, or do not mock anything, and just use real API deployment. You can find a short overview of each of the strategies below.

Mock all dependencies

With this strategy, you must mock all responses from other APIs, and in some cases, queries to a database.

The development process may look like this.

1. Developer commits code.

2. Then it goes through all quality gates including code review, code style check, build, unit tests, etc.

3. When the build is ready, you do a temporary deployment of the API.

4. Tests runner will use the temp API to run all API tests. API tests can mock all external dependencies in the source code or just simulate other services like SoapUI tool can do.

5. The last step is the deployment of the real API.

Use a real deployment

In this approach you do not mock any dependencies, you just deploy your APIs as is to one of your environments, e.g. dev, test, stage, etc. It’s a real deployment, so the API uses all external services to get its job done.

The development process may be the following.

• Developer commits code.
• Code goes through all quality control gates including code review, code style check, build, unit tests, etc.
• When the build is ready, you deploy your API.
• Tests runner use API to run all API tests. The API tests can insert temporary data to databases in order to check some specific cases and then remove it after testing is complete.
• The last step is notifying the team about the testing results. Of course, here you can add more steps like fail deployment process and return old API’s deploy.

Our solution

In our project, we decided to use a real API and test them after deployment. The main reason being that we also want to test all integrations and be sure that they all work properly. Of course, there are some other reasons like a huge amount of data to query, huge effort to mock all external APIs, we can deploy any build at any time, etc.

Most of our APIs are built on top of AWS Lambda or AWS Fargate technologies. So, there are no dedicated servers for them. As you may have guessed, we also don’t want to have a separate server for API tests runner. So, we went with AWS Fargate as a test runner. We will describe it in more detail in the next chapter.

The API development and testing process is simple.

1. Developer commits changes to GitHub.

2. Then TeamCity executes all quality control gates and creates a build package.

3. In the Octopus Deploy, we deploy a new build package to one of our environments.

4. The last step we have is Run API Tests, which starts ECS Task with API tests.

• It takes the latest tests package from S3 bucket build results.
• It executes all API tests and uploads the results to S3 bucket, which are available via S3 bucket viewer through CloudFront distribution. You can find plenty of bucket viewer implementations available on the internet.
• At the end, it sends a link to test results by email using SNS topic.

API Tests Runner

We want to mention one important point before describing the API tests runner — in our project API test are written by QA automation team and developers. So, we need a language that suits both parties, is easy to learn and work with. Thus, we decided to go with a Groovy language and use the Spock testing framework. As a task runner, we use a Gradlew wrapper.

The API tests runner solution consists of a few main parts.

1. Dockerfile. It defines the tests runner docker image, which is used in the ECS Fargate task

FROM gradle:4.10-jdk-slim

ENV BUCKET_NAME=integration-tests
	
WORKDIR /

RUN apt-get update && apt-get install -y curl python
	
RUN apt-get install -y python-pip 
RUN pip --no-cache-dir install --upgrade awscli
	
COPY . /e2e_tests
	
WORKDIR /e2e_tests
RUN chmod 777 $(find ./ -name '*.sh')
ENTRYPOINT ["python", "./run.py"]

2. Run.py script. A Python script that reads all input parameters and passes them to tests runner.

import subprocess
import os

package=os.environ['TESTS_PACKAGE']
bucket=os.environ['BUCKET_NAME']
project=os.environ['TESTS_PROJECT']
env_type=os.environ['TESTS_ENV']
package_version=os.environ['PACKAGE_VERSION']
notification_topic=os.environ['SNS_TOPIC']

try:
    subprocess.check_call(["sh", "run-tests.sh", bucket, project, package, env_type, package_version, notification_topic], stderr=subprocess.STDOUT)
except subprocess.CalledProcessError as err:
    print("Unexpected error: {0}".format(err))

3. Run-tests.sh script. It is the main worker. It executes several steps:

• Download tests package from S3
• Start a Gradlew test task
• Upload test results to S3
• Send an email with results via SNS

bucket=$1
project=$2
packageUrl=$3
env_type=$4
package_version=$5
notification_topic=$6

aws s3 cp s3://$bucket/$project/Builds/$env_type/$packageUrl $packageUrl

echo 'Package downloaded'
unzip $packageUrl
echo 'Package extracted'
chmod 777 gradlew
echo 'Run tests'
./gradlew test --no-daemon

test_results_folder=$project/TestResults/$env_type/$package_version

echo 'Copy results to S3'
aws s3 cp /e2e_tests/build/spock-reports/ s3://$bucket/$test_results_folder/ --recursive

echo 'Send results by email'
aws configure set region us-east-1
results_uri=$api_url/$test_results_folder/index.html
echo "Project: $project
Environment: $env_type
Build: $package_version
Results: $results_uri" >> results.txt

aws sns publish --topic-arn "$notification_topic" --message file://results.txt --subject "API Tests Results - $project"

4. CloudFormation script. DeploysAPI tests runner infrastructure. We don’t list it here, but it just creates ECS Service and task definitions with corresponding roles and permissions.

Conclusion

Each project needs its own API testing approach. We hope that our journey here will help you to elaborate your own testing strategy and establish a high-quality delivery pipeline.

Additionally, we’d like to mention that our API tests runner is just an approach. It can be modified to use any programming language or testing framework that suits your team.

If you have any questions, topic suggestions, or want to discuss the API testing approach in your organization, contact GreenM here.