I started this post a couple of weeks ago, only to be reminded last week of just how critical the ability to authenticate your API requests is. We had made essential changes to our authentication service that were working fine in our development environment. Once these changes however were pushed to our pre-prod environment, suddenly my test user was failing to authenticate. When we tried to release the next service, suddenly every API test using OAuth authentication was failing in setup. Here I was in the stark realisation that until we fixed the issue, many of the most important API tests on our various back end systems could not be run. It was a very nervous time for me until we managed to diagnose and fix the problem.

When we tried to release the next service, suddenly every API test using OAuth authentication was failing in setup.

When we tried to release the next service, suddenly every API test using OAuth authentication was failing in setup.

Here I was in the stark realisation that until we fixed the issue, many of the most important API tests on our various back end systems could not be run.

Alexander Oliveira Dunn 2019: This Article

Don’t worry, I have not finished my series on .NET Core pagefactory; However I have a writer’s block on using blocks so I thought I would share some of my experience of API test automation. This will probably turn into a series, but today I am just going to start with a few things to consider before a developer even starts to set up the project.

1) How will your Auth process work?

You are going to have some form of authorisation (sorry I am British!) on your api calls right?

I know how this goes. Having to get auth for your api calls in development is a royal pain in the ass, so many teams add auth as one of the final steps.

So there you are as a dedicated tester, writing a bunch of tests. The technology doesn’t matter. I prefer writing in the same langauage as the developers, but it could just as easily be a Postman collection with well written set up and assertion code. Perhaps you already have your tests set up in a CI/CD pipeline or maybe they are just runnning locally for now: The important thing is that they have been running for some time now and picking up unintended changes and regressions quickly.

Then just as you are preparing for release, the required authorisation process is enabled. These are some of the possible scenarios:

• no auth required
• a fixed API key
• OAuth requiring the user to enter username and password through a user interface
• A cookie

No authentication or a fixed API Key? No problem – Carry on.

Otherwise, if you do not have an automated means to generate the required token or cookie for your test suite, all those tests are now useless for integration in your release pipeline. That’s not to say that they didn’t serve a purpose during development, but now any chance of using them to catch post release regressions quickly has passed.

2) Do you have swagger?

Ok so I should say OpenApi, but it just doesn’t offer the same opportunities for puns. OpenApi is a standard for documenting restful APIs. Your api is described in a YAML file that can be used to document the API for users and even provide a browser based UI for basic testing.

There are a couple of ways that OpenAPI can be used as part of your development process:

1. Specification first development

As part of your design process, you define the endpoints, requests, responses and errors in a json file called swagger.json. This file can then be used to automatically generate code for server and clients in a host of programming languages, and even provide the UI for testing / exploring / integrator training.

Advantage

Both application developer and test developer can use the tools at swagger.io to start the server and client implementations with a few clicks. This includes the models and you just need to then add the logic for your endpoints.

Disadvantage

We all know that most plans are not correct. It is inevitable that as the API is developed, changes will be need to made to some of therequest or response models: This will require a new specification and changes to the models in both test and application code.

2. Implementation first development

Application developers write the API as normal, and use a swagger generator e.g. Swashbuckle in C# .net to generate the swagger.json file as part of the build.

Advantage

• Developers have freedom to explore the right approach for the problem.
• As your API and models change, the documentation is automatically updated.

Disadvantages

• It is much easier to start a project without having fully scoped and designed it.
• As a test developer you lose the benefit of having your models etc ready prepared.
• Writing tests before / alongside the implementation becomes much harder.

You may have gathered that I am very much in favour if the specification first approach, but certainly not everyone agrees. Both have their merits.

3. Don’t share models

You should be writing a black box tests. Whether the specification was complete up front, or evolved through the project, your tests should be there to verify that the API continues to perform as specified as the application code base develops.

Anything that fails an API Test for a published API should cause the release of a newly versioned API. Non breaking additions are OK, but anything that changes behaviour should not happen.

If you share the data models that your application developers are using, when they make a breaking change to the models, the tests magically change too. This is how breaking changes get released without people realising.

If I don’t have a swagger definition to work from, I prefer to generate my own models; Online tools e.g. my favourite json2csharp can quickly generate your models from valid response json. Perfect where you already have working Postman tests to build from. This has the added benefit of effectively testing the user experience for the consumers of your API.

Alternatively you could copy the application model code entirely into your project, just don’t link directly to the actual models that the application uses.

If you share the data models that your application developers are using, when they make a breaking change to the models, the tests magically change too. This is how breaking changes get released without people realising.

Alexander Oliveira Dunn 2019: This Article

TLDR:

Before starting on a project to write an automated test suite for an API in development, ensure that:

• You have a means to get any required authorisation token in an automated manner.
• You have an OpenAPI (swagger) specification planned, either up front or autogenerated. It not only helps with testing and writing tests, but also makes documentation far quicker.
• It is understood that your black box tests will not share the application models.

A reminder:

If you want to ask me a question, Twitter is undoubtedly the fastest place to get a response: My Username is @AlexanderOnTest so I am easy to find. My DMs are always open for questions, and I publicise my new blog posts there too.

Series Navigation

It’s all about the “Auth” >>