As I was sitting waiting for the train to work for the first time in 2019 I read an amusing LinkedIn article by Matt Roberts, one of my recruiter contacts, about the difference between unit and integration tests.

So this reply is for you Matt…..

I would suggest that there are more levels of testing than two, as illustrated by Jason Gorman in the tweet below.

What should we test at different levels of our “test pyramid”? pic.twitter.com/fbxqCaEUs1

— jasongorman (@jasongorman) January 1, 2019

Extending your analogy

Unit Tests

These SHOULD have been written by the original developer at the time the feature was developed. It may have been before the production code was written (Test Driven Development / TDD) or shortly after. The unit tests should demonstrate that the code acts as intended and will mock or fake any external dependencies.

Unit tests should be incredibly fast to run and allow and should provide confidence if changing the code (refactoring) that the new code hasn’t broken the assumptions of the original programming and changed its behaviour. Unit tests also serve to document the expected behaviour of the code.

In the domain of the bolt above, unit tests might include:

For the backplate and shoot component (The bit on the door):

• There is a sliding shoot
• The shoot has an attached knob
• The shoot slides easily in the backplate
• The end of the shoot is the correct diameter
• There are sufficient screws to attach the bolt to the door

For the catch component (The bit on the frame):

• The internal diameter of the catch is large enough to work
• The catch length is long enough
• There are sufficient screws to attach the catch to the frame

The unit tests verify that the lock can work as intended, but provide no eveidence of whether it will work on our particular door.

Integration Tests

This is where I would diverge from your above description a little. Integrations tests are to verify that the connection between two components work as intended.

Integration tests might include:

• With door closed and not bolted, the shoot will easily slide fully into the catch
• With door bolted, the shoot can easily slide out of the catch

As hinted at by the pyramid in the diagram, there should usually be less integration tests than unit tests, but they will probably be slightly slower to run.

Integration tests may be written by either developers or SDETs and usually involve the integration between internal parts of the software. As these integrations take place through an Application Programming Interface or API, they are often referred to as API tests

More important than the level of the tests is that they are integrated into a continuous integration pipeline AS SOON AS POSSIBLE and run regularly…

System Tests

This is actually the level that is referred to as an integration test in the original article. As demonstrated, it is perfectly possible for something to work entirely as intended, yet fail to perform the function expected of it.

System tests should be run against an entire deployment and should demonstrate that the whole system behaves as intended.
As this is the level that the commercial side of the business and customers see, tests at this level often follow some form of Behaviour Driven Development (BDD) approach that explains the purpose of the test in plain language.

System tests in this case might include (In BDD style):

• GIVEN that the door is closed, AND that the door is not bolted, WHEN the user tries to open the door, THEN the door should open.
• GIVEN that the door is closed, AND that the door is bolted, WHEN the user tries to open the door, THEN the door should not open.
• GIVEN that the door is closed, AND that the door is bolted, WHEN the user slides the bolt open, THEN the door should now not be bolted.

In the case of a software system, these tests are usually many times slower than unit and integration tests. This is especially true if they are testing through a user interface (as opposed to a system level API.) System tests can also only be run against a fully deployed system and so can only start to be run once a system reaches a certain level of completeness.

There will usually be far fewer of these than the other levels as hinted at by the often referenced Test Pyramid above.

Advice for recruiters:

If you have been tasked with finding a new SDET / Developer in Test for a client:

• Get a clear view of the test levels that they will be expected to work at. Whilst an experienced SDET should be familiar with writing unit tests for their own frameworks, they probably shouldn’t be writing unit tests for the production code. If the client needs unit tests written for legacy code, perhaps an experienced developer (i.e. not test) might be a better fit. (SDETs might however be expected to monitor unit tests produced and provide guidance to developers.)
• Ensure that you understand what technologies are required, and whether the new employee will have the time to skill up on a new technology or MUST already be experienced with it.
• Ensure that the candidate is happy to make the technology shift if one is required.
• Help the client to understand that if they need someone to set up test automation and continuous integration, they need someone with a lot more experience than to expand an exisiting team.

TLDR:

A solid development team will have tests running at many different levels. No one level is more important.

More important than the level of the tests is that they are integrated into a continuous integration pipeline AS SOON AS POSSIBLE and run regularly (every day if not every completed pull request) to catch regressions that cause them to fail.

When a bug is discovered, a new test at as low a level as possible (unit -> integration -> system) should be written that will fail until it is fixed.