Automated testing is one of the most important techniques in our arsenal. Palantir’s software performs better, lasts longer, costs less to operate and maintain, and is more stable – and this is almost entirely due to our rigour in applying automated testing.
We believe very strongly in Michael Feathers’ attitude that code without tests is by definition legacy code; which means that far too much “new” code is legacy from the moment it was written.
Test Driven Development
Test Driven Development (TDD) is the starting point for all of our testing. No line of production code is written until the test for it is written. By starting with the test we gain a few major advantages:
- We ensure that all our code is fully and properly tested, as opposed to the usual approach of slapping in a few superficial tests after the fact.
- We find that the code written tends to be more precise and targeted than normal.
- The negative cases are often catered for better.
But the most important reason is that it allows us to change the code confidently and without fear. In most systems – with missing or superficial tests – the developers are afraid of making sweeping changes, even for the better, because they’re afraid of what might break if they do it. Our developers do not live with this fear – they can make sweeping changes confidently, run the tests, fix the tests that are failing and then know that everything is working. This ability to quickly know whether the full solution is working consistently results in much better code delivered cheaper and quicker.
Behaviour Driven Development
It is not sufficient to use Test Driven Development alone however. The reason is simple: what if the tests as written do not tie up with the functional requirements? We make use of Behaviour Driven Development (BDD) to ensure consistency of solution with functional requirements. What this means is that tests are written in normal language by the Business Analyst, or the Product Owner or other such business person. Business language, making use of business terms. The only constraint is a particular form to the test language (Given that … When … Then …).
The technical testers then provide the BDD system with the actions it must perform in response to certain language. In this way, the test language drives the automated test. This allows the business to easily add new tests to the test corpus as well as modify existing tests to cater for new situations and requirements. It allows the business to review and understand the automated tests, providing a high degree of uniformity between the solution and the requirements.
Acceptance Test Driven Development
This is similar to Behaviour Driven Development above, making use of the same techniques and tools, except that the tests are not the overall functional tests, but the core Acceptance Tests. In a sense, we use ATDD as the starting point of the overall solution, and then “fill in” the functional tests using Behaviour Driven Development.
If these tests pass, then the client is normally happy. Note that for Palantir, this is the starting point of quality, not the ending.
Consumer Driven Contract Testing
Palantir have been pioneers of this technique to ensure fast and effective integration between disparate systems, long before it was popularised in the Thoughtworks Technology Radar. We use this technique regularly when connecting to third party services, creating automated tests that duplicate our usage of the service, which we can provide to the service provider if there are any issues discovered.
This also helps us identify when service providers change the underlying behaviour of their services without notification, which is a common problem in the industry.