TDD: The what, how, why, when?

What is TDD?

Test Driven Development, is an agile delivery practice where an engineering team chooses to write a test case for the functionality they are developing before trying to implement that functionality.

The idea behind this approach is that you gradually build up a solution to problem case by case until you’ve implemented the whole solution by first having a test for each requirement of the system you’re working on.

Almost everyone and their dog seem to have strong opinions about TDD and there are cases where it doesn’t make sense to use TDD given niche circumstance X but generally I want to be working with a team that embraces TDD. I’m getting ahead of myself however.

How to do TDD:

While, write a test first isn’t exactly a complex idea to grasp there are different approaches to TDD but most approaches seem to follow a pattern of red, green, refactor.

• RED: write a failing test
• GREEN: write as little code as possible to get the the new test AND all the previous tests passing
• REFACTOR: look for simplifications / improvements in your code without breaking the tests

Repeat until done! :-) Simple right?

There are other variations of this such as writing a high level, end to end, test first and then using that to drive out other lower level changes. Obviously the end to end test wont pass until a lot of smaller changes have been made so the cycle becomes red, amber, refactor, green with amber being an ignored test. I generally prefer to work bottom up starting with unit tests but I’m not opposed to starting abstract and gradually filling in the blanks.

When starting out with TDD it can be very tempting to jump ahead to a finished / more complex solution. I do advise being quite adherent to the ‘by the book’ approach when starting TDD and then when your team is comfortable writing a test first become a little more pragmatic.

I think this is where a lot of people find TDD frustrating, you can see where the code is going and want to get there quickly but if you rush to the end you’ll inevitably miss something important you’d have found on the journey. Maybe an edge case goes untested leading to defects later on for example. Even when I can see how the code will probably look I feel it is useful to make sure I’ve covered my bases.

A common criticism of TDD is that you focus on passing tests rather than building the feature. You write a test, then it passes, then you write another more complex test to drive out the functionality. I’ve heard it argued that this can cause you to miss the forest for the trees. I believe it is very important that a test relates back to a high level concept. Instead of, ‘should call the db mapper function empty cart when…’ the test should be related to the business intent, ‘empties the shopping cart when…’ You can do a more behavioural (BDD-esq) development with TDD even with lower level tests.

Why do TDD? / Benefits of TDD for a delivery team:

Following TDD leads to a gradually built up implementation which should only be as complex as it needs to be to solve the problem at hand. Each branching path in the code will have a test case covering the expected behaviour because the test case was written first and the branch is a response to the declared intent of the test.

Your test suite evolves alongside and has shaped your code. If you follow TDD your test suites should capture the intent as well as the functionality of your implementation. I have heard this referred to as, “living documentation” which certainly sounds cool. In a code base produced with TDD looking at the tests is a fantastic way to understand how and why code is written the way it is quickly.

The edge cases you’re aware of are tested because you wrote a test for them before handling them in code.

A team builds a comprehensive safety net against breaking changes. Everyone is human and we have short, imperfect memories. One month after writing some code it is likely that I won’t remember exactly what it does, let alone why it does things a certain way but if I have a thorough test suite for that code then I can feel considerably more confident when I need to make changes.

TDD will not prevent code from having defects entirely but it will help ensure those defects are not repeated when caught.

Good reasons not to use TDD:

They exist! Like any engineering practise its important to weigh the benefits and trade offs rather than blindly following dogmatism. In my experience there are two use cases when not using TDD might be a good choice.

When learning something very new to you or your team:

To be effective whilst coding with TDD it helps to have familiarity with the tools and ecosystem you’re working with. I personally find it hard enough to learn a new framework and language without immediately starting with testing tools. Think, hello world level of familiarity with an ecosystem. Obviously code written like this will not be, at least in my mind, production worthy and after getting through a 101 you can bet I’ll be looking into testing.

When quality isn’t the main concern:

If you’re rapidly prototyping an idea or a series of ideas in the early days of a project or product then maybe the benefits of TDD are not worth the development cost to your team. When your codebase / idea starts to solidify I’d highly recommend going back and adding the missing test coverage.

Bad reasons not to use TDD: (Warning this section contains opinions…)

TDD is too much effort:

Being afraid to change untested code and the resulting hours worth of manual regression testing which accompany each change is also a lot of effort if you ask me. TDD initially can feel like a lot more work but as a codebase becomes more mature the insights a comprehensive test suite can give you will greatly accelerate a team.

I don’t need to test my code because, ‘egocentric statement here’:

Sure buddy, you’re the only infallible developer. ;-)

I’ve successfully built a product before without any tests:

TDD isn’t the only way to write software. This industry has an alarmingly high failure rate for projects and anything to help build a more maintainable, higher quality solution is worthwhile in my book.

I just don’t want to:

Well as I stated at the beginning TDD is a choice, a practice that a team needs to choose to buy into or not. If you genuinely don’t see any upside then don’t do it.

Testing this code is hard:

That sounds like a code smell! Maybe it is time to pull apart a class and split some responsibilities? Okay. I’ll admit, middleware and framework glue are hard to test effectively and doing so isn’t super valuable because we hope the framework well… works.

Refactoring becomes much harder:

This shouldn’t be the case. TDD should make refactoring easier with a higher degree of confidence that you haven’t made a breaking change. However when starting out with TDD it can be easy to test your self into a brittle implementation by not testing well…

I’ve actually had this issue on a project, the test suite was very brittle to change, even with no actual functionality changes. This was really counter productive and frustrating. The brittle-ness occurred because every method was tested directly in a class where everything was public.

To fix it the team determined which methods were actually being used externally to the class, made everything else private and changed the tests to verify the private methods worked only via the exposed public methods. This allowed us to change the private methods and refactor the working of the class without breaking the implementation.

In summary…

I personally find TDD a very useful practice for teams to follow. The initial inertia of writing a test first pays dividends in the future when a code base has matured and achieved a non-trivial level of complexity.

I’ve had to work on a codebase with tens of thousands of lines of untested code in individual classes before and it isn’t an experience I’d like to repeat if I can help it. Every change was a nightmare which involved hours of manual regression testing and even then the system was in such a state that having a full dev team watching it was just providing life support.

Had that codebase been produced using TDD would it be such a monstrosity? I can’t guarantee it would be great but it certainly wouldn’t have been as bad! If a class is becoming too complex to test its a sign it is doing too much.

TDD isn’t going to solve all your problems but in most cases it is going to be a worthwhile investment for an engineering team. You’ll produce high quality, well tested, well understood, easily maintained and extensible code. That code will have insights as to engineering decisions taken captured in test cases and extending and making changes will be easier because you’ll know if you’ve made a breaking change.