Testing the Hard Stuff
My favorite class to teach is TDD. I like to challenge developers to stretch their brains into new territory. They do it every day in other ways. Learning TDD is a level higher than solving algorithm and design challenges. It is about changing the way you approach programming altogether. And my favorite situation is to see the light go on in the mind of someone who starts out as a skeptic. “I’m here because my boss said I had to go” kind of people. Great fun.
Sometime I get challenges back. By now, after doing this for a few years, I can answer many of the predicatable questions pretty easily. Every once in a while I get some harder questions. I thought I would share some of these and the answers I came up with.
No Return Value
Here is one that led me into some useful learning for myself. In my class, I use an example that simply transforms some inputs into some output. The example shows how to use TDD to evolve the transformation algorithm. keep it simple to illustrate the “how” and “why” of TDD. Once, just once in many classes, someone asked how you test a method that does not return a value. I had to pause for a moment, my head being comfortable in the world of the example and follow-on exercises that had the same general shape – testing functions with a return value. I don’t normally say much about testing behavior and using test doubles in my introductory course. But that is where the answer took me. In such a case, the method is hopefully doing something useful and you want to determine whether or not it did the right thing, but the method itself can’t tell you directly. So you need to check its state through some other method or through a collaborator that holds information indicating the result. That collaborator may be a test double or a real object. And so forth into behavior-based testing. I don’t normally go any farther than than in the first day. I save it for an advanced class.
Timers
How do you test something that uses a timer? For example, in a program version of a game that has a time limit for making an answer, things happen if the timer runs out. Something gets triggered. Of course we want to test the things that happen at the trigger time. But do we actually need to test that the tigger is fired? Do we test that those things happen when time runs out? You don’t want the test to take too long – you may run it many times a day as part of your unit test suite. So do you set up a test case with a short time in place of a real-world long time? Is the time-length parameter available to change for testing? Would that be a good design for the production case? Is a short time sufficiently representative of reality? These are some of the questions to ask.
Random Elements
How do you test something that had a random element? What if the randomization makes it impossible to predict an expected test value? An example is the initial layout of a game board such as MineSweeper. My answer is to assume the randomization will work and/or do enough visual inspection to convince yourself that you are using the randomization support of the language correctly. This might mean having your tests actually draw the screen or some low fidelity representation of it to the console. Then choose some secondary effect that is predictable as your test case. Does the game board initialization create the right number of game pieces? Are there enough letters in the jumbled word? Things like that. Any verification of results is better than the old school “code and assume success” model.
Hard-to-Know Expected Value
What if the test data is really complicated, so complicated that it is hard to know what the right test result is? An example I heard once was this: we are allowing the user to rotate a 3-D model of a a human brain. How do we know if the rotation is correct? My answer was to use a simpler 3D shape as the test case – a soccer ball perhaps or maybe just a cube. Make the test simple enough to manage and still give you useful information that the code is doing what it should be doing.
Test Logic == Tested Logic
A variation of this is a test result that is just too darn hard to calculate without using a program equivalent to the unit under test. Maybe a genetic sequence or a Fast Fourier Transform. You don’t want to reproduce the logic of the UUT in the test itself. I usually advise that you just bite the bullet and “hand craft” a test case or two using a calculator, slide rule (what is that?) or some other program.
Lots of Test Data
How do you manage a very large volume of test data? Normally we think of using data from some external source (file or database) for such cases. This introduces a new dimension into your unit testing – an external dependency. Does the unit test now need a pre-test to determine that it has the right external data? How sophisticated a pre-test? Just the right file name? A checksum on the data? Another approach is to generate the data when the test is run. You get guaranteed reproducability that way. But it may be too slow.
Concurrency
What about multi-threaded code? This is getting to be a hot topic now that computers are showing up with more than one processor. Hard stuff to TDD? It can be. I will leave the answer to experts. Check out these books:
Test Driven: TDD and Acceptance TDD for Java Developers by Lasse Koskela
Clean Code by Robert C. Martin
