Greg Ostravich

Summary: This book is intended as an introduction to JUnit, an automated testing framework for Java developers.

Introduction
Dave Thomas and Andy Hunt, who brought you the book The Pragmatic Programmer, have designed a series of books to introduce readers in a hands-on way to CVS, JUnit, and build automation. The books are designed as a way for the lead Java developers or architects on a project to introduce these topics to developers that have no experience with these tools. The CVS and JUnit books have been published, but the third book in the series is still being written at the time of this review. This review is for the JUnit book in the series. The book is intended for people who have not used JUnit before, but even though I have used JUnit in the past, I learned from this book as well. In addition to being a tutorial on JUnit, the book covers testing principles and guidelines for testing that would not be specific to any one technology. Even though the book talks about JUnit, the authors recognize that there are other testing frameworks available.

Why JUnit and Unit Testing are useful.
I really enjoyed the book and I liked that the authors explained up front that unit testing is for developers -- not for testers, managers, or others. It really needs to be understood as a tool that helps the developer, and I appreciate that they make that point. In the introduction, the authors also try to make clear how extensive unit tests need to be, but I felt they didn't pin it down. They cover this topic more toward the end of the book as a part of refactoring but I still wasn't clear on how much is too much for developing tests on new code. The reason I mention the issue about new code, is that I found myself doing tests that I wasn't sure were "too many" for unit testing a small project I did. I did not do Test Driven Development (TDD) as the authors touch on in the end of the book so maybe that's why it wasn't so clear to me. I felt the authors made a good case for testing. They mentioned how unit tests can be documentation for the code, but left out how nice it is that if you do unit testing before or during coding then the documentation (unit tests are a form of documentation) is also fresh -- it's much more accurate -- than if you test at the end of a project.

Something to be aware of
I didn't really have any negatives that I found in the book. I found a few typos which I don't mention here and I found one thing that might cause a problem for people who don't have any experience with JUnit. Since this book is targeted at beginners and introducing them to the process, the authors should mention Appendix B, which explains JUnit installation in Chapter 2, not later. I have used JUnit before, but because I was using a new machine, I had to install the latest SDK and JUnit jars so I could do the exercises and I ran into problems. I was able to resolve the problems (classpath issues) but there should be an earlier reference to Appendix B, which warns users of this problem and shows them how to set up JUnit.

The Where and How of Testing using JUnit
I thought the authors did a good job of explaining where to put your tests. Although I had written unit tests for a small project I did and I had read the FAQ and tutorial available at JUnit.org, I still learned a lot from this book. One thing I learned about were how to test for exceptions. I hadn't done that in my unit tests before, and the authors had an example where the developer catches the exception and if the exception was thrown, asserts true. If not, then the developer fails by calling the fail method in the try block. They point out that these kind of asserts in exception handling are a form of documentation and suggest letting JUnit do the heavy lifting by passing some exceptions to the JUnit framework instead of trying to catch everything.

Suites: How to use them
From my own experience, I wasn't clear on the proper use of Test Suites. Suites are the way you organize your collections of tests. The authors explain that through reflection, JUnit tests all the testXXX methods that are there, but in any given test class you can use a suite to ensure that only the tests you want run are run. In their example there is a very long test that is excluded from the suite. They mention a skeleton for JUnit as an alternative to specify some tests in case you don't want to run all tests. The example they give is a test that runs for hours or days. You don't want those tests run during every code change. They suggest if you want a subset of tests to run, you pass each test as an argument to the main that is your test skeleton. Then you add each suite by that name. If you specify no parameters, then you add all test suites. I thought that was pretty clever. The other trick they include is to have a suite that you include from other testing code where you can include the shorter tests you want. If there are no lengthy tests, you can include the entire class as an option too. The idea they present is that you can build your set of test components this way, including only the components that may be tested in your everyday testing. I understand the set-up and teardown of a unit test, but I didn't understand how the per-suite setup inner class worked in the code example in the book. Organization of unit tests is also good information -- they suggest how to organize your tests so you don't have to re-add something to each individual unit test in case the whole suite needs it. When naming your tests in a suite, the authors suggest using "pendingTest" to prepend on test classes that are not ready for prime-time yet. I'm not sure if that's a good idea because I would think a test could get forgotten. At least with the name "pendingTest" it's easily searchable to see if you've left any tests out.

What to test for: testing principles
This is an area of the book that I feel really shines. I had no formal testing experience except for testing I have done in the past on individual projects and my recent foray into unit testing using JUnit. The authors give you a Testing Tutorial -- with some really great acronyms on what to test for. I won't go into great detail on all of them because if you're interested you should buy the book. One of the acronyms they give is that when you are testing, you should check your "Right BICEP." The "Right" means are the results correct? If it ran correctly, how would I know? The "B" refers to boundary conditions. When the authors talk about descriptions on boundaries for testing, I realized there were areas I hadn't considered and will always look for now. They suggest that through good OO design, you restrict within the object creation for some boundaries. For example, you cannot enter degrees outside of 0-359 range without throwing an exception for a constructor of a Degrees class. The "I" in the "BICEP" is a reminder to check Inverse Relationships. If I'm testing the square root, then test the square of that number matches your original number. The "C" means cross-check your results using other calculations. Use the square root function in the assert, but to check the inverse take that number times itself instead of using a square function for the other value you are asserting against. The "E" means can you force error conditions to happen. Finally, the "P" represents Performance. Are performance characteristics within bounds? You can use Mike Clark's JUnitPerf to check that. Other suggestions they make are to use large amounts of data for tests, and to automate the data using a data file. I will enclose an example of that below. The one thing to be very careful of when using a data file is that bad data can really mess you up during testing.

#
# Comment here
#
assertvalue	testvalue or values in space separated tokenized list.

The authors list the acronym "CORRECT," which contains a list of boundary conditions to test for in any code. I won't go into what each letter stands for, but I will tell you the "T" is for Time and the authors point out an interesting tidbit regarding how many hours are in a day. In the U.S., where most states observe Daylight Savings Time, you have a 23 hour day in the Spring and a 25 hour day in October and there are libraries out there that do not account for this boundary condition very well.

Good tests are A-TRIP -- Automatic, Thorough, Repeatable, Independent, and Professional. The anecdotal story about a group of programmers -- one using unit testing and one not was great! There was a group using unit tests and that group's manager had never heard of their methods. The group that did not use unit tests had a manager that was familiar with their code because QA was constantly reporting bugs.

The authors point out that bugs will clump in sections of code. Any bugs found "in the wild" mean you have a hole in your "net" of tests through which the bug escaped. If you find a bug, write the test you missed first and validate that's where your problem is. Then write the code to fix it.

Mock objects
The description on mock objects really helped me to understand how to extend the interface of an existing object and override the mock object stubs. The only problem is Thomas and Hunt don't talk about when really to use mock objects. They give examples of resources that are expensive and you would want to mock, but I wonder about tests where you really want to gauge performance. Do you set up mocks for one set, then the real deal for the other and depending on time and when the tests are run pick and choose from what set you use? Or do you just use decorators like JUnitPerf when performance is an issue and mock everything up for all your automated unit tests. They seem to make the argument that repeatable consistent tests are very important so you're not chasing phantom bugs -- this might be where mock objects really shine. The other thing that was important when they discussed mock objects is that you're not often needing to test the mock object, but in fact conditions before invoking or using the object. They show an example where the unit test can be factored out of the mock object.

Regression testing and Legacy Code
The authors suggest you pick the most painful parts. What the authors called the "mud ball" code has been better described by Mike Clark as a big ball of barbed wire. If you're going to put your hand in there make sure to clean that portion up so it's only painful once. Don't use JUnit to test everything and the authors suggest testing the parts you are concerned about breaking in any changes to legacy code. There is a success story example where JUnit is used to discover any problems introduced in a key piece of code for low level code. The team in this story made a rule that if any code had a problem twice, then it must have a JUnit test by the end of the week. The authors also suggest bringing both test code and production code to the code review.

Other areas to test: miscellaneous things to be aware of
There is an example the authors give using a recipe list program that tests individual elements of the recipe. One thing I have done in the past is override the equals method of an object to compare two of those object's elements instead of the usual default reference comparator. I don't know if that's preferable to what the authors do in their code example.

Good programming practice when testing GUIs is to refactor so that the GUI isn't tightly coupled. Then test those objects separately from the GUI. This isn't specific to JUnit but it's nice that they remind us of this.

The authors talk about Invariants -- items that are always in a certain state throughout the life of a class. For example, an array index would never be less than 0 or greater than the maximum number of items. The one thing that was not clear is how often we should be testing these invariants. They present a code example where they are tested once. Is that enough?

The authors also mention an idea for testing parameters. That is, that for some input, you have common tests that everyone uses and relies on -- to prevent multiple efforts that may be missing the mark. This seems like an agile approach and may make sense for some larger projects.

Appendix A lists a series of "Gotcha's" -- I won't enumerate them here. You can see them in the book. One of them I thought was interesting and hadn't thought about is related to floating point errors. You could see an error where an assertion fails because 1.000000 is not equal to 1.000000 -- which can happen because your printing of the float doesn't show the entire float. Besides checking for equivalence within tolerance for floats, the authors mention you may see date and time differences because an assertion is off by a few milliseconds.

Summary
The key thing about this book is that it is both good for a beginner to JUnit and a beginner to unit testing. I don't consider myself a beginner, but the book pointed out areas of the code I should be testing that I might not have covered in existing unit tests I've written. Having examples and making you run the existing code examples is a useful exercise to acquaint the inexperienced with JUnit. They also have other exercises to help you understand unit testing better and there is a nice summary in the back of the book of the different conditions to check for when developing unit tests. There is also a list of URLs and references to testing frameworks and related projects. I enjoyed this book and would recommend it.

If you are interested in purchasing this book, go to the Pragmatic Programmer's site at pragmaticprogrammer.com to order it. If you use our Denver Java User Group discount code (DenverJUG_0011), you get 20% off the price of the book. (DenverJUG receives no benefit from this discount or from book sales.)