My hierarchy of testing

I originally wrote this when I worked at Apple, and it was the most popular thing I ever wrote.

This document comes from a long career in software testing where I went through more than a dozen full release cycles.

I’ve executed and observed 5 major types of testing, seen the bugs they catch, and seen the overall quality of projects that rely on different types of testing. I don’t have objective data to back any of this up, so this is purely my subjective observations. After you read this, you’ll see that I value the subjective very highly in testing despite the fact that pretty charts and graphs can be compelling.

I think this is important because everyone has scarce testing resources and needs to get the best bang for the buck. Many companies invert their emphasis and I think that goes a long way to explaining why much of the software out there is poor quality.

So, here are what I see as the five types, in order from most to least important.

Ad hoc testing

This type of testing goes by man…