Always verify that the bug exists before you go looking for the cause.
I just finished tracking down a bug that boiled down to one of two things, depending how you interpret the situation: (1) wrong expectations, or (2) not understanding the algorithm.
Instead of verifying the bug, I set off looking for the cause. After quite a while I felt no closer to finding it, so I decided to attack it in reverse. In doing so, I convinced myself that the output was correct. Then I was able to explain it to the person who reported the bug to me and all was well.
In this particular case, the code was relatively young. I wrote it a month or two back to generate the "related search" feature on Yahoo Search. For example, when you search for "jeremy" you'll see several related searches at the top of the result page.
The algorithm is really quite powerful and produces some fascinating insights. I implemented and tuned it, I did not invent it.
Anyhoo, part of the problem was that the code normally works with millions and millions of lines of input (and may take days to finish). But the test case that "proved" the bug contained maybe 20 lines on input. In working with millions of lines of input, there's a lot of noise that we throw out. You see, the haystack in which the needle is buried also contains a fair amount of dung. But the dung threshold for millions of lines of input is vastly different from what you'd use for 20 lines of input.
The net result was that the 20-line run produced less data than expected, even though the code was doing just was it was designed to do.
Had I spent 5 minutes up front doing a sanity check, I'd have noticed the "bug" quite a while ago.
Lesson learned. Verify bug first, then look for bug cause.
Posted by jzawodn at May 06, 2003 04:19 PM
