On 6/9/2011 7:56, Warp wrote:
> Darren New<dne### [at] sanrrcom>  wrote:
>> Why would I want to run the unit tests for the XML parser after making
>> changes to the color picker dialog box?  Or, more generally, why would I
>> want to run unit tests that are only executing code that hasn't changed
>> since last time the unit tests passed?
>
>    No amount of unit tests can catch all possible bugs. You may have a
> thousand unit tests for a small module, yet there may still be bugs there.
>
>    In some cases these bugs *might* be triggered by seemingly unrelated
> changes somewhere else in the program, even if there is no connection
> between the two modules (might be less likely in "safe" languages, but
> probably not impossible).

I'd agree with this, except that unit tests are supposed to be independent. 
(Indeed, that's my main complaint about relying on unit tests to tell you if 
something is broken.) So if there's a bug in the XML parser, say, it 
shouldn't even be running in the same process as the unit tests for the 
color picker.

Now, unsafe languages, that's interesting. But I don't think that's a unit 
test thing as much as it's a test-the-hell-out-of-everything thing. Sure, 
the more you test code in different circumstances, the more you're likely to 
find uncaught violations of the language specification. I guess that makes 
sense, tho. That's the first good reason I've heard. :-)

And yes, something like not cleaning up a global might get caught in unit 
tests even in a safe language. Indeed, there are several places in XNA where 
static variables are used that should be instance variables. Indeed, the 
fact that calling "Exit()" during a game not only exits that game at the end 
of the current frame, but also all other future games instantiated in the 
same process, makes it very difficult to automate an entire set of tests in 
one process. :-)

>    Unless running the unit tests takes hours, does it hurt to run them?

No. But I've seen recommendations that one mocks everything *because* that 
makes the unit testing fast enough to test everything every time, as if this 
gave a clear benefit over just testing what you changed.

-- 
Darren New, San Diego CA, USA (PST)
   "Coding without comments is like
    driving without turn signals."

Post a reply to this message