Thinking on it a bit, it sounds like unit testing is just a way to add 
Eiffel-style programming-by-contract to your code without any actual 
language support for it.
-- 
Darren New, San Diego CA, USA (PST)
    Ada - the programming language trying to avoid
    you literally shooting yourself in the foot.

Post a reply to this message

Darren New wrote:
> Thinking on it a bit, it sounds like unit testing is just a way to add 
> Eiffel-style programming-by-contract to your code without any actual 
> language support for it.

I thought the term was "design by contract"?

Well whatever. Contracts seem like a nice idea; you make it clear who's 
supposed to check for error conditions - whether it's supposed to be the 
caller or the callee. And you can enforce it by enabling checks at 
runtime. Having said that, I don't recall using it much myself in my own 
code.

On top of that, any kind of testing is going to be easier if a 
function's output depends only on its current inputs...

-- 
http://blog.orphi.me.uk/
http://www.zazzle.com/MathematicalOrchid*

Post a reply to this message

Orchid XP v8 wrote:
> Darren New wrote:
>> Thinking on it a bit, it sounds like unit testing is just a way to add 
>> Eiffel-style programming-by-contract to your code without any actual 
>> language support for it.
> 
> I thought the term was "design by contract"?

Yeah. Brain-O.

> Well whatever. Contracts seem like a nice idea; you make it clear who's 
> supposed to check for error conditions - whether it's supposed to be the 
> caller or the callee. And you can enforce it by enabling checks at 
> runtime. Having said that, I don't recall using it much myself in my own 
> code.

Yep. Every time you use "assert" to check your inputs, you're doing that. 
Except with minimal language assistance.

Unit testing just takes those asserts outside the function.

> On top of that, any kind of testing is going to be easier if a 
> function's output depends only on its current inputs...

Yep. The problem with "unit testing" per se is that it only tests a unit. It 
doesn't test anything except the invariants of the unit you're testing.

-- 
Darren New, San Diego CA, USA (PST)
    Eiffel - The language that lets you specify exactly
    that the code does what you think it does, even if
    it doesn't do what you wanted.

Post a reply to this message

Darren New wrote:

> Yep. Every time you use "assert" to check your inputs, you're doing 
> that. Except with minimal language assistance.

Eiffel has preconditions *and* postconditions, which most people don't 
seem to bother checking with assert() calls.

> Unit testing just takes those asserts outside the function.

Whilst Haskell does have an assert() function, it seems to be more 
popular to use QuickCheck.

QuickCheck does automated randomised testing. Give QC a property that's 
supposed to hold for any inputs, and it generates inputs at random and 
checks whether the property holds.

The big snag, of course, if that if your code fails only under very 
specific, rare circumstances, randomised testing might not hit the bug. 
(This is why books on testing talk about testing limits, flow control 
paths, etc.)

Also, asserts or design by contract will catch the source of a crash in 
a live running program. (Assuming that enabling the tests doesn't slow 
the program to the speed of a glacier...)

>> On top of that, any kind of testing is going to be easier if a 
>> function's output depends only on its current inputs...
> 
> Yep. The problem with "unit testing" per se is that it only tests a 
> unit. It doesn't test anything except the invariants of the unit you're 
> testing.

I thought the idea was that you then combine several units into a larger 
unit, and test that (and so on, recursively). Then again, maybe that's 
what they mean by "integration testing"...

-- 
http://blog.orphi.me.uk/
http://www.zazzle.com/MathematicalOrchid*

Post a reply to this message

Orchid XP v8 <voi### [at] devnull> wrote:
> Eiffel has preconditions *and* postconditions, which most people don't 
> seem to bother checking with assert() calls.

  A little thought experiment: How would you write a comprehensive
postcondition to a function that sorts a list/array of elements?
(Assume there is no reliable third-party sorting function available.)

-- 
                                                          - Warp

Post a reply to this message

>   A little thought experiment: How would you write a comprehensive
> postcondition to a function that sorts a list/array of elements?
> (Assume there is no reliable third-party sorting function available.)

Sorting a container is at best O(n log n), but determining whether one 
is already sorted is only O(n). :-)

-- 
http://blog.orphi.me.uk/
http://www.zazzle.com/MathematicalOrchid*

Post a reply to this message

Warp wrote:

>   A little thought experiment: How would you write a comprehensive
> postcondition to a function that sorts a list/array of elements?
> (Assume there is no reliable third-party sorting function available.)

Here's a better one: You write a compiler. How the **** do you determine 
whether it actually works properly? (!)

-- 
http://blog.orphi.me.uk/
http://www.zazzle.com/MathematicalOrchid*

Post a reply to this message

Orchid XP v8 <voi### [at] devnull> wrote:
> >   A little thought experiment: How would you write a comprehensive
> > postcondition to a function that sorts a list/array of elements?
> > (Assume there is no reliable third-party sorting function available.)

> Sorting a container is at best O(n log n), but determining whether one 
> is already sorted is only O(n). :-)

  Hmm, that's not what I asked.

-- 
                                                          - Warp

Post a reply to this message

Orchid XP v8 wrote:
> Darren New wrote:
> 
>> Yep. Every time you use "assert" to check your inputs, you're doing 
>> that. Except with minimal language assistance.
> 
> Eiffel has preconditions *and* postconditions, which most people don't 
> seem to bother checking with assert() calls.

Yes. The postconditions are checked with unit tests. And people try to check 
invariants that way too. And unit tests generally don't try to check loop 
invariants.

> QuickCheck does automated randomised testing. Give QC a property that's 
> supposed to hold for any inputs, and it generates inputs at random and 
> checks whether the property holds.

Again, you're checking invariants, which I find is usually not that useful.

> I thought the idea was that you then combine several units into a larger 
> unit, and test that (and so on, recursively). Then again, maybe that's 
> what they mean by "integration testing"...

Yes, that's integration testing.  Then there's "functional testing", where 
you test the program does what the customer actually wants. Then there's 
system testing, where you make sure the program does what the customer 
actually wants even when interacting with other code (basically).

-- 
Darren New, San Diego CA, USA (PST)
    Eiffel - The language that lets you specify exactly
    that the code does what you think it does, even if
    it doesn't do what you wanted.

Post a reply to this message

Warp wrote:
> Orchid XP v8 <voi### [at] devnull> wrote:
>> Eiffel has preconditions *and* postconditions, which most people don't 
>> seem to bother checking with assert() calls.
> 
>   A little thought experiment: How would you write a comprehensive
> postcondition to a function that sorts a list/array of elements?

Trivial.

Forall(i < length-1) assert(j[i]<=j[i+1]);

Basically, the definition of sorting. Of course, you need to be able to put 
qualifiers in your assertions.

-- 
Darren New, San Diego CA, USA (PST)
    Eiffel - The language that lets you specify exactly
    that the code does what you think it does, even if
    it doesn't do what you wanted.

Post a reply to this message

Orchid XP v8 wrote:

> Whilst Haskell does have an assert() function, it seems to be more 
> popular to use QuickCheck.

Of course, the *other* thing that's popular is constructing software in 
such a way that it cannot malfunction in the first place. (E.g., if 
negative numbers aren't allowed, use unsigned integers. Now you don't 
have to check whether the input is negative, because it can never *be* 
negative.) Obviously, there are limits to how well this approach works...

-- 
http://blog.orphi.me.uk/
http://www.zazzle.com/MathematicalOrchid*

Post a reply to this message

Darren New wrote:
> Warp wrote:
>> Orchid XP v8 <voi### [at] devnull> wrote:
>>> Eiffel has preconditions *and* postconditions, which most people don't
>>> seem to bother checking with assert() calls.
>> 
>>   A little thought experiment: How would you write a comprehensive
>> postcondition to a function that sorts a list/array of elements?
> 
> Trivial.
> 
> Forall(i < length-1) assert(j[i]<=j[i+1]);
> 
> Basically, the definition of sorting. Of course, you need to be able to
> put qualifiers in your assertions.}

A sorting function that, given any input, returns [1,2,3,4,5], will pass 
that test.

Try again.

Post a reply to this message

Nicolas Alvarez wrote:
> Darren New wrote:
>> Warp wrote:
>>> Orchid XP v8 <voi### [at] devnull> wrote:
>>>> Eiffel has preconditions *and* postconditions, which most people don't
>>>> seem to bother checking with assert() calls.
>>>   A little thought experiment: How would you write a comprehensive
>>> postcondition to a function that sorts a list/array of elements?
>> Trivial.
>>
>> Forall(i < length-1) assert(j[i]<=j[i+1]);
>>
>> Basically, the definition of sorting. Of course, you need to be able to
>> put qualifiers in your assertions.}
> 
> A sorting function that, given any input, returns [1,2,3,4,5], will pass 
> that test.

Heck, for that matter anything returning a zero-length array would pass. OK, 
to be *comprehensive*, you would have to add that the result is a 
permutation of the original. You could do this by counting how many times 
each original value occurs in the list and making sure it occurs in the 
result the same number of times.

-- 
Darren New, San Diego CA, USA (PST)
    Eiffel - The language that lets you specify exactly
    that the code does what you think it does, even if
    it doesn't do what you wanted.

Post a reply to this message

On 06/03/10 11:47, Darren New wrote:
>> On top of that, any kind of testing is going to be easier if a
>> function's output depends only on its current inputs...
> 
> Yep. The problem with "unit testing" per se is that it only tests a
> unit. It doesn't test anything except the invariants of the unit you're
> testing.

	I wouldn't call it a "problem".

-- 
Guitar for sale. Very cheap. No strings attached.

Post a reply to this message

Darren New wrote:

> Heck, for that matter anything returning a zero-length array would pass. 
> OK, to be *comprehensive*, you would have to add that the result is a 
> permutation of the original. You could do this by counting how many 
> times each original value occurs in the list and making sure it occurs 
> in the result the same number of times.

Let's hope Warp doesn't ask us to test for a *stable* sort...

Post a reply to this message

Neeum Zawan wrote:
> On 06/03/10 11:47, Darren New wrote:
>>> On top of that, any kind of testing is going to be easier if a
>>> function's output depends only on its current inputs...
>> Yep. The problem with "unit testing" per se is that it only tests a
>> unit. It doesn't test anything except the invariants of the unit you're
>> testing.
> 
> 	I wouldn't call it a "problem".

Well, a limitation.  I do very little programming where the invariants of 
the class are particularly difficult to ensure.  That would be 
"datastructures" and "file systems", which I use system/language libraries for.

-- 
Darren New, San Diego CA, USA (PST)
    Eiffel - The language that lets you specify exactly
    that the code does what you think it does, even if
    it doesn't do what you wanted.

Post a reply to this message

Darren New wrote:
> Neeum Zawan wrote:
>> On 06/03/10 11:47, Darren New wrote:
>>>> On top of that, any kind of testing is going to be easier if a
>>>> function's output depends only on its current inputs...
>>> Yep. The problem with "unit testing" per se is that it only tests a
>>> unit. It doesn't test anything except the invariants of the unit you're
>>> testing.
>>
>>     I wouldn't call it a "problem".
> 
> Well, a limitation.  I do very little programming where the invariants 
> of the class are particularly difficult to ensure.  That would be 
> "datastructures" and "file systems", which I use system/language 
> libraries for.

Or, to rephrase, sure, a SQL server can have gazillions of unit tests that 
make sense. But I don't write SQL servers. That's been done. :-)

-- 
Darren New, San Diego CA, USA (PST)
    Eiffel - The language that lets you specify exactly
    that the code does what you think it does, even if
    it doesn't do what you wanted.

Post a reply to this message

Darren New <dne### [at] sanrrcom> wrote:
> Nicolas Alvarez wrote:
> > Darren New wrote:
> >> Warp wrote:
> >>> Orchid XP v8 <voi### [at] devnull> wrote:
> >>>> Eiffel has preconditions *and* postconditions, which most people don't
> >>>> seem to bother checking with assert() calls.
> >>>   A little thought experiment: How would you write a comprehensive
> >>> postcondition to a function that sorts a list/array of elements?
> >> Trivial.
> >>
> >> Forall(i < length-1) assert(j[i]<=j[i+1]);
> >>
> >> Basically, the definition of sorting. Of course, you need to be able to
> >> put qualifiers in your assertions.}
> > 
> > A sorting function that, given any input, returns [1,2,3,4,5], will pass 
> > that test.

> Heck, for that matter anything returning a zero-length array would pass. OK, 
> to be *comprehensive*, you would have to add that the result is a 
> permutation of the original. You could do this by counting how many times 
> each original value occurs in the list and making sure it occurs in the 
> result the same number of times.

  That was one warning example at school aboit how easy it is to write
pre- and postconditions which actually are not comprehensive enough.
The sorting example is superb to demonstrate that, because many people,
like you, think that it's enough to go through the array after the sorting
is done and check that every value is larger or equal to the previous value.
The counter-example to this is, of course, to make the "sorting" fill the
array with the same value (eg. the first one), and it would pass the test.

  At the same time it poses an interesting thought problem: *How* would you
write a post-condition that truly checks that the resulting array has the
same values as the original, but sorted? And a bit more difficult: What
would be the asymptotically fastest possible way of doing the check?

-- 
                                                          - Warp

Post a reply to this message

Warp wrote:
>   That was one warning example at school aboit how easy it is to write
> pre- and postconditions which actually are not comprehensive enough.

Yes, it's a good point. :-)

> The sorting example is superb to demonstrate that, because many people,
> like you, think that it's enough to go through the array after the sorting
> is done and check that every value is larger or equal to the previous value.

Obviously had it not been an offhand post to p.o-t I would have put more 
thought into it. I thought you were asking more about the technology of how 
it was done, and I was saying "obviously you need universal qualifiers like 
this..." rather than trying to actually be comprehensive.

>   At the same time it poses an interesting thought problem: *How* would you
> write a post-condition that truly checks that the resulting array has the
> same values as the original, but sorted? 

Sure. As I said, you can loop thru the original array, counting the number 
of times old[i] appears in old, then for each of those loop thru new[j] 
counting the number of times old[i]==new[j].  So it's at worst O(N^2).

It's not hard to specify, which is what I thought you were getting at. Hard 
to do efficiently, but that's what people do in debug builds.

For example, MS has a rule that you're not allowed to publish performance 
reviews of their beta software. At one point, they gave a beta of Excel to a 
journalist, and they tried it out, and published that it was terribly slow. 
Well, it was a debug build, and every time you did an update, it would 
calculate everything with the optimized path, then it would iteratively 
calculate every single cell over and over until it stabilized (or they 
looped more than the number of cells), and then compared the two. Of course 
it was slow.

Once you're confident your sort works, you don't have to test you haven't 
introduced new values in production. :-)

-- 
Darren New, San Diego CA, USA (PST)
    Eiffel - The language that lets you specify exactly
    that the code does what you think it does, even if
    it doesn't do what you wanted.

Post a reply to this message

On 06/04/10 08:32, Darren New wrote:
> Neeum Zawan wrote:
>> On 06/03/10 11:47, Darren New wrote:
>>>> On top of that, any kind of testing is going to be easier if a
>>>> function's output depends only on its current inputs...
>>> Yep. The problem with "unit testing" per se is that it only tests a
>>> unit. It doesn't test anything except the invariants of the unit you're
>>> testing.
>>
>>     I wouldn't call it a "problem".
> 
> Well, a limitation.  I do very little programming where the invariants
> of the class are particularly difficult to ensure.  That would be
> "datastructures" and "file systems", which I use system/language
> libraries for.

	I think you took me too seriously. I meant that it does precisely what
it claims to. It's not meant for all kinds of tests. Just one kind.

	I suppose limited is more appropriate, but it still sounds too
negative.<G> It's like saying mutt is limited because it doesn't send
mails and needs the sendmail command. It just was never meant for
sending mail.

-- 
Guitar for sale. Very cheap. No strings attached.

Post a reply to this message

Neeum Zawan wrote:
> 	I think you took me too seriously. I meant that it does precisely what
> it claims to. It's not meant for all kinds of tests. Just one kind.

I understand. I consider it a "problem" when a test tests something that 
doesn't really tell you things are working.  Especially the way it's hyped 
as making your code easy to refactor and blah blah blah.

> 	I suppose limited is more appropriate, but it still sounds too
> negative.<G> It's like saying mutt is limited because it doesn't send
> mails and needs the sendmail command. It just was never meant for
> sending mail.

Given that you have to put a fair amount of work into unit tests, and that 
what unit tests ensure isn't especially interesting in most cases, I'd say 
yes, "limited" ought to sound negative.

Automated testing is great.  Automated testing of a single class when that 
class does something profound all by itself is useful. Most of *my* classes 
are designed for interacting with other classes, tho, rather than for 
maintaining particular invariants.  Places where the classes are designed to 
maintain invariants are places where I'm usually using a library anyway.

-- 
Darren New, San Diego CA, USA (PST)
    Eiffel - The language that lets you specify exactly
    that the code does what you think it does, even if
    it doesn't do what you wanted.

Post a reply to this message