Unit testing advice

Ruby
21

Hi –

On Sun, 3 Aug 2008, James B. wrote:

something running before I can get my brain into test mode. Certainly
with something like Rails, there’s a lot to do before any tests are
written, since you can’t write unit tests before you know what your
models are (and generating the model so conveniently writes the test
file for you :slight_smile:

Isn’t the idea of TDD and BDD that you discover what classes and method are
needed by writing the tests first? So there should be no model in place
prior to writing the test; the initial failure of the test is what drive the
creation of the model.

That may be the ideal for some people, but not for me when I’m
modeling a domain and especially when I’m working out a database
schema. I don’t consider unit tests to have superseded the other tools
that support those activities, including blank pieces of paper and
index cards and so on. I guess you could write unit tests for a Rails
app before creating the models, if you mocked up the objects’
attributes, their associated objects, and so on (since there would
presumably be no database yet), and then rename your files so they
don’t get clobbered when you generate the test files… but it seems
like it would be terribly arduous, with no real gain, and I don’t
think I’ve ever seen anyone do it.

literally written code test-first from the time they first learned how
to program onward. I suspect the answer is no – and if that’s the
case, it means that there is no evidence for the position that it’s
always, automatically bad to write code without a test.

Has anyone ever seen a Learn To Program or Learn Language Blub book that did
TDD? I doubt such a thing exists. Instead, people are shown code,
encouraged to write code, then (in so many words) told that what they were
shown and told is not the right way to code. On the other hand, having a
unit test for the 1-liner helloworld.rb seems massively goofy.

The thing is, all the people who argue that testing first is the right
way to code did not themselves learn to code that way. That doesn’t
prove or disprove anything, but it does make me wonder whether
teaching someone test methodology right out of the starting gate is
demonstrably the best way to go about teaching someone programming. I
tend to think it isn’t, though I also think that there are good and
bad ways to introduce testing into the mix (the best way probably
being to present it as essentially what they’ve been doing all along,
but more structured; and the worst way being the “OK, the fun is over,
now let’s get serious” stuff).

David

22

David A. Black said…

One thing to remember is that TDD is about development, and that not
all instances of entering code on a keyboard are development.

I just wanted to add that TDD, as well as being about development, is
also about maintenance. By producing tests, you produce an asset, a very
valuable one for us software folk, but also for the business. I place a
much higher value on the balance sheet to code that is under test to
code that isn’t.

23

David A. Black said…

Has anyone ever seen a Learn To Program or Learn Language Blub book that did
tend to think it isn’t, though I also think that there are good and
bad ways to introduce testing into the mix (the best way probably
being to present it as essentially what they’ve been doing all along,
but more structured; and the worst way being the “OK, the fun is over,
now let’s get serious” stuff).

I started on coding sheets. We wrote tests first. We used to
“desk check” our code too. We had to, it might take a couple of days to
get the cards punched before getting one, perhaps two, compilations per
day. I’m in my forties, so this is ancient, but not prehistoric history.

That said, I lean towards what David is saying, that it’s a skill to be
learned after experimenting/playing with code. After all, we all learn
to speak before we write. And being able to write well takes a lot more
skill than simply being able to write what we say.

24

On Sat, Aug 2, 2008 at 5:17 PM, Shadowfirebird
[email protected] wrote:

teaching someone test methodology right out of the starting gate is
day. I’m in my forties, so this is ancient, but not prehistoric history.
I wish I had learned this stuff ten years ago.
course, but two out of three is a hell of a good hit rate in this
much use in those times.

Check out Test-Driven Development By Example by Kent Beck. It brought
my understanding of TDD to a new level.

Regards,
Michael G.

25

I wish I had learned this stuff ten years ago.

For almost my entire career people have assumed that I can look at
their code and tell what it is supposed to do. I can’t; no-one can,
with any accuracy. All I can do is look at code and say what it
does. Which is probably not the same thing, or I wouldn’t be
looking at the code in the first place…

But if there are automated tests, they will tell you what the code
should do. And, better yet, if the programmers have been using them,
they will be accurate and up to date. They might not be complete, of
course, but two out of three is a hell of a good hit rate in this
subject.

As far as I am concerned, that’s good enough to spark a religious
conversion.

As for test-driven-design, I think it’s a fine ideal. Mostly when I
code I know in advance where I’m going, and in that case it pays to
use TDD. But sometimes I haven’t the faintest idea what I’m doing; I
don’t know how many classes I need or what jobs go in what classes –
or even if the idea is at all workable. I suspect that TDD isn’t of
much use in those times.

Shadowfirebird.

On Sat, Aug 2, 2008 at 9:19 PM, marc [email protected] wrote:

demonstrably the best way to go about teaching someone programming. I

–
Me, I imagine places that I have never seen / The colored lights in
fountains, blue and green / And I imagine places that I will never go
/ Behind these clouds that hang here dark and low
But it’s there when I’m holding you / There when I’m sleeping too /
There when there’s nothing left of me / Hanging out behind the
burned-out factories / Out of reach but leading me / Into the
beautiful sea

26

David A. Black wrote:

Hi –

On Sun, 3 Aug 2008, James B. wrote:

app before creating the models, if you mocked up the objects’
attributes, their associated objects, and so on (since there would
presumably be no database yet), and then rename your files so they
don’t get clobbered when you generate the test files… but it seems
like it would be terribly arduous, with no real gain, and I don’t
think I’ve ever seen anyone do it.

Quote:

I guess you don’t use the Rails code generation then.

Steven: No, not at all. I use TDD to generate code manually.

So you don’t even do model generation with Rails?

Steven: Not at all. Especially when I’m pairing with somebody who’s
never used Rails before or who has never done TDD before I find it
especially important not to rely on the generator to give me test stubs,
to give me things that I should know or that I should be providing for
myself. When the failings specifications tell me that I need it, because
that would be contradicting what I’m trying to teach.

–
James B.

www.happycamperstudios.com - Wicked Cool Coding
www.jamesbritt.com - Playing with Better Toys
www.ruby-doc.org - Ruby Help & Documentation
www.rubystuff.com - The Ruby Store for Ruby Stuff

27

Hi –

On Sun, 3 Aug 2008, James B. wrote:

think I’ve ever seen anyone do it.

Steven: Not at all. Especially when I’m pairing with somebody who’s never
used Rails before or who has never done TDD before I find it especially
important not to rely on the generator to give me test stubs, to give me
things that I should know or that I should be providing for myself. When the
failings specifications tell me that I need it, because that would be
contradicting what I’m trying to teach.

OK, now I’ve seen one person :slight_smile: I imagine there are more. Luckily we
don’t need a consensus; we can all do what we’re comfortable with.
It’s interesting that Steve does it this way, and I’d like to see it
in person.

It doesn’t appeal to me not to use the model generator. For one thing,
I assume that it would mean having to run sequential migrations, where
you wrote a test or spec that tested for the presence of, say, a
first_name attribute on the user; then add that column to the
database; then test for last_name; then add that column; and so forth.
I don’t mean to sound like I’m parodying what Steve is describing. I
assume that this, or something like it, is what you’d end up doing.

I’m not sure what Steve means by test stubs in this context. The model
generation just gives you a file with an empty class definition for
your tests, and I think something similar happens with RSpec. (Just to
clarify: I’m talking about model generation, not scaffold generation,
which does give you all sorts of tests and which I dislike strongly
and consider manifestly counter-productive.)

David

28

David A. Black wrote:

It doesn’t appeal to me not to use the model generator. For one thing,
I assume that it would mean having to run sequential migrations, where
you wrote a test or spec that tested for the presence of, say, a
first_name attribute on the user; then add that column to the
database; then test for last_name; then add that column; and so forth.
I don’t mean to sound like I’m parodying what Steve is describing. I
assume that this, or something like it, is what you’d end up doing.

Parenthetically, you are “allowed” to edit a pre-existing migration
file, if you
can keep track of which VERSION your production database has.

Within that loop, yes, write a test that fails because MyModel has no
has_many
NotherModel, then pass it by adding a column :nother_model_id to the
migration.

29

James B. wrote:

Steven: Not at all. Especially when I’m pairing with somebody who’s
never used Rails before or who has never done TDD before I find it
especially important not to rely on the generator to give me test stubs,
to give me things that I should know or that I should be providing for
myself. When the failings specifications tell me that I need it, because
that would be contradicting what I’m trying to teach.

I don’t know if we are discussing the same thing. This Rails command…

script/generate model MyModel

…does not create “stubs”. It creates all the files that back up a
Model,
including its my_model_test.rb and fixtures/my_models.yml files.

I can’t imagine not using it, because all it saves you is a lot of rote
typing.
Writing a test that fails because “fixtures/my_models.yml” does not
exist is not
TDD, because that’s not production code.

I often write a test that fails because MyModel does not exist yet, to
start the
cycle.

I think Dave and Steve are discussing some tool that reads Ruby classes
and
spits out stubbed test cases for each method.

30

Phlip wrote:

James B. wrote:

So you don’t even do model generation with Rails?

Steven: Not at all.

I don’t know if we are discussing the same thing. This Rails command…

script/generate model MyModel

…does not create “stubs”. It creates all the files that back up a
Model, including its my_model_test.rb and fixtures/my_models.yml files.

That’s exactly what is being talked about.

And the issue is that it creates a model file before the developer has
written a failing test indicating the need for a model file.

One might assume that the developer is going to end up in the same place
anyways, so the code might as well be automatically generated, but it’s
contrary to “write the spec, have it fail, then write the code to make
the spec pass”.

I’m not arguing in favor of dogmatic testing practices, justing pointing
out that there is at least one person who actually walks the walk.

(Besides, when a developer selects Rails as their toolkit they’ve
already made many code choices long before there are any tests to drive
those decisions, so the TDD/BDD purity thing is sort of a moot point.)

–
James B.

www.happycamperstudios.com - Wicked Cool Coding
www.jamesbritt.com - Playing with Better Toys
www.ruby-doc.org - Ruby Help & Documentation
www.rubystuff.com - The Ruby Store for Ruby Stuff

31

Hi –

On Sun, 3 Aug 2008, Phlip wrote:

Parenthetically, you are “allowed” to edit a pre-existing migration file, if
you can keep track of which VERSION your production database has.

Yes, but then you get into migrating backwards, which I increasingly
think is almost never appropriate. (Though not quite never, maybe.)

Within that loop, yes, write a test that fails because MyModel has no
has_many NotherModel, then pass it by adding a column :nother_model_id to the
migration.

I’m afraid I don’t see the advantage of that, at least in every case,
over bootstrapping at least part of a database schema and model
structure first. Certainly the kind of thing you’re describing does
happen, but if I’m creating an Order model and I know that it’s going
to belong to Customer, I’m not averse to saying so up front. Also,
with a large table, you could get into an awful lot of migrations.

It all comes out in the wash, so to speak, since once the project is
bootstrapped in some way, the different approaches to bootstrapping
tend to converge.

David

32

On Aug 3, 2008, at 12:03 PM, James B. wrote:

the walk.

(Besides, when a developer selects Rails as their toolkit they’ve
already made many code choices long before there are any tests to
drive those decisions, so the TDD/BDD purity thing is sort of a moot
point.)

Yeah, I think this is a good example of the dogma getting in the way
of common sense. To use Rails and not allow it to generate files and
by extension satisfy it’s own conventions for you just sounds silly to
me.

My opinion is that the goal of testing is to add some degree of
protection to the software you build, not to straight-jacket you as a
programmer.

James Edward G. II

33

James G. wrote:

On Aug 3, 2008, at 12:03 PM, James B. wrote:

common sense. To use Rails and not allow it to generate files and by
extension satisfy it’s own conventions for you just sounds silly to me.

My opinion is that the goal of testing is to add some degree of
protection to the software you build, not to straight-jacket you as a
programmer.

I’ve been wondering about resistance to “purist” TDD/BDD, and if there’s
some aspect of “listen to your body” to observe. For example, when
exercising or dieting, there are assorted rules or regimes to follow,
but if something is hurting, or if you’ve constant cravings for some
food or another, then maybe that’s an important sign to deviate from the
original plan, and not simply a lack of will power or a failure to “do
it right.”

Rather than people applying ad hoc approaches to Test-second
Development, maybe there are some better practices to be shared for what
seems to be a common situation.

–
James B.

www.happycamperstudios.com - Wicked Cool Coding
www.jamesbritt.com - Playing with Better Toys
www.ruby-doc.org - Ruby Help & Documentation
www.rubystuff.com - The Ruby Store for Ruby Stuff

34

James B. wrote:

That’s exactly what is being talked about.

And the issue is that it creates a model file before the developer has
written a failing test indicating the need for a model file.

So write the failing test, then run script/generate. Manually writing
all the
files that script/generate creates is just heroism.

35

Hi –

On Mon, 4 Aug 2008, James B. wrote:

script/generate model MyModel

decisions, so the TDD/BDD purity thing is sort of a moot point.)
I don’t think anything in Rails militates against a test-driven
approach, though – in fact, by creating test files for you, and
having the default ‘rake’ command be to run your tests, it creates a
very hospitable environment for testing. If you don’t write tests,
you’re always reminded of their (non)existence by the presence of the
files.

David

36

On Aug 3, 2008, at 3:29 PM, James B. wrote:

I’ve been wondering about resistance to “purist” TDD/BDD, and if
there’s some aspect of “listen to your body” to observe. For
example, when exercising or dieting, there are assorted rules or
regimes to follow, but if something is hurting, or if you’ve
constant cravings for some food or another, then maybe that’s an
important sign to deviate from the original plan, and not simply a
lack of will power or a failure to “do it right.”

Yeah, I think you are making a lot of sense here.

For example, I haven’t been able to get into RSpec. I know a lot of
people really like it and it’s gained a lot of popularity recently,
but it’s not for me.

The interface just “feels” wrong to me. To give a random example, I
can’t imagine how I would explain to a programming student something
like, “Now we can check that valid?() method you just wrote using
be_valid().” I would much rather be saying, “OK, let’s call valid?()
a few times and see if we get the results we expect.”

Even if RSpec becomes the one true way to handle Ruby testing, I’m
going to have to respectfully decline to follow suit. I’m glad people
like it; I’m glad it’s bringing new people to testing; I’m glad it’s
encouraging such good practices. It’s just not for me.

James Edward G. II

37

Well, I’m not a professional Ruby developer, but I am someone that
writes code for a living, and I’ve been an IT contractor.

And while I love the idea of TDD, if I were using it professionally
then I simply wouldn’t be able to afford to be that dogmatic. Even if
I always, always, wrote the tests first, sooner or later I would run
up against code written by somewhere else who didn’t – and as a
contractor, of course, I never got to choose what programming and
testing methodologies were being used at all. (“You want me to indent
every other line by four tabs? Okay, you’re paying.”)

I’m not criticising anyone for being gung-ho about DTT; I would simply
advise everyone to learn how to write tests second, as well.

Shadowfirebird.

On Sun, Aug 3, 2008 at 9:38 PM, Phlip [email protected] wrote:

–
Me, I imagine places that I have never seen / The colored lights in
fountains, blue and green / And I imagine places that I will never go
/ Behind these clouds that hang here dark and low
But it’s there when I’m holding you / There when I’m sleeping too /
There when there’s nothing left of me / Hanging out behind the
burned-out factories / Out of reach but leading me / Into the
beautiful sea

38

On Sun, Aug 3, 2008 at 5:15 PM, James G. [email protected]
wrote:

For example, I haven’t been able to get into RSpec. I know a lot of people
really like it and it’s gained a lot of popularity recently, but it’s not
for me.

The interface just “feels” wrong to me. To give a random example, I can’t
imagine how I would explain to a programming student something like, “Now we
can check that valid?() method you just wrote using be_valid().” I would
much rather be saying, “OK, let’s call valid?() a few times and see if we
get the results we expect.”

I like RSpec in a lot of ways, but this stuff annoys me too. The
trouble is the lack of bare assertions and the dogmatic assumption
that introducing them could be evil. But I don’t know, my inner
Rubyist and non-conformist asks me, why shouldn’t I just be able to do
this?

class Thing
def valid?
false
end
end

def assert(thing)
(!!thing).should be_true
end

describe “A thing” do
it “should be valid” do
@thing = Thing.new
assert @thing.valid?
end
end

It’s not like this code is hard to follow, but it’s annoying that I
need to create a custom hack to support the most basic thing (a simple
assert).

But hacking in some less magic alternatives to RSpec isn’t exactly
appealing. Things like Shoulda and test/spec and friends already do
that. What I’d really want in Test::Unit to make it feel like a good
solution ‘out of the box’ is:

  • Some standard mocking library - Mocha is very pretty IMO, but any
    would do

  • Full Test::Unit compatibility to play nice with the massive amount
    of pre-existing tests out there

  • Flexibility to extend the framework to go in the direction I want
    it to. I’m personally quite sick of people telling me that I’m doing
    something wrong because it doesn’t read like English, or because I use
    a setup method that creates a ‘dependency’ between the tests. Though
    these are valid conversations to have, people need to stop acting like
    there are established laws, and should give us the freedom to bend our
    tools to our work flow, not the other way around.

  • The ability to specify my test cases as strings rather than methods
    like test_something_should_do_something

  • The ability to have contexts and nested contexts.

  • Nice built in extras to do pretty HTML reports, coverage reports, etc.

I know there are some efforts heading in this direction, but I feel
like standard Ruby should really ship with something that accomplishes
all of the above. I don’t hate BDD style syntax, but it’s the last
reason why I use RSpec. The lack of compatibility with other testing
tools creates a divide that I don’t like, too.

It mostly seems that people who are accustomed to Test::Unit are more
tolerant and willing to write specs if necessary, but that BDD
advocates who are convinced they’ve learned “The O. True Way” of
doing things really have a strong aversion to going back to xunit
style syntax. Seems like a mental hangup to me.

Anyway, what I’ve said applies only to a small percentage of people
I’ve encountered in the community, but they have seemed to be quite
vocal.

I am mostly thinking that RSpec owes a huge chunk of its success to
its glittering side-features, and not the BDD syntax magic. I wonder
if others feel the same way.

-greg

39

Hi –

On Mon, 4 Aug 2008, James B. wrote:

I’ve been wondering about resistance to “purist” TDD/BDD, and if there’s some
aspect of “listen to your body” to observe. For example, when exercising or
dieting, there are assorted rules or regimes to follow, but if something is
hurting, or if you’ve constant cravings for some food or another, then maybe
that’s an important sign to deviate from the original plan, and not simply a
lack of will power or a failure to “do it right.”

I definitely think so. We’re all (in theory) discerning, careful
practioners who are constantly monitoring the conditions under which
we’re working, dealing with exceptional situations, re-evaluating our
technologies, and so forth. All you’re saying, as I read it, is that
we should not stop behaving like this, just because the technology in
question is test-related. That makes perfect sense to me. I never
signed on to switch off my critical faculties for test-writing
purposes.

Rather than people applying ad hoc approaches to Test-second Development,
maybe there are some better practices to be shared for what seems to be a
common situation.

It’s common but I think it’s also probably always going to involve
some ad hoc thinking, since the circumstances under which one might be
retro-fitting tests can vary so widely. I like the idea of thinking it
through systematically, though (and I don’t claim to have done so at
this stage).

David

40

On Aug 4, 2008, at 9:07 AM, Dave B. wrote:

I like to write my code first, tests second. Otherwise how do I know
what I’m supposed to be testing? (There’s no spec, I make things up
as I
go along. This is a hobby, not work.)

I actually prefer to do the tests first. This let’s me think through
a question like, “How do I want to use this code?” I find that helps
prevent me from designing awful interfaces.

At times though, I will write some code and then cover it in tests.
I’m not a super dogmatic person.

James Edward G. II