"unit" vs. "functional" specs

Testing RSpec
1

I went back and forth on whether to send this to this list or the
ruby-talk
list, so feel free to tell me to take a hike…

I’m working on a personal project where I’m trying to keep a really high
bar
on quality. One of the ways I’m doing that is to stick very close to the
BDD/TDD cycle as suggested in the RSpec book. However, I’m actually
using
rspec for both the feature-level (BDD) tests and the unit-level (TDD)
tests.
I’m also striving for 100% test coverage using Simplecov.

That said, I just realized that, to enforce that cycle, it’s also
important
that I ask “do my unit tests by themselves provide 100% test coverage?”.
Because, otherwise, it’s easy to get excited and write code to make the
feature past, that isn’t TDD’d.

Anyway, all that to ask: is there some way to configure Simplecov to
emit
code coverage stats for just a subset of all the tests that are running?
That is, I want to set up my rakefile such that it runs all my tests,
but
only reports on code coverage for the unit tests.

Am I going about this with the right mindset? Any suggestions?

2

On 10 Dec 2010, at 22:29, Andrew W. wrote:

I went back and forth on whether to send this to this list or the ruby-talk
list, so feel free to tell me to take a hike…

I’m working on a personal project where I’m trying to keep a really high bar on
quality. One of the ways I’m doing that is to stick very close to the BDD/TDD
cycle as suggested in the RSpec book. However, I’m actually using rspec for both
the feature-level (BDD) tests and the unit-level (TDD) tests. I’m also striving
for 100% test coverage using Simplecov.

That said, I just realized that, to enforce that cycle, it’s also important that
I ask “do my unit tests by themselves provide 100% test coverage?”. Because,
otherwise, it’s easy to get excited and write code to make the feature past, that
isn’t TDD’d.

Anyway, all that to ask: is there some way to configure Simplecov to emit code
coverage stats for just a subset of all the tests that are running? That is, I
want to set up my rakefile such that it runs all my tests, but only reports on
code coverage for the unit tests.

Am I going about this with the right mindset? Any suggestions?

I haven’t used simplecov myself, but I can answer the mindset question.

If I were you, I would want three numbers: the overall coverage from
your unit + functional tests, the coverage from your unit tests alone,
and the coverage from your functional tests alone. As long as you’re
consistently working outside in, and driving each change to the code
with some kind of test I would be happy with that, but it would be
interesting to see where the coverage comes from.

As you learn to trust yourself to do things test-first, you’ll find you
need the code coverage figure less and less, because your conscience
knows it’s 100%. These days, unless I’m rescuing a legacy project, I
only really use code coverage to tell me about unused code that I can
delete.

cheers,
Matt

[email protected]
07974 430184

3

Why don’t functional specs count towards your coverage metric? It sounds
like you’re shooting for 100% unit test coverage – why? If you write
code in order to make your functional specs pass, then that code was
TDD’d…right?

Pat

4

When doing BDD, especially at the start, I found it really easy to
write
more code than you need to, when implementing a particular functional
spec.
I also found it really easy to miss the point where you should move into
the
inner cycle (unit cycle). Using the unit test coverage metric to make
sure
you only write what you need and that all your public method interfaces
are
specified at the unit level seems laudable. Of course 100% test coverage
by
itself doesn’t mean that much, but examining the code you have missed
and
the reasons why your application of BDD failed to cover the code is
instructive.