Testing .Net Newbie

Hi,

I’ve been a .Net developer for a number of years now and I’m a big fan
of TDD / BDD. I have been following these principles for a couple of
years and use tools such as NUnit for testing purposes.

I am now looking at moving across to using RoR for a number of projects
and I’m still getting my head around how everything works. I’ve been
reading up on RSpec and Cucumber and I have one question looming in my
head. (This may be my .Net head messing things up for me).

Normally in .Net when TDDing a controller class I would mock out other
library classes that the controller calls, such as Repository layer
classes. However I notice when using Cucumber people very rarely mock
anything and I’m assuming this is because Cucumber is meant for testing
the full stack (Integration testing). I’d normally do this with
something like Selenium in .Net.

My question is, when creating a controller (or other class) that relies
on other library classes is it best to just use RSpec rather than
Cucumber to mock out the other dependencies and then bring in Cucumber
scenarios later when you know your RSpec tests are passing?

Thanks

John

On Thu, Jan 14, 2010 at 5:46 AM, John P. [email protected]
wrote:

Normally in .Net when TDDing a controller class I would mock out other
library classes that the controller calls, such as Repository layer
classes. However I notice when using Cucumber people very rarely mock
anything and I’m assuming this is because Cucumber is meant for testing
the full stack (Integration testing).

Cucumber is intended for Acceptance Testing. The entry point into the
system
can vary. With Rails you can go through the browser (driving Selenium,
Watir, etc from Cucumber), start at the router, or even go directly to
the
model in cases where that’s appropriate.

re: Integration testing, everybody has a different definition. Before
Rails
came along, the prevalent definition that I was aware of was “testing
the
behaviour of two non-trivial components together.”

More recently, the definition that makes most sense to me comes from
Growing
Object Oriented Software [1]. I don’t have the book in front of me, but
from
memory it is something like “testing your code with other code that you
don’t have any control over.” Because we need a database for all levels
of
Rails testing, this suggests that all Rails testing is Integration
Testing.

In Rails’ terminology, however, “integration testing” means very
specifically testing the full stack minus the browser.

I’d normally do this with something like Selenium in .Net.

My question is, when creating a controller (or other class) that relies
on other library classes is it best to just use RSpec rather than
Cucumber to mock out the other dependencies and then bring in Cucumber
scenarios later when you know your RSpec tests are passing?

“is it best?” is always relative. YMMV. That said, the outside-in
process of
BDD using Cucumber and RSpec together works well like this (borrowed and
modified from the 2nd post in
http://groups.google.com/group/cukes/browse_thread/thread/2598d6fcd29bfd86):

  1. Identify business need (the very outside)
  2. Discuss what feature would solve this need (feature injection)
  3. Write a Cucumber scenario that interacts with the system boundaries
    (that
    have yet to be written)
  4. Run the scenario and watch it fail because the code doesn’t exist yet
  5. Write a little piece of the system boundary code that was missing and
    caused the test to fail
  6. Run the scenario and watch it fail again, this time because the
    code doesn’t do what it needs to yet
  7. Write a code example in RSpec
  8. Run the code example and watch it fail
  9. Write enough code to make it pass
  10. Run the scenario and check its status
  11. Repeat 7-10 until you have enough code (and not more than just
    enough)
    to make the scenario pass.

One catch-phrase that comes to mind is “use Cucumber to learn what code
to
write, use RSpec to write the right code.”

You should know that there is a trend these days among
Cucumber/RSpec/Rails
users to not write controller specs on the basis that controllers are
covered by the Cucumber scenarios. The benefit is a faster suite that
appears to have less duplication between testing layers. The drawback is
that you lose the benefit of discovering model interfaces when you’re
specifying the controllers in isolation.

My view is that this makes sense for the boilerplate stuff, but
customizations of the controller are well served with isolated specs in
RSpec.

More answer than you bargained for, I imagine.

HTH,
David

[1]
http://www.amazon.com/Growing-Object-Oriented-Software-Guided-Tests/dp/0321503627

Thanks for this David.

I suppose the thing I am so used to is testing classes in isolation
whereas Cucumber is about testing everything together.

Do Rails developers generally not test things in isolation using mocking
and stubbing?

John

On Thu, Jan 14, 2010 at 6:33 AM, John P. [email protected]
wrote:

Thanks for this David.

I suppose the thing I am so used to is testing classes in isolation
whereas Cucumber is about testing everything together.

Did you do all of your testing in NUnit?

When I was working on .NET apps, I used FitNesse for acceptance testing
and
NUnit for isolation testing. With Rails, these same roles are played by
Cucumber and RSpec.

Do Rails developers generally not test things in isolation using
mocking and stubbing?

Some do all the time, some never do. I probably do more than most, but
I’ve
not experienced a lot of the pain that folks who are new to mocks and
stubs
experience. You’ve got to know when it’s appropriate.

HTH,
David

ps - please be sure to quote the relevant portions of the email you’re
responding to so people reading email on their phones don’t have to go
digging around for context.

David C. wrote:

Did you do all of your testing in NUnit?

When I was working on .NET apps, I used FitNesse for acceptance testing
and
NUnit for isolation testing. With Rails, these same roles are played by
Cucumber and RSpec.

Generally what I do is drive all my individual classes out using NUnit
and Rhino.Mocks and then do the acceptance testing later with FitNesse.

I think this is the part that I’m confusing myself with as most Cucumber
information talks about using scenarios to drive the code out. So
Cucumber comes first, whereas I used to do the Acceptance testing after
all the other TDD stuff. Maybe that is me doing it wrong though.

John

On 14 Jan 2010, at 12:33, John P. wrote:

Do Rails developers generally not test things in isolation using mocking
and stubbing?

Yes, that’s what RSpec is for.

John P. wrote:

Generally what I do is drive all my individual classes out using NUnit
and Rhino.Mocks and then do the acceptance testing later with FitNesse.

I think this is the part that I’m confusing myself with as most Cucumber
information talks about using scenarios to drive the code out. So
Cucumber comes first, whereas I used to do the Acceptance testing after
all the other TDD stuff. Maybe that is me doing it wrong though.

John

Hi John. Maybe this will help:

It’s a good book. But when you get to the part about Webrat and
Selenium, just know that there are other options if you need them.

Peace,
Phillip

On Jan 14, 2010, at 8:46 AM, Phillip K. wrote:

after
all the other TDD stuff. Maybe that is me doing it wrong though.

John

Hi John. Maybe this will help:

Pragmatic Bookshelf: By Developers, For Developers

D’oh! I should have thought of that! Thanks for the plug :slight_smile:

Cheers,
David

Phillip K. wrote:

Hi John. Maybe this will help:

Pragmatic Bookshelf: By Developers, For Developers

It’s a good book. But when you get to the part about Webrat and
Selenium, just know that there are other options if you need them.

Already reading that one thanks :slight_smile: It is a good book and it’s helping
me get my head around it all. It’s just a case of understanding when to
develop using Cucumber and when to user RSpec on it’s own. I think this
will come with experience.

John

hi

2010/1/14 John P. [email protected]

me get my head around it all. It’s just a case of understanding when to
develop using Cucumber and when to user RSpec on it’s own. I think this
will come with experience.

there’s also this very good presentation by Ben M. about when to use
which (cucumber or rspec) with the analagy with bikes and bike-gears,
only I
don’t find the link :frowning:

hth (a bit),
joaquin

John P. wrote:

Already reading that one thanks :slight_smile: It is a good book and it’s helping
me get my head around it all. It’s just a case of understanding when to
develop using Cucumber and when to user RSpec on it’s own. I think this
will come with experience.

John

Others can explain it better than me (and have), but in a nutshell:

  • use Cucumber for full stack (as the user would see it) testing. Some
    call it User Acceptance Testing (UAT)
  • use RSpec for unit/functional testing (controllers, models, views,
    helpers)

I think of it like this: RSpec is for the developer, Cucumber is for the
user. That may be a gross oversimplification, but it works for me. :slight_smile:
But one thing to keep in mind is that you can (and maybe should) use
them together. Through Cucumber, you establish the context of what the
user will expect. Through RSpec, you establish expectations for the
different pieces of the application. That’s why it’s called “Outside In
Development”: You start with the outside of the black box (what the user
interacts with) and then step into the black box (the nuts and bolts
that the user doesn’t care about, but you, the developer, do).

Yes, it will come with experience. Just be patient, learn from your
mistakes, and ask lots of questions! That you have testing knowledge
already will help. Once the RSpec/Cucumber light goes on, there will be
no stopping you!

Peace,
Phillip

On Thu, Jan 14, 2010 at 9:26 AM, Joaquin Rivera P.
[email protected]wrote:

It’s a good book. But when you get to the part about Webrat and
don’t find the link :frowning:

http://mwrc2009.confreaks.com/14-mar-2009-15-00-bdd-with-cucumber-ben-mabey.html

He leaves out a couple of steps here and there for the sake of time in
the
presentation, but this gives a great overview.

Cheers,
David

On Jan 14, 2010, at 12:15 pm, David C. wrote:

re: Integration testing, everybody has a different definition. Before Rails came along, the prevalent definition that I was aware of was “testing the behaviour of two non-trivial components together.”

More recently, the definition that makes most sense to me comes from Growing Object Oriented Software [1]. I don’t have the book in front of me, but from memory it is something like “testing your code with other code that you don’t have any control over.” Because we need a database for all levels of Rails testing, this suggests that all Rails testing is Integration Testing.

And yet, JB Rainsberger sticks absolutely uncompromisingly to the first
definition[1]. I wonder if you could take it to yet another extreme and
include tests for objects with private methods as “integration tests”.

The thing I got most from GOOS is to protect all domain code with
adapters. If all Rails testing is integration testing, that means a lot
of duplication and coupling that could be reduced… I have yet to do it
for real though. But GOOS is the book that convinced me it’s worth
trying.

Ashley

[1] Integration Tests Are a Scam


http://www.patchspace.co.uk/
http://www.linkedin.com/in/ashleymoran

hey there
there’s also http://bddcasts.com with railscasts like bdd screencasts

haven’t check it much myself, but might deserve a look

hth,
joaquin

2010/1/14 Ashley M. [email protected]

On Jan 14, 2010, at 1:17 pm, John P. wrote:

I think this is the part that I’m confusing myself with as most Cucumber
information talks about using scenarios to drive the code out. So
Cucumber comes first, whereas I used to do the Acceptance testing after
all the other TDD stuff. Maybe that is me doing it wrong though.

The book David mentioned earlier - before the one he wrote himself :slight_smile: -
GOOS[1] has a diagram on p40 that shows how the unit test cycle fits
inside the acceptance test cycle. You can extend this to as many layers
as you like. I wrote a blog post which has a pretty picture of the
idea[2]. Ignore the text, it’s long and rambling and is about something
else. I thought it was a great insight, until I realised Kent Beck
wrote about self-similarity something like 5+ years ago[3].

Ashley

[1]
http://www.amazon.com/Growing-Object-Oriented-Software-Guided-Tests/dp/0321503627
[2]
http://blog.patchspace.co.uk/2009/12/customers-input-and-russian-doll-of.html
[3]


http://www.patchspace.co.uk/
http://www.linkedin.com/in/ashleymoran

Good explanation Phillip. It gives a clear idea…

Thanks

Regards,
Diwakar

Others can explain it better than me (and have), but in a nutshell:

  • use Cucumber for full stack (as the user would see it) testing. Some
    call it User Acceptance Testing (UAT)
  • use RSpec for unit/functional testing (controllers, models, views,
    helpers)

I think of it like this: RSpec is for the developer, Cucumber is for the
user. That may be a gross oversimplification, but it works for me. :slight_smile:
But one thing to keep in mind is that you can (and maybe should) use
them together. Through Cucumber, you establish the context of what the
user will expect. Through RSpec, you establish expectations for the
different pieces of the application. That’s why it’s called “Outside In
Development”: You start with the outside of the black box (what the user
interacts with) and then step into the black box (the nuts and bolts
that the user doesn’t care about, but you, the developer, do).

Yes, it will come with experience. Just be patient, learn from your
mistakes, and ask lots of questions! That you have testing knowledge
already will help. Once the RSpec/Cucumber light goes on, there will be
no stopping you!

Peace,
Phillip