Test doubles: expect "x" and don't care about anything else

I’ve had one of my recurring doubts about test doubles come up again.

The full post is here but I’ll abbreviate the content in this message
in any case:

Thinking about switching to RR · wincent.com

Basically, in one of my controller specs I wanted to verify that the
following line was being called and doing the right thing:
@comment = Comment.find params[:id]
I had a mock for this set up, but it broke when unrelated code in the
model was modified (a complex callback which itself called
Comment.find).

The problems were as follows:

  • A mock was more than I really needed, as I didn’t want to go
    through the complication of returning a substitute object.

  • The expectation set on the mock was too strict, because the other
    message send to Comment.find, the one I didn’t care about, was
    triggering a failure.

  • A proxy would suffice, because all I really wanted to confirm was
    that the “find” message was sent, without actually interfering with
    the returned object.

  • I basically wanted to set an expectation “that this class will
    receive this message with these params”, but the frameworks didn’t
    allow me to do that because in reality you can only assert “that this
    class will receive this message with these params and not receive
    that message with any other params at any time

In my blog post I detailed the possible options for avoiding the
problem, and the easiest ended up being: forget mocks and proxies
entirely and instead test the side-effect (that the expected object
ends up getting assigned to the “@comment” instance variable).

So the workaround worked, but RSpec’s own mock framework, and from
what I can tell, the alternatives such as Mocha, RR et al, wouldn’t
really let me make the kind of assertion that I wanted to make: ie.
“confirm this message gets sent at some point, but don’t modify the
behaviour at all, and don’t interfere with or worry about any other
messages that get sent to the object, including messages sent to the
method that I’m setting the expectation on”.

In my ideal test-double framework, I’d like to really assert two
things about the line of code in question:

  1. That Comment.find gets called with a specific param at some
    point in time.
  2. That the @comment instance variable gets the expected value
    assigned to it.

I literally don’t care about what other messages get sent to Comment,
nor about other places in the code where Comment.find might get called
with other parameters, and in any case I don’t want to actually modify
the behaviour or substitute my own return values. But it seems I can’t
do this with existing test double frameworks, and it makes it hard to
write minimal specs with one expectation and as few test doubles as
possible (ideally zero or one) per “it” block.

Ideally I’d want to write something like:

it ‘should find the comment’ do
proxy(Comment).find(@comment.id.to_s)
do_put
end

it ‘should assign to the @comment instance variable’ do
assigns[:comment].should == @comment
do_put
end

Note that with the “proxy” syntax above I’m trying to say:

  • I expect this message and these params to be sent at some point

  • I don’t care if other messages are sent

  • I don’t even care if the “find” method is also called with
    different params

  • I don’t care about the order of the messages

  • I don’t want to interfere with or substitute the return value

I don’t know whether the syntax is adequate, or whether some keyword
other than “proxy” would be required. Another alternative I thought of
was:

it ‘should find the comment’ do
spy(Comment)
do_put
Comment.should have_received.find(@comment.id.to_s)
end

Or similar… Basically saying that I want the double framework to spy
(proxy and record) all messages to the specified receiver, and that
afterwards I’m going to retrospectively check that among the recorded
messages is the one I’m looking for.

What do other people think?

  • is what I’m wanting to do a reasonable approach?

  • are there any test double frameworks out there which would allow
    me to work in this way?

Cheers,
Wincent

On 28 Jun 2009, at 13:07, Wincent C. wrote:

I had a mock for this set up, but it broke when unrelated code in
the model was modified (a complex callback which itself called
Comment.find).

I’d like to know more about how this happened. How did the model
object’s behaviour leak into the controller spec?

that the “find” message was sent, without actually interfering with
entirely and instead test the side-effect (that the expected object
In my ideal test-double framework, I’d like to really assert two
things about the line of code in question:

  1. That Comment.find gets called with a specific param at some
    point in time.
  2. That the @comment instance variable gets the expected value
    assigned to it.

So why not use

Comment.stub!(:find).with(123).and_return(mock(Comment))

it ‘should find the comment’ do

Or similar… Basically saying that I want the double framework to

Cheers,
Wincent

cheers,
Matt W.

+447974 430184

El 28/6/2009, a las 23:04, Matt W.
escribió:

the following line was being called and doing the right thing:
@comment = Comment.find params[:id]
I had a mock for this set up, but it broke when unrelated code in
the model was modified (a complex callback which itself called
Comment.find).

I’d like to know more about how this happened. How did the model
object’s behaviour leak into the controller spec?

This was a spec for the controller’s “update” action, which does a
“save” on the record. At one point a change was made to the model to
do some complex updates in the after_save callback, and these involved
doing another Comment.find call, but with different parameters.

Comment.stub!(:find).with(123).and_return(mock(Comment))
Because there are actually two “find” calls here:

  • the one I actually care about
  • the other one in the after_save callback which is irrelevant to the
    controller

I original used “should_receive”, not “stub”, so RSpec complained
about getting “find” with the unexpected parameters. If I change to
“stub” then I’m losing my assertion (no longer checking that the
message gets sent), injecting a different return value (adding
complexity), for no visible benefit (may as well just throw away the
expectation).

Cheers,
Wincent

On 28 Jun 2009, at 23:02, Wincent C. wrote:

This was a spec for the controller’s “update” action, which does a
“save” on the record. At one point a change was made to the model to
do some complex updates in the after_save callback, and these
involved doing another Comment.find call, but with different
parameters.

If I understand this correctly, there was only one call from
Controller → Comment that you wanted to test; the other one was a
call from Comment → Comment that happened as a side-effect.

So I’m wondering: if you’d returned a fake (mock, stub, whatever)
comment from your stubbed Comment.find, would that have solved the
problem?

So why not use
about getting “find” with the unexpected parameters. If I change to
“stub” then I’m losing my assertion (no longer checking that the
message gets sent), injecting a different return value (adding
complexity), for no visible benefit (may as well just throw away the
expectation).

What I often do is put a stub in first, which will work in all the
examples, then put a should_receive in one of the examples if (as
seems to be the case here) it’s important to me to test the
collaboration between the objects. So it would look like this:

describe “#update” do
before(:each)
@comment = mock(Comment)
Comment.stub!(:find).and_return(@comment)
end

it “should call the model to try and find the comment”
Comment.should_receive(:find).with(123).and_return(@comment)
do_request
end

it “should assign the comment to the view”
do_request
assigns[:comment].should == @comment
end

So the stub works in the background, then when you want to actually
assert for the collaboration, you can override it with a
should_receive. I find this pattern works really well for me.

cheers,
Matt W.

+447974 430184

El 29/6/2009, a las 9:33, Matt W.
escribió:

object’s behaviour leak into the controller spec?

This was a spec for the controller’s “update” action, which does a
“save” on the record. At one point a change was made to the model
to do some complex updates in the after_save callback, and these
involved doing another Comment.find call, but with different
parameters.

If I understand this correctly, there was only one call from
Controller -> Comment that you wanted to test; the other one was a
call from Comment -> Comment that happened as a side-effect.

Exactly.

So I’m wondering: if you’d returned a fake (mock, stub, whatever)
comment from your stubbed Comment.find, would that have solved the
problem?

Yes, but there’s something about that that felt somehow harder than it
should be.

I mean, I had a working, dead-simple controller spec. I made an
unrelated change in the model, and the controller spec broke.
Returning a fake would certainly fix the breakage, but the spec would
no longer be dead-simple and it somehow feels like one step foward,
two steps back, that due to innocuous model changes I have to increase
the complexity of my controller tests.

So what I’m really pining for is the ability to say “expect this
message, but don’t return fakes – just proxy the message through and
return the real return value – and don’t worry about any other
messages”.

Cheers,
Wincent

Matt W. wrote:

object’s behaviour leak into the controller spec?
So I’m wondering: if you’d returned a fake (mock, stub, whatever)

  1. That the @comment instance variable gets the expected value
    controller
    seems to be the case here) it’s important to me to test the
    do_request
    end

You probably know this, but for the benefit of others… Pat made a
change a while back that makes it so the stubbed return value will still
be returned even if an expectation is added. Meaning, assuming the stub
is in the before block, you can change the expectation to:
Comment.should_receive(:find).with(123) # this will still return
@comment

-Ben

El 29/6/2009, a las 16:26, Ben M.
escribió:

You probably know this, but for the benefit of others… Pat made a
change a while back that makes it so the stubbed return value will
still be returned even if an expectation is added. Meaning,
assuming the stub is in the before block, you can change the
expectation to:
Comment.should_receive(:find).with(123) # this will still return
@comment

I didn’t know that, but it’s pretty awesome. Basically means that
RSpec mocks can double as proxies now. That’s pretty neat.

Cheers,
Wincent

El 29/6/2009, a las 17:15, Wincent C.
escribió:

RSpec mocks can double as proxies now. That’s pretty neat.
Er, I stand corrected. I went back looking for the change and found it
(commit 72facc08), and then I re-read your description. Forget what I
said about proxying.

Cheers,
Wincent