Mocking/Stubbing behavior question

Testing RSpec
1

I’ve run into some strange behavior in porting my specs from rspec to
rspec2.
I am wondering if I am doing something wrong, or if I’ve misunderstood
something, or if this is some kind of bug.

Look at the specs below: a
Both examples will pass if run singly.
The second one will fail if I run both examples.

Can someone tell me why the second one fails the way it does?

It has to do with the conditional assignment of @my_foo, but I’m not
sure why @stupid_mock is getting disconnected.
My thinking is that the class Bar is returning the same object in both
tests, and that the stub call in the second test is being sent to a
different object.

Still, it did not behave this way in the last version, so I don’t know
what I’m missing.

require ‘spec_helper’

class Foo
end

class Bar
def self.my_foo
@my_foo ||= Foo.new
end
def self.perform
my_foo.do_something
end
end

describe Foo do

before(:each) do
@stupid_mock = double(‘wtf’)
Foo.stub(:new => @stupid_mock)
end

it “passes here” do
@stupid_mock.should_receive(:do_something).and_return(‘value’)
Bar.perform
end

it “fails here” do
@stupid_mock.stub(:do_something => ‘value’)
Bar.perform
# double “wtf” received unexpected message :do_something with (no
args)
end

end

2

On Apr 25, 2011, at 1:36 PM, Matthew Van Horn wrote:

My thinking is that the class Bar is returning the same object in both tests,
and that the stub call in the second test is being sent to a different object.
@my_foo ||= Foo.new
Foo.stub(:new => @stupid_mock)
# double “wtf” received unexpected message :do_something with (no args)
end

end

btw - I just realized, that should be: “describe Bar do”
and, I can solve the problem by doing: Bar.stub(:my_foo => @stupid_mock)
instead of Foo.stub(:new => @stupid_mock) but I really don’t like
stubbing methods on the class I am testing.

– matt

3

Em 25-04-2011 14:58, Matthew Van Horn escreveu:

The second one will fail if I run both examples.
know what I’m missing.
def self.perform
it “passes here” do

btw - I just realized, that should be: “describe Bar do”
and, I can solve the problem by doing: Bar.stub(:my_foo =>
@stupid_mock) instead of Foo.stub(:new => @stupid_mock) but I really
don’t like stubbing methods on the class I am testing.

The problem is that “new” is not a regular method on Ruby. I don’t mind
mocking some methods of the class being tested in some cases.

For instance, I have a class that does lots of calculation based on
numerous input data. It would be impossible to provide all possible
combinations of input data for testing the calculation of some methods.

Consider the following class:

class InterestingCalculator
include Singleton
def a
@a ||= hard_calculation_based_on(@several_inputs)
end

 def b
     return 0 if a < 0
     calculate_using a
 end

end

Then I would write something like:

describe InterestingCalculator do
example “b should be 0 when a < 0” do
InterestingCalculator.instance.stub!(:a).and_return(-10)
InterestingCalculator.instance.b.should == 0
end

example “b should do something great when a >= 0” do
InterestingCalculator.instance.stub!(:a).and_return(10)
InterestingCalculator.instance.b.should == 786
end
end

At least, this makes sense to me…

Best regards,

Rodrigo.

4

On Apr 25, 2011, at 11:30 PM, Rodrigo Rosenfeld R. wrote:

The second one will fail if I run both examples.
class Foo

end
and, I can solve the problem by doing: Bar.stub(:my_foo => @stupid_mock)
instead of Foo.stub(:new => @stupid_mock) but I really don’t like stubbing methods
on the class I am testing.

The problem is that “new” is not a regular method on Ruby. I don’t mind mocking
some methods of the class being tested in some cases.

It doesn’t have to do with the “new” method - if I did:
class Bar def self.my_foo
@my_foo ||= Foo.make_one_of_these
end
…
end

and

Foo.stub(:make_one_of_these => @stupid_mock)

It still would fail.

It is more of a pass-by-value/pass-by-reference issue. Because of the
conditional assignment, the variable @stupid_mock in the failing example
seems to be pointing to a different object than the one in the passing
example, even though under the old rspec, they seemed to be the same.

5

Ok, now I understand what is your issue.
class Fooend
class Bardef self.my_foo @my_foo ||= Foo.newenddef self.perform
my_foo.do_somethingendend
describe Foo do
before(:each) do @stupid_mock = double(‘wtf’) Foo.stub(:new =>
@stupid_mock)endit “passes here” do
@stupid_mock.should_receive(:do_something).and_return(‘value’)
Bar.performendit “fails here” do @stupid_mock.stub(:do_something =>
‘value’) Bar.perform # double “wtf” received unexpected message
:do_something with (no args)endend

The first time Bar.foo is being called, it caches Foo.new. Since you
stubbed Foo.new to return a mock before running Bar.foo for the first
time, this mock is cached inside Bar, so any subsequent call will return
the first created mock object. This obviously won’t happen if you run
only the other spec and that is why it passes in this case.
This is not about passing by reference, but it is about caching
Bar.my_foo.
The only way I can think of RSpec not being affected by this is to
reloading all classes before each example, which would be both
complicate and costly.
You’ll probably have to rethink your testing strategies, avoiding
caching class methods or not mocking something that could affect them,
or initializing them first before running the specs…
Best regards,
Rodrigo.

6

On Apr 26, 2011, at 5:39 PM, Matthew Van Horn wrote:

@my_foo ||= Foo.new
Foo.stub(:new => @stupid_mock)
# double "wtf" received unexpected message :do_something with (no args)

The only way I can think of RSpec not being affected by this is to reloading
all classes before each example, which would be both complicate and costly.

You’ll probably have to rethink your testing strategies, avoiding caching class
methods or not mocking something that could affect them, or initializing them
first before running the specs…

Best regards,

Rodrigo.

The example at the top of this thread fails the same way with
rspec-1.3.1, 1.3.2, and 2.5.0. I’m guessing that this was not the real
example that you saw passing in rspec-1 and failing in rspec-2, in which
case this example probably doesn’t trigger whatever differences you were
seeing between rspec-1 and rspec-2. Are you able to show us the real
code?

I see now the crux of the issue - mocks are implicitly cleared out before each
example, so even though the same object is being returned, RSpec is sneakily
substituting a doppelganger for @stupid_mock.

RSpec is doing nothing sneaky. The code in the before hook assigns a new
double to the @stupid_mock instance variable for each example, but the
my_foo method memoizes the @stupid_mock from the first example at the
class level, so it returns that instance for all subsequent requests in
the same runtime.

This apparently wasn’t the case in the last version, where the instance variable
would continue to be connected to the same object.

before(:each) blocks have been run before each example since they were
first introduced, so this is not accurate. Whatever behavior you were
seeing was not due to RSpec keeping instance variable assignments across
examples.

This might be intended behavior, but it feels like a bug because that memoizing
behavior is so common.

Agreed it is common, but the common approach to stubbing things that are
memoized is to stub the method (in this case my_foo).

Because I can’t set up the double in a before(:all) block due to the above
behavior, the solution is to add:
Bar.instance_variable_set(:@my_foo, nil) # Forcibly clear Bar’s cached
instance
to the before(:each) block.

I’d recommend you just stub the my_foo method instead. You said you
prefer to avoid doing that earlier this thread, but I think it’s a
cleaner solution as it is isolated to the current example. Keep in mind
that if the double ends up the memoized value for my_foo, it will remain
so for the rest of the suite, whereas if you just stub my_foo for the
examples you need the double, it will be limited to those.

HTH,
David

7

On Apr 26, 2011, at 4:02 PM, Rodrigo Rosenfeld R. wrote:

my_foo.do_something

it “passes here” do
end
You’ll probably have to rethink your testing strategies, avoiding caching class
methods or not mocking something that could affect them, or initializing them
first before running the specs…

Best regards,

Rodrigo.

I see now the crux of the issue - mocks are implicitly cleared out
before each example, so even though the same object is being returned,
RSpec is sneakily substituting a doppelganger for @stupid_mock. This
apparently wasn’t the case in the last version, where the instance
variable would continue to be connected to the same object.

This might be intended behavior, but it feels like a bug because that
memoizing behavior is so common.

Because I can’t set up the double in a before(:all) block due to the
above behavior, the solution is to add:
Bar.instance_variable_set(:@my_foo, nil) # Forcibly clear Bar’s
cached instance
to the before(:each) block.

– matt

8

On Apr 27, 2011, at 6:40 AM, David C. wrote:

it “fails here” do
The first time Bar.foo is being called, it caches Foo.new. Since you stubbed
Foo.new to return a mock before running Bar.foo for the first time, this mock is
cached inside Bar, so any subsequent call will return the first created mock
object. This obviously won’t happen if you run only the other spec and that is why
it passes in this case.

The example at the top of this thread fails the same way with rspec-1.3.1,
1.3.2, and 2.5.0. I’m guessing that this was not the real example that you saw
passing in rspec-1 and failing in rspec-2, in which case this example probably
doesn’t trigger whatever differences you were seeing between rspec-1 and rspec-2.
Are you able to show us the real code?

I’ll have to see if I can dig it up later and verify. I’ve refactored
quite a bit along the way of this upgrade, so the tests are not all in
the same place. I am reasonably sure my suite was passing, but I could
be wrong. Going from memory, in the original version, I stubbed the
:do_something method on the double before it got memoized, because I
wanted that as the default behavior except for the one example where I
set the expectation.
Perhaps that is what worked in 1.3.1 and not now.

I see now the crux of the issue - mocks are implicitly cleared out before each
example, so even though the same object is being returned, RSpec is sneakily
substituting a doppelganger for @stupid_mock.

RSpec is doing nothing sneaky. The code in the before hook assigns a new double
to the @stupid_mock instance variable for each example, but the my_foo method
memoizes the @stupid_mock from the first example at the class level, so it returns
that instance for all subsequent requests in the same runtime.

Email is bad for tone - ‘sneakily’ is meant tongue-in-cheek. Although,
it is what came to mind when I tried to fix the problem by doing:
before(:each) do
@stupid_mock ||= double(‘wtf’)

and it still failed. I’m still kind of unclear as to which object is
holding that instance variable (and when), so it seemed weird to me that
it was getting reassigned.

Because I can’t set up the double in a before(:all) block due to the above
behavior, the solution is to add:

Bar.instance_variable_set(:@my_foo, nil) # Forcibly clear Bar's cached

instance

to the before(:each) block.

I’d recommend you just stub the my_foo method instead. You said you prefer to
avoid doing that earlier this thread, but I think it’s a cleaner solution as it is
isolated to the current example. Keep in mind that if the double ends up the
memoized value for my_foo, it will remain so for the rest of the suite, whereas if
you just stub my_foo for the examples you need the double, it will be limited to
those.

Hmm. I guess, I’ll re-think my opinions on unit testing. The isolation
is a good feature, and helps attenuate my fears of stubbing methods of
the object under test leading to ineffective tests down the road.

– matt