Ordering dependency in tests?

I’m baffled. If I do:

$ bundle exec ruby -S rspec --tty A_spec.rb
$ bundle exec ruby -S rspec --tty B_spec.rb

I get no errors. But then if I do:

$ bundle exec ruby -S rspec --tty A_spec.rb B_spec.rb

I get an error on B_spec. And if I reverse the order:

$ bundle exec ruby -S rspec --tty B_spec.rb A_spec.rb

I get an entirely different error on A_spec. I swear that I’m not doing
any before(:all) anywhere, but clearly there’s some state that is
persisting between the two spec files. I’ve tried inserting
$stderr.puts() messages to gain some insight as to what’s happening, but
they seem to be suppressed (is that expected?).

So: any ideas of gotchas to look out for?

In the meantime, I’m going to start commenting out blocks of tests in
A_spec and see when B_spec stops failing.

On Feb 24, 2011, at 11:55 AM, Fearless F. wrote:

$ bundle exec ruby -S rspec --tty B_spec.rb A_spec.rb

I get an entirely different error on A_spec.

What are the failures you’re seeing?

A little additional information: every time I run the tests, it creates
a new PremiseGroup model. Shouldn’t the db get rolled back between
tests?

Here is PremiseGroup, listed here in its entirety:

class PremiseGroup < ActiveRecord::Base
has_many :premise_group_members, :dependent => :destroy
has_many :premises, :through => :premise_group_members
RESIDENTIAL = self.create(:name => “residential”)
end

When I put in a debugging statement:

before(:each) do
$stderr.puts(“PremiseGroup count = #{PremiseGroup.count}”
end

… it reports that there are 151 PremiseGroups, then 152 on the next
run, then 153 on the next. Naturally, calling ‘rake db:test:prepare’
resets it, but shouldn’t the database be rolled back between tests
regardless? I’m not 100% sure this is the cause of my problems, but it
smells like it…

On Feb 24, 2011, at 12:36 PM, Fearless F. wrote:

A little additional information: every time I run the tests, it creates
a new PremiseGroup model. Shouldn’t the db get rolled back between
tests?

Here is PremiseGroup, listed here in its entirety:

class PremiseGroup < ActiveRecord::Base
has_many :premise_group_members, :dependent => :destroy
has_many :premises, :through => :premise_group_members
RESIDENTIAL = self.create(:name => “residential”)

^^ this is probably the problem ^^

This line creates a new record in the database every time
premise_group.rb is evaluated by the Ruby interpreter. This happens as
the app is being bootstrapped and outside any transactions controlled by
the Rails testing infrastructure (which RSpec sits on top of).

Options include making that a method instead of a const:

def self.residential
find_or_create_by_name(:name => “residential”)
end

HTH,
David

David C. wrote in post #983675:

On Feb 24, 2011, at 11:55 AM, Fearless F. wrote:
What are the failures you’re seeing?

When running A before B, B’s assertion that:
@common_options[:premise_group].premises.size.should == 3
fails because premise_group.premises.size == 5 (even though only 3
premises have been created).

When running B before A, A’s assertion that:
e1.ranking(premises[0]).should be_within(0.0001).of(0.0)
fails because a call to sort() fails because there is a nil in some list
that ought to be a Float. (I haven’t dug deeper into that one.)

But to reiterate, either tests completes without error when run alone.
Still digging…

  • ff

cattr_accessor, class vars / class instance vars, constants,
globals…all things that maintain state across test runs, and so could
lead to errors like this. I’d start looking there.

Pat

David C. wrote in post #983690:

RESIDENTIAL = self.create(:name => “residential”)
^^ this is probably the problem ^^

Ah! got it. FWIW, I wrote a query to the Rails group several months
ago wondering if this construct was legit and got feedback that it was
legit. Your explanation makes good sense. I’ll change that and see
what happens.

If you don’t hear back from me, assume it’s working! :wink: And thanks in
advance.

  • ff

yet more info…

The two errors appear to be closely related. When A and B are run as
individual
tests,
PremiseGroup#premises
is returning expected values. When run together, premise_group.premises
returns “phantom” premises (which show up as nil values). For test B,
the phantoms causes the count to be wrong. For test A, the nil values
cause nils to show up in a list that is passed to sort().

I’m still baffled, but I must be closer to the answer.

On Feb 24, 2011, at 12:58 PM, Fearless F. wrote:

David C. wrote in post #983690:

RESIDENTIAL = self.create(:name => “residential”)
^^ this is probably the problem ^^

Ah! got it. FWIW, I wrote a query to the Rails group several months
ago wondering if this construct was legit and got feedback that it was
legit.

I think it’s OK as a const with find_or_create, but with find, it is
clearly problematic.

Cheers,
David

SOLVED – it was an AR caching problem. I had Factory code that was
essentially doing:

def make_me_a_premise(opts = {})
opts = MODEL_PREMISE_DEFAULTS.merge(opts)
premise = Factory(:premise)
Factory(:premise_group_member,
:premise => premise,
:premise_group => opts[:premise_group])

premise
end

… which was properly creating a premise <=> premise_group association,
but not informing the in-memory premise_group about the new association.
adding:

opts[:premise_group].reload

fixed everything. @David and @Pat, thanks for helping point the way.

  • ff

[PS: Is there a more Rails-y way do make the association that won’t run
into this problem? And what about the case where the association
carries a value with it – you can’t just do a push.]

@Pat: thanks for the suggestions.

It’s perhaps not only what David suggested – I seem to be suffering
from some cached AR value. I changed my constant declaration to:

RESIDENTIAL = find_or_create_by_name(“residential”)

This keeps the number of instances down to a 1. That’s good. But if I
put this in the setup of my test:

puts(@premise_group.premises.size)
@premise_group.reload
puts(@premise_group.premises.size

I get

8
0

Since my setup code is pushing elements onto @premise_group.premises, I
can see that this would cause trouble when @premise_group is written out
to the DB. My head hurts just thinking about where I need to sprinkle
reload() statements in my code – maybe the coffee will kick in and
clarity will result.