So… In the recent five-hundred-and-forty-seventh iteration of the
“What’s the point of mocks?” discussion on this list, David C.
ended a message with:
[ . . . ] Really, I think
that’s what we should all be striving for. Not so much “should I use
mocks or not,” but “when should I choose to use them?”
And here… Here you made me really pause and think.
David: Now that you’re a Pragmatic author – have you read Andy’s
Pragmatic Thinking and Learning? Damn good book. The most valuable
part of it to me was in the first couple of chapters, which laid out
the Dreyfus model of skill acquisition. Having learned about it a few
months ago, I’m starting to see it everywhere now. I don’t intend to
rehash the whole theory, but it basically says that the development of
a skill (any skill) follows a predictable pattern:
- Novice (needs clear checklists or recipes; not ready for theory or
problem-solving) - Advanced Beginner (can solve typical problems and alter the recipe
within parameters) - Competent (understands the theory behind the recipes and can act
creatively) - Proficient (acts on principles rather than recipes; using the skill
is nearly automatic) - Expert (defines and innovates the field; works on intuition rather
than rules)
From reading a fair amount on TDD/BDD – including the Rspec Book – I
think there’s widespread consensus that learning TDD/BDD is a totally
separate skill from programming itself. One can be an expert
programmer with decades of experience, but still be at zero when it
comes to writing and using tests. (Or even an “anti-learner,” hostile
to the concept itself.)
My thesis in this message is that there is a hurdle built into
learning the skill of TDD/BDD with RSpec: that there’s a steep curve
somewhere in the early part of stage 2 – “Advanced Beginner” – that
makes it more difficult than it ought to be to ascend to competence.
The hurdle is that some easy things are easy, but other easy things
are very complicated. Not just hard to do, but hard to learn. Hard
to understand why stuff works.
The tasks where the curve begins can be identified by following the
high-level trends of conversations on this list. Usually it’s on
Rails. Models are easy enough; heck, models are fun. But then we try
to do controllers, and… Boom. Brick wall. You can run the
generator, but… What is it doing? How do you change it? Let’s say
I’m a novice to Rails, too: what the hell is that routing spec?
That last sentence is the sharpest way I can think to phrase the
problem. “What the hell is that spec?” Rails is easy to start
rolling with for the default use cases, but moderately hard to
understand when you’re ready to do more complex things. Its nuances
are scattered in a lot of places, and some of the connections are
invisible. RSpec on Rails, even in the default cases, exposes more of
those nuances and requires you to understand them. ActiveRecord
stands on its own, so models are fine (and views are mostly just
tedious) but you can’t even begin to understand that generated
controller spec until you understand Rails and all of its nuances
better.
And if you’re trying to learn both at once? And you don’t want to go
deeper into your Rails code until your specs pass? …Learning
deadlock. The failure to understand one keeps you from making
progress with the other. In the ideal case the specs would help to
teach you Rails, but in practice it doesn’t work that way. They specs
are too opaque without Rails knowledge.
When I set out to consciously learn this stuff, I was determined to
really learn it. I didn’t want to type any code I couldn’t
understand. So I ran the RSpec scaffold in a ‘dummy’ app and used it
only as a reference to build my own by hand. The first controller
spec I tried was for a nested resource with parent and child objects.
(Bad choice perhaps.) It took me almost two days to write that first
full spec with all the mocks and stubs, and get it to pass on every
action. I was on contract and even to me that felt hard to justify.
And then I had to deal with authentication and filters, and then I
felt the pain again.
I’m a stubborn ass, so I plodded through. Eventually I even figured
out what I was doing. (I think.) But I believe I have seen people
run into this curve and view it as a brick wall. Many give up,
sometimes concluding that RSpec is beyond them – after all, everyone
says it’s “easy,” so if they can’t write a controller spec quickly or
see the immediate value of this stuff, they must be too dumb for it.
It isn’t just Rails either. That’s the most common entry point, and
controllers seem to be the most common wall. But I hit a wall again
when I needed to write an adapter for a complex SOAP interface,
building some abstractions on top of soap4r. “Where do I start?” was
quite hard for me. I fished around for blog posts and examples to
give me some precedent. Eventually I figured out a reasonably
efficient way to mock out the SOAP responses without building a whole
fake server, and I think I’m better for the exercise. But as soon as
I got beyond “this is all my code, so I understand it,” I hit that
curve again.
…This is a real problem. It’s not entirely the fault of RSpec (in
which I include RSpec-Rails) as a framework. Some of it’s built into
the ideology: some types of systems really are hard to specify or
interact with, and we don’t acknowledge that enough. Or we say “If
it’s painful to specify you’re doing it wrong.” Not always true, and
a dangerous statement for a novice who’s only feeling the sorts of
pain everyone feels when they first begin exercising.
Some of it’s built into Rails and “Rails-like” MVC frameworks This is
off the topic here, but I can’t shake the feeling that if DHH had had
RSpec from the very start and applied some of its philosophies
faithfully, Rails would be different in some fundamental ways.
Instead, some things are coupled in ways that require RSpec-Rails to
jump through some weird hoops. It all works, but it isn’t all
simple. And beginners can’t grok beyond simple.
Some of it’s the fault of the community. The problems I see here are
subtle and certainly not malignant. This list, for instance, is
excellent for its purpose – it has probably the highest
signal-to-noise ratio of any active technical list I follow, and
everyone’s eager to help at every level. I’ve never seen anyone
sniped at for asking beginner questions. That’s great. But the
questions and the answers are both coming from such a broad range of
skill levels, and sometimes that makes for a disconnect that’s tough
for learning.
That’s what I saw in the quote above that triggered this weird polemic
from me. In the Dreyfus model, there’s no question that many people
here would be classed as Expert – almost by definition in David’s or
Aslak’s case. Me, I see myself on the level of “mid-to-high
Competent, struggling toward Proficient.” Whether that’s accurate or
not, I’d be the wrong person to say. Studies show that most
incompetent people are very bad at gauging their incompetence, and I
might be one of them. Studies also show that Experts are generally
not good at teaching Novices, because they no longer think in terms
of rules and recipes. Making an Expert follow a recipe is terribly
frustrating and a waste of everyone’s time. OTOH telling Novice “use
your own judgment” is even more frustrating – and may result in the
loss of that Novice. The Novice doesn’t have his or her own
judgment yet.
When we say things like, “I think that’s what we should be striving
for – not ‘should I use mocks or not,’ but ‘when should I choose to
use them?’” …That problem is, if we buy into that Dreyfus
hierarchy, that is a harmful statement to deliver to an audience
that’s broadly distributed along the skill curve. The question of
“When should I use X?” is an appropriate question for someone at
Competent or Proficient levels. But it would be very challenging for
most Advanced Beginners to suggest an answer, and a Novice wouldn’t be
able to answer it at all. (As for Experts: they probably don’t even
need to think about it. They just know.)
We need more awareness of this. It’s not a bad thing, and I think
there are probably more benefits in having a skill-level-distributed
list like this than to set up some sort of “beginner’s” or “learner’s”
list or anything. (For one thing, most programmers who’d benefit
would have too much ego to join it.) We just need to recognize – and
I include myself, as I spun off the theory tangent in that last
thread – that a Novice question needs a Novice or Advanced Beginner
answer, and that saying “It’s really more complicated than that” isn’t
helpful to certain classes of question.
Community-wise: I also think we need better learning resources. The
RSpec Book is great. Followed faithfully, I have a hunch it can bring
an Advanced Beginner up to Competent with remarkable efficiency. I
think it could work as well for some Novices. But not all. It
depends on their thinking patterns, and whether they need or want to
be grounded in theory before they get going with the recipes. The
Mastermind puzzle is a good starting exercise, but there’s too much
explanation going on before the rest of the practical reference
begins. (Again, for some people. This was my impression. For
others it might be just right.)
The RSpec.info site is…problematic. It’s quirky in its information
structure, and I’ve gotten lost sometimes clicking around to find
something I knew was there. I don’t feel comfortable pointing
people to it as a start-from-zero starting point. The explanations
are clear and up-to-date, but it’s hard to find a handhold. The
Peepcode videos worked better for me, though they were a bit too long
for my ADD-driven brain.
At one point I was going to write a “Getting Started With RSpec” sort
of primer. It was going to be informed by my own growth curve, and
truly start with “Step by step, okay, just do this.” No generators,
write the code by hand, but get actual specs working quickly in actual
projects.
Of course, my “do this” would be somewhat different from David’s –
I’ve been vocal enough about doing a lot of my tests a different way
– so I felt a bit awkward about breaking from orthodoxy when I don’t
feel like an expert myself. But I felt bold. I started it, got a
little distracted and overplanned the demo app (a wiki to host the
primer itself, but I wanted some new sorts of features) and it’s
languishing. This thread makes me wonder if I should put a little
more foreground time into it. Not the stupid wiki, but the document.
I’m rambling now, and I’ve spent way too much of my workday on this
message, so I’ll just summarize:
- Some easy things are complicated in RSpec;
- Incomplete understanding of RSpec and a framework being spec’ed can
create learning deadlock; - Novices need different answers than competent practitioners;
- The community as a whole should be aware and respectful of the
growth curve difficulty.
If anyone had the patience to read this far, any thoughts? Am I
identifying real issues, or am I just full of hot air?
–
Have Fun,
Steve E. ([email protected])
ESCAPE POD - The Science Fiction Podcast Magazine
http://www.escapepod.org