Features and form filling - going declarative?

Testing RSpec
1

I’m working on writing features for a wizard. The wizard collects
information from a number of different forms, and you can navigate
through
it in a number of ways. Anyhow one of these forms is a customer form
collecting name, and email.

In the context of the wizard I feel that the following scenarios

Scenario:
Given I step to customer
And I fill in my customer details correctly

Scenario:
Given I step to customer
And I fill in my customer details incorrectly
Then I should see an error

are preferable to

Scenario:
Given I step to customer
And I fill in email with [email protected]
And I fill in name “Fred Bloggs”

Given I step to customer
   And I fill in email with ''
   And I fill in name "Fred Bloggs"
 Then I should see an error

# add table for different combinations of form fields that cause

errors
# consider checking that errors are appropriate

note: Given I step to customer is nested step doing all sorts to get to
the
form

What do you think?

I’m looking for some input on this, and in particular am wondering where
should I put the more specific tests for form validation, error messages
etc. in my test hierarchy, or even if I should test them at all (could
you
argue they’re in built rails functionality).

All best

Andrew

2

Andrew P. wrote:

   And I fill in name "Fred Bloggs"

note: Given I step to customer is nested step doing all sorts to get
to the form

What do you think?

I’m not sure if you where just giving examples but its always a good
idea to give your scenarios titles. Scenario titles generally describe
what is different between the other scenarios.

Also your Given ‘And’ steps feel a little like thens (Its generally a
good idea to avoid talking about user interaction in givens [1] unless
they happened earlier):

And I fill in email with [email protected] mailto:[email protected]

So to ensure that it is clear ‘filling in the email’ happened in the
past I would write it as:

And I have filled in email with ‘email’.

I’ve tried both combinations as your mentioned. I think one of your
questions is what is better:

declarative steps - ‘And I fill in my customer details incorrectly’

imperative steps - ‘And I fill in email with [email protected],’ ’
mailto:[email protected]And I fill in name “Fred Bloggs”’

Ben Mabley has written a great article on imperative vs declarative
steps [2] which is a great read.

I don’t think there is a ‘best way’, It really depends on your
customer. What is really important to them (be it that you might be
mocking the role of the customer) and how would they express it. Is this
a form which is payment/login and is fundamental to the business. If so
the customer may want to express the scenarios in detail.

If its a hugely complex form which would leave your with a massive
number of steps at a low level it might (each customer is different) be
hard to read and provide a lot of noise over the true value.

add table for different combinations of form fields that cause

errors

consider checking that errors are appropriate

Mentioning actual values in scenarios means you can take advantage of
scenario tables. They allow your to test edge conditions without having
lots of scenarios which is great!

Something I’ve been trying out is dropping in and out of different
levels of abstraction in your scenario. For example:

@@@
Scenario: Bad passwords
Given I filled in the form with the essential valid details
And the password was entered as ‘blah’
When …
Then I should see ‘error’

| password | error |
| womble | No wombles allowed as password |
| password | Terrible password |
@@@

I’ve found so far that customers really like this. It is a nice way it
pull out the really interesting part to the customer and leave the other
details (that are often noise) at a higher level. And it also means you
can use Scenario tables to exercise edge conditions.

WDYT?

I’m looking for some input on this, and in particular am wondering
where should I put the more specific tests for form validation, error
messages etc. in my test hierarchy, or even if I should test them at
all (could you argue they’re in built rails functionality).

When it comes down to JavaScript form validation I have started to test
these using screwunit [3]. As for controller based validation and errors
I use specs. Error messages can play such a fundamental part in a site
that I’m always keen on having some spec tests at the controller level
for them. While these test can be a little brittle, I’ve seen too many
error messages get changed or mangled.

I also test forms and error ouput in Features. But more than ‘test a
form’ I’m really testing a user value holds and it just so happens that
this value is achieved through some form process. Its also important to
remember that features cross cut through the whole application stack
when testing unlike my spec tests.

Sorry thats a bit of a brain dump, I hope it provides you with some
helpful info.

:(|)
Joseph W.

http://www.joesniff.co.uk

[1] GitHub: Let’s build from here · GitHub
[2]
http://www.benmabey.com/2008/05/19/imperative-vs-declarative-scenarios-in-user-stories/
[3] http://github.com/grockit/screw-unit-server/tree/master

3

On 7 Nov 2008, at 06:09, Andrew P. wrote:

   And I fill in name "Fred Bloggs"

note: Given I step to customer is nested step doing all sorts to
get to the form

What do you think?

Good questions! You’re getting into some pretty subtle stuff here, and
it’s hard to even articulate what I think about it as it’s changing
all the time as I get more experience working with cucumber.

I guess as Joseph has said, you want to balance out keeping the noise
down in the feature without hiding away important information about
the desired behaviour.

One technique is to have a single noisy ‘declarative’ scenario that
explicitly walks around filling things in, then all the rest use more
‘imperative’ style steps where most of the detail is buried away. I
think, and I know that this sounds wishy-washy, you need to try out
both and see how they feel to you.

In the example you gave, I where the specification seems be be about
valid email addresses, I would probably make my step something like
this:

 Given I step to the customer stage of the wizard
 When I fill in a valid name
 And I fill out an email address "blah"
 Then I should see an error

Thereby hiding the irrelevant “Fred Bloggs”, but surfacing the
important “blah”

I’m looking for some input on this, and in particular am wondering
where should I put the more specific tests for form validation,
error messages etc. in my test hierarchy, or even if I should test
them at all (could you argue they’re in built rails functionality).

Hmm…

Again, this is getting from science to art. I think you need the
features to tell you whether the whole things hangs together (e.g. can
the form display a validation message at all?), but when you start to
iterate more tightly with specific examples of a valid / invalid
object, I that’s where I would drop into unit tests and write code
examples against the model object (or whatever you use for validation).

Having said that, I’m putting more and more specification into
cucumber features - the more steps you write, the more you build up a
re-usable language for describing your system, the easier it gets.

Make any sense? Does that help at all?

cheers
Matt

http://blog.mattwynne.net

4

On Fri, Nov 7, 2008 at 9:58 AM, David C. [email protected]
wrote:

On Fri, Nov 7, 2008 at 9:54 AM, Matt W. [email protected] wrote:

One technique is to have a single noisy ‘declarative’ scenario that
explicitly walks around filling things in, then all the rest use more
‘imperative’ style steps where most of the detail is buried away.

You’ve got those backwards :slight_smile:

declarative: when I fill in the form correctly
imperative: when I enter “David” in the “Name” field …

Although - maybe we need to reconsider this naming, because you could
argue that either are declarative, whereas imperative, meaning
important/urgent, doesn’t really convey the explicit nature. I think
abstract/concrete could work. Maybe general/specific.

Other ideas?

5

On Fri, Nov 7, 2008 at 9:54 AM, Matt W. [email protected] wrote:

One technique is to have a single noisy ‘declarative’ scenario that
explicitly walks around filling things in, then all the rest use more
‘imperative’ style steps where most of the detail is buried away.

You’ve got those backwards :slight_smile:

declarative: when I fill in the form correctly
imperative: when I enter “David” in the “Name” field …

Cheers,
David

6

David C. wrote:

declarative: when I fill in the form correctly

Cheers,
David

rspec-users mailing list
[email protected]
http://rubyforge.org/mailman/listinfo/rspec-users

I like abstract/concrete
My other suggestions:

  • Explicit / Implicit
  • High level / Low level
  • Coarse / fine

–
Joseph W.
http://www.joesniff.co.uk

7

On 7 Nov 2008, at 15:02, David C. wrote:

On Fri, Nov 7, 2008 at 9:58 AM, David C.
[email protected] wrote:

On Fri, Nov 7, 2008 at 9:54 AM, Matt W. [email protected]
wrote:

One technique is to have a single noisy ‘declarative’ scenario that
explicitly walks around filling things in, then all the rest use
more
‘imperative’ style steps where most of the detail is buried away.

You’ve got those backwards :slight_smile:

Doh! What a chump. Sorry for the confusion.

declarative: when I fill in the form correctly
imperative: when I enter “David” in the “Name” field …

Although - maybe we need to reconsider this naming, because you could
argue that either are declarative, whereas imperative, meaning
important/urgent, doesn’t really convey the explicit nature. I think
abstract/concrete could work. Maybe general/specific.

Other ideas?

To be honest, I find both words so boring I don’t really read them,
and hence make those sort of mistakes. But that’s my own stupidity at
fault, not the words’.

I am lacking inspiration to find any better ones, though.

It’s more about whether the step is paramaterized, right? Is there a
term in the cucumber code for differentiating the two?

cheers,
Matt

8

Matt,

Thanks for reply much appreciated and very helpful

I’m definitely feeling a need/desire to write features at different
levels as you say, and in this case I am trying both, though I haven’t
quite worked out a nice way of organising things as yet. I definitely
want to seperate my ‘noisy’ from ‘quite’.

In the example I gave the noisy version would have things about blank
names as well as invalid emails. I’d probably use a scenario table,
and might even include checks for the content of error messages. I
just put a small part in the example to give a flavour.

I do think this is a bit of an art discussion. Its a credit to
cucumber that we can have such discussions as I think it has cracked
most of the science part, which for such a young tool is really
impressive. I think that the art of organisation is a key element in
expressing intention clearly. If something is in the right place is
much easier to find out what it is trying to do. Maybe there is a need
for a feature hierarchy in the same way there is a test heirarchy …

All best

Andrew

2008/11/7 Matt W. [email protected]:

9

Joseph,

Thanks for your reply, much appreciated.

The scenarios don’t have titles just for this discussion. Felt including
them would be a distraction.

I agree that some of the Givens should be Whens in these examples e.g.

Given I step to customer
When I fill in my customer details incorrectly
Then I should see an error

I’ve read the Ben M. article many times its great. Thats where I got
the
declarative and imperative terms from.

I’m not doing any javascript testing yet, I’l put screwunit on my toread
list.

The depends on the customer answer is an interesting one. In this case I
am
the customer (and the developer) and I’m not sure :).

I quite like and have used scenario tables a number of times. In this
case
though I feel that they would pollute the feature. Its as if I want two
levels of features, high level ones to test the overall wizard
behaviour,
and low level ones to test each screen. The tables would be highly
appropriate in the lower level tests. I do wonder though if edge cases
really belong in features. I think of them as more of a developer than
customer concept. However the tables make it really nice to write plain
text
low level (dare I say unit) features, which have the benefit of clearly
documenting intention in a way that any other test mechanism I’ve seen
so
far cannot.

The brain dump is much appreciated and helps

Cheers

Andrew

2008/11/7 Joseph W. [email protected]

10

+1 for abstract and concrete as it gives an indication of what the
story should contain i.e.

I should get an error

vs

I should be told my email must have an @ sign and be …

Andrew

2008/11/7 David C. [email protected]:

11

David C. wrote:

declarative: when I fill in the form correctly

To me the words imperative and declarative make sense in the programming
language context… and we are just applying it to roles instead of
computers. For example, this is the explanation given in my programming
languages book[1]:

…declarative languages, in which the focus is on what the computer
is to do, and the imperative languages, in which the focus is on how
the computer should do it.

Now swap out languages with scenarios and computer with user role:

…declarative scenarios, in which the focus is on what the role is to
do, and the imperative scenarios, in which the focus is on how the
role should do it.

So, I don’t see the word imperative meaning urgent, I see it meaning
explicit. While declarative means the scenario is at a higher level and
not concerned with exactly how the scenario is implemented (much like
use cases.)

That said these terms are programming terms so probably aren’t the best
if they are to be customer facing. I like general/specific.

-Ben

[1] Programming Language Pragmatics by Michael L. Scott

12

“Andrew P.” [email protected] writes:

Given I step to customer
Given I step to customer

I’m looking for some input on this, and in particular am wondering where should I put the more specific tests for form validation, error
messages etc. in my test hierarchy, or even if I should test them at all (could you argue they’re in built rails functionality).

I typically go with the first style, because I’m lazy and don’t want to
write out a bunch of steps and their associated definitions. Also, I
think that “fill in my customer details incorrectly” expresses the
intent better. But really that’s what the scenario title is for I
guess. So mostly it’s just that I’m lazy, cause declarative steps are
less work up front and are more resilient to change.

Pat

13

Joseph W. wrote:

Ben Mabley has written a great article on imperative vs declarative
steps [2] which is a great read.

+1! That article was a real eye-opener to me.

But I keep realizing that, if you keep following the temptation to
super-abstract your specs - from scenarios into matchers, and in other
places - you end up reinventing Ruby. Or AppleScript. Or sendmail.cf.

It’s like the XML-heavy “configuration files” in Java that Rails tries
so
hard to get away from; at some point, your code becomes a generic
execution
environment for your config file, which is now really the code, which
you
then refactor into meta-config files…

If you get too declarative, I think you make it impossible for end users
to
write stories; they can write them, sure, but they’ll all require new
step
matchers, which will require actual coding.

I don’t think it’s actually a solvable problem - something, somewhere
has to
know how to fill in the form. Either you do it in the scenario, where
it
gets repeated a lot, or you refactor it into the matcher, which requires
enough programming ability to understand refactoring, abstraction, and
function calls in the first place.

Jay L.

14

Bit of a ramble, but see inline comments.

Cheers,
Dan

2008/11/7 Joseph W. [email protected]

explicitly walks around filling things in, then all the rest use more
Although - maybe we need to reconsider this naming, because you could
argue that either are declarative, whereas imperative, meaning
important/urgent, doesn’t really convey the explicit nature. I think
abstract/concrete could work. Maybe general/specific.

I agree with abstract and concrete, and in fact this is where scenarios
first came from. A business analyst thinks in abstract terms (“we’re
going
to have to be able to withdraw cash”), and the tester’s job is then to
map
that to concrete examples (“start with $100, request $80, make sure we
get
$80 and have $20 left in the account”). The tester’s description will
necessarily be finer grained.

The scenario title should be at the analyst (more abstract) level of
granularity: “withdraw funds from an account with sufficient balance”

The imperative/declarative terminology is a bogus one. Imperative means
“how”, and declarative means “what”. Any level of abstraction is both a
how
and a what depending on your perspective. There’s an exercise you can do
with any statement to demonstrate this. Simply ask “how?” to move down
the
stack of abstraction or “what for?” to move up. It’s turtles in both
directions. Here’s an example:

SQL is the canonical example of declarative, so:

select count(*) from accounts where balance < 0

But if I ask “what for?” I get:

find out how many accounts are overdrawn ("more" declarative)

What for?

calculate our exposure to dollar fluctuations

What for?

what is our operating risk in the current financial climate

What for?

... etc

Now going the other way:

select count(*) from accounts where balance < 0

How?

set up a counter variable, then scan the accounts table and

increment
whenever you see a negative balance

How?

cue up a number of subtasks that will scan segments of the accounts

table, then aggregate the results

How?

... etc.

Having said all that, it’s important to be able to define your own
abstractions over the operations you carry out in your application.
“Login
as Dan” is a sensible chunk to have a conversation about. At that point
I
would probably use ruby methods to aggregate multiple steps into
coarser-grained abstractions.

Of course each of these abstractions tells you something new about your
domain, and in that regard it has value to someone coming along later
and
reading your examples. ("These three steps always seem to occur together

perhaps we should have a name for that".)

Cheers,
Dan

ps. David is right - the list-of-instructions is imperative, the
what-I-want
is declarative.