💳 Secure Payment

Full-Service Web & Software Agency · Klamath Falls and Redding

BDD testing framework

Work with Sean

A behavior-driven development (BDD) testing framework earns its place when it runs the scenarios reliably, stays easy to maintain and reports each failure in the scenario’s own words.

In custom apps, the Sean Dinwiddie’s Webmastery team writes its scenarios in Gherkin and runs them in two ways, and keeps exploring both:

  • as executable steps with cucumber-js, Cucumber’s official JavaScript implementation;
  • as tests named after the scenarios in the codebase’s own runner, such as Vitest for TypeScript and Hspec for Haskell.

Twelve requirements come first; then one cart scenario runs both ways, both meet the app’s own reducer, and one changed line breaks both, each failing under the scenario’s name.

What a framework has to do

Either way, the framework has twelve requirements to meet, though three of them matter chiefly for the few end-to-end journeys, as the end of this lesson shows:

  1. Gherkin as written: The framework should run scenarios written in Gherkin, in the owner’s words, or name its tests after them, so the owner can read what runs.
  2. Scenario organization: The framework should group scenarios under the Feature they test, in a feature file or a describe block named for it.
  3. Parameterization: The framework should run one scenario, a Scenario Outline in Gherkin, once for each row of its Examples table, so a rule with many cases stays one scenario.
  4. Scenario tags and labels: The framework should let scenarios carry tags or labels, so they are easy to sort and filter. A tag marks scenarios for a purpose, such as a regression run.
  5. Reporting and logging: The framework should report each result under its scenario’s name, so a failure reads in the written scope’s own words, and keep the log that explains it.
  6. Integration with development tools: The framework should run in the continuous integration pipeline on every change, so nothing merges with a failing scenario.
  7. Parallel execution: The framework should run scenarios in parallel, across several environments or configurations, to save time.
  8. Environment configuration: The framework should let a run name its environment, such as development, testing or staging, and never production, where a scenario that places an order places a real one.
  9. Cross-browser testing: For the few end-to-end journeys, the framework should drive the browsers the site’s customers use, on the phones and screens the written scope names; the scenarios below the interface need no browser at all.
  10. Reusable steps: A step definition matches its line wherever it appears, so one Given serves every scenario that starts from it, and a Background states the Givens a whole Feature shares, as Writing BDD scenarios for software modules shows.
  11. Community and support: The project should be actively maintained, with documentation that keeps up with its releases, so a question about a step has somewhere to go.
  12. Compatibility: The framework runs in the codebase’s own language and toolchain: cucumber-js or Vitest for TypeScript, Hspec for Haskell.

One scenario, run both ways

Take the cart scenario that Module 4’s From scenario to slice builds into a slice, and run it both ways. The two routes meet at the same code:

One scenario, two runners The scenario Adding a product to the cart branches to two runners: cucumber-js, with step definitions, and Vitest, with one test named for the scenario. Both lead to the same reducer and selectors. Adding a product to the cartcucumber-jsstep definitionsVitesta test named for itreducer and selectors
Two routes, one destination. cucumber-js matches each line of the scenario to a step definition, Vitest runs one test named for it, and both arrive, along the violet arrows, at the reducer and selectors the app itself uses.

First, as executable steps: cucumber-js reads the feature file as written, and each step definition matches one line; the When and the Thens call the same reducer and selectors the app uses.

// features/step_definitions/cart.steps.ts
import assert from 'node:assert/strict'
import { Given, Then, When, World, setWorldConstructor } from '@cucumber/cucumber'
import { cartSlice, itemAdded, selectLines, selectTotalCents } from '../../src/features/cart/cartSlice'

type Product = { productId: string; name: string; priceCents: number }

// A fresh world for every scenario.
class CartWorld extends World {
  products = new Map<string, Product>()
  cart = cartSlice.getInitialState()
}
setWorldConstructor(CartWorld)

Given('{string} is in stock at ${int}', function (this: CartWorld, name: string, dollars: number) {
  this.products.set(name, { productId: name.toLowerCase().replace(/ /g, '-'), name, priceCents: dollars * 100 })
})

Given('the cart is empty', function (this: CartWorld) {
  this.cart = cartSlice.getInitialState()
})

When('the customer adds {string} to the cart', function (this: CartWorld, name: string) {
  const product = this.products.get(name)
  if (!product) throw new Error(`No product named ${name}`)
  this.cart = cartSlice.reducer(this.cart, itemAdded(product))
})

Then('the cart contains {string}', function (this: CartWorld, name: string) {
  assert.deepEqual(selectLines({ cart: this.cart }).map((line) => line.name), [name])
})

Then('the cart total is ${int}', function (this: CartWorld, dollars: number) {
  assert.equal(selectTotalCents({ cart: this.cart }), dollars * 100)
})

cucumber-js looks for feature files, and the step definitions beside them, in features/ at the project’s root, so the cart scenario, as Writing BDD scenarios sets it out, goes in features/cart.feature; the app’s own feature folders, such as features/cart/ below, sit under src/.

cucumber-js looks only for JavaScript step files on its own, and Node’s own type stripping needs file extensions this file’s imports leave out, so a TypeScript one like this loads once a transpiler such as tsx is registered in the cucumber-js configuration:

// cucumber.mjs
export default {
  requireModule: ['tsx/cjs'],
  require: ['features/step_definitions/**/*.ts'],
}

cucumber-js’s transpiling guide gives that setup for CommonJS packages; it runs unchanged in a "type": "module" package too, where the guide also offers tsx’s ESM register, loaded through import.

Run the second way, the scenario is one test in the codebase’s own runner, named for it, with its steps as comments:

// features/cart/cart.test.ts, from From scenario to slice
import { describe, expect, it } from 'vitest'
import { cartSlice, itemAdded, selectLines, selectTotalCents } from './cartSlice'

const widgetA = { productId: 'widget-a', name: 'Widget A', priceCents: 2500 }

describe('Feature: Cart', () => {
  it('Scenario: Adding a product to the cart', () => {
    // Given "Widget A" is in stock at $25, and the cart is empty
    const before = cartSlice.getInitialState()
    // When the customer adds "Widget A" to the cart
    const after = cartSlice.reducer(before, itemAdded(widgetA))
    // Then the cart contains "Widget A", and the cart total is $25
    expect(selectLines({ cart: after }).map((line) => line.name)).toEqual(['Widget A'])
    expect(selectTotalCents({ cart: after })).toBe(2500)
  })
})

Both run the same reducer and selectors, and both fail in the scenario’s own words. To try it, change quantity: 1 to quantity: 0 in itemAdded, in the cartSlice.ts from From scenario to slice, and run both: each fails on the cart total, under the scenario’s name.

$ sed -i.bak 's/quantity: 1 })/quantity: 0 })/' src/features/cart/cartSlice.ts
$ npx cucumber-js 2>&1 | grep -E '#|Error|!=='
  1) Adding a product to the cart # features/cart.feature:3
       And the cart total is $25 # features/step_definitions/cart.steps.ts:33
           AssertionError [ERR_ASSERTION]: Expected values to be strictly equal:
           0 !== 2500
$ npx vitest run 2>&1 | grep -E 'FAIL|AssertionError'
 FAIL  src/features/cart/cart.test.ts > Feature: Cart > Scenario: Adding a product to the cart
AssertionError: expected +0 to be 2500 // Object.is equality

A framework that meets these requirements runs every scenario on every change and reports each failure in the words the owner read. The lecture Redux Toolkit and RTK Query Best Practices lists what to test at each layer, from reducers and selectors to endpoints and components.

That completes Module 2: each criterion a scenario the owner can read, reviewed as a group and run on every change, failing in the owner’s words. Module 3 comes next: functional reactive programming (FRP), taught through the Redux Toolkit state layer these scenarios test.

Copyright Sean Paul Payne Dinwiddie
All Rights Reserved