Skip to content

Latest commit

 

History

History
711 lines (445 loc) · 12.5 KB

acceptance_specs.md

File metadata and controls

711 lines (445 loc) · 12.5 KB

[fit] How I Learned To Love Acceptance Testing


Jason Carter

Software Engineering Team Lead

@jason URUG


Extreme Programming

Pair Programming

Test Driven Development

Rad Hype Videos

autoplay mute loop


A quick story...


le me

fit left


Test Driven Development


...so that means, write a unit test, implement it

fit right


Unit tests gave me confidence that my class worked as expected


On unit tests...


But there was a problem...


fit


So I got better at writing integration tests...


Integration tests gave me confidence that my classes worked together as I expected


On integration tests...


fit


So how do you make sure that you're building the right thing, and that it works?


fit


¯\(ツ)


Fast forward a year...


First big project as a team lead

fit


Overengineering

Rejections galore


Start with an acceptance test


Acceptance testing

Formal testing with respect to user needs, requirements, and business processes conducted to determine whether or not a system satisfies the acceptance criteria and to enable the user, customers or other authorized entity to determine whether or not to accept the system.

-- International Software Testing Qualifications Board


Acceptance Testing

A test that must pass for the user story you're working on to be complete -- Jason Carter


Turns out this is totally part of XP


fit


The name acceptance tests was changed from functional tests. This better reflects the intent, which is to guarantee that a customers requirements have been met and the system is acceptable.

-- Don Wells, extremeprogramming.org


Now you may be thinking...


Isn't that what QA is for?

fit left


They have better things to do


They're great at finding edge cases

fit left


They're great at identifying wonky behavior


They should be your last line of defense, not your first


fit


The acceptance test tells you when you're done... immediately


Pattern becomes:

  • Write the acceptance test to know when the feature is done
  • Test drive the implementation with unit and integration specs
  • Refactor safely knowing that the feature still works the way it should

So how do we accomplish this in a Rails app?


Selenium


Usually people have strong opinions on Selenium...


Should be used as a last resort

left fit


That's QAs job!

right fit


It fails for no reason!

left fit


Me

fit


So we set out to make it palatable to me...


...and to product and QA.


Topics

  • General principles
  • Avoiding "intermittent failures"
  • Vanilla
  • Setup, Actions, and Assertions
  • Page Object pattern
  • Random musings

Principles


An acceptance test should...


Exercise your app to its fullest


Avoid mocking as much as possible


Only simulate what a user could do


Give you confidence that your app is working as it should


Be the first test you write in the case of a regression


Avoiding "intermittent failures successes"


Race conditions


Explicit vs implicit wait


Not scoping your finders


Asynchronous code


Mixing setup and actions


Vanilla


describe 'when deleting an existing Review', js: true, type: :request do
  # A bunch of lets
  context 'When the user is a review creator' do
    it 'can be deleted' do
      find('.workspace-proof-title', text: proof.title).click
      within '.workspace-proofs-table' do
        within 'tr', text: 'some proof' do
          expect(page).to have_text('In Review ( 1/2 )')
        end
      end
      within '#side-panel' do
        find('span', text: 'Delete Review').click
      end
      within '.show-mavenlink-alert' do
        expect(page).to have_selector('h2', text: 'Delete')
        expect(page).to have_selector('button', text: 'Delete')
        expect(page).to have_selector('button', text: 'Cancel')
        click_button('Delete')
      end
      within '#side-panel' do
        expect(page).to have_button('Create Review')
      end
      find('.js-panel-close-link').click
      find('.workspace-proof-title', text: 'Some Proof').click
      within '#side-panel' do
        expect(page).to have_button('Create Review')
      end
      within '.workspace-proofs-table' do
        within 'tr', text: proof.title do
          expect(page).to have_no_text('In Review ( 1/2 )')
        end
      end
    end
    end
  end
 end

Pros


At the very least we're avoiding intermittent failures


Cons


Only a dev can read this, and probably only one with context


Mixes assertions and actions


Tough to create a new test immediately


It offends my DRY sensibilities


If I change the DOM, I have to update a bunch of stuff...


... and I might not know what the behavior was in the first place?


But what about comments, Jason?


Comments are just another thing to keep up to date


Sure comments are sometimes necessary


A good method name is generally a fine comment


And if you need comments, chances are your method is doing too much*


* Jason opinion


So if we don't like comments, how do we make this more understandable?


Setup, Actions, and Assertions


describe 'when deleting an existing Review', js: true, type: :request do
	# a bunch of lets
   it 'can be deleted' do
     navigate_to_proof_tab(proof_employee, workspace)
     click_proof_title(proof)
     it_should_have_proof_with_status(proof, 'In Review ( 1/2 )')
     click_delete_review
     confirm_action 'Delete'
     it_has_create_review_option
     it_persists_the_delete(proof)
     it_should_not_have_proof_with_status(proof, 'In Review ( 1/2 )')
   end
 end

Pros


Definitely easier to read


Reusable methods!


Abstracted away the DOM


Cons


Not always easy to tell what page we're on


Not a great way to show that an action leads to an assertion


Not pictured is our giant function library


Page Object Pattern


describe 'inviting people to a workspace', js: true, type: :request do
  describe 'the invitation flow' do
    context 'when adding to consultants' do
      context 'when the account is not free or trialing' do
      	let(:project_invitation_form) { Pages::ProjectInvitations::Form.new('maven') }
        let(:inviting_user) { users(:jane) }
        let(:invited_user) { users(:alice) }
        let(:workspace) { workspaces(:jane_car_wash) }

        it 'will invite a maven to a workspace' do
          login(inviting_user)
          visit workspace_path(workspace)
          invite_a(team, invited_user)

          expect(project_invitation_form).to have_save_message_as_default_checkbox
          expect(project_invitation_form).to have_no_subject_and_message_tipsy
          expect(project_invitation_form).to have_no_upgrade_call_to_action
          expect(project_invitation_form.send_invitation).to have_sent_an_invitation
        end
      end
    end
  end
end

module Pages
  module ProjectInvitations
    class Form
      include Capybara::DSL

      def initialize(team)
        @team = team
      end

      def has_save_message_as_default_checkbox?
        page.has_css?('#save_message_as_default')
      end

      def send_invitation
        click_button 'Send invitation'
        self
      end

      def has_sent_an_invitation?
        within('.flash-container') do
          page.has_css?('.notice', text: '1 invitation was sent for this project')
        end
      end

      def has_not_sent_an_invitation?
        within('.flash-container') do
          page.has_css?('.notice', text: 'The invited user must be a member of your account.')
        end
      end
    end
  end
end

A couple things to note


def has_sent_an_invitation?
  within('.flash-container') do
    page.has_css?('.notice', text: '1 invitation was sent for this project')
  end
end

# Selenium does some magic for us
# has_sent_invitation? -> have_sent_an_invitation

Returning self lets us chain things together, and use the to syntax

def send_invitation
  click_button 'Send invitation'
  self
end

describe 'inviting people to a workspace', js: true, type: :request do
  describe 'the invitation flow' do
    context 'when adding to consultants' do
      context 'when the account is not free or trialing' do
      	  # ...
          expect(project_invitation_form.send_invitation).to have_sent_an_invitation
        end
      end
    end
  end
end

Pros


Much easier to tell what page we're on!


We can scope actions and assertions to pages (or subpages!)


Still pretty readable! In fact, lets us see cause and effect much easier


Martin Fowler said its awesome!


Cons


I haven't found any yet...


Other musings


expect(project_invitation_form.send_invitation).to [
	have_sent_an_invitation,
	have_notified_some_service,
	have_done_something_else
]

What if I want to make the same assertions on multiple pages?


Shared assertions between page objects


What about common tasks like logging in?


Shared actions between page objects or done before setting up the page object.


3rd party integrations?


We integrate with a 3rd party via Delayed Job


We can assert that Delayed Job is called, but what about that it does the right thing?


def run_conceptshare_jobs_immediately
  stub(Delayed::Job).enqueue do |job|
	job.perform if job.respond_to?(:queue_name) && job.queue_name == 'conceptshare'
	true
  end
end

context 'when adding to a workspace mapped to conceptshare' do
  it 'will create a conceptshare mapping for the participation' do
    run_conceptshare_jobs_immediately
    mock.any_instance_of(Conceptshare::AccountClient).create_user_if_necessary(invited_user)
    mock.any_instance_of(Conceptshare::AccountClient).add_user_to_account(invited_user)
    mock.any_instance_of(Conceptshare::AccountClient).add_user_to_project(invited_user, workspace, "Commentator")

    login(inviting_user)
    visit workspace_path(workspace)
    invite_a(team, invited_user)
    project_invitation_form.send_invitation
  end
end

This gets us partway there


How do I make sure the 3rd party does what I want?


VCR?


What if you still got the behavior wrong?


Clarify with QA and product up front


Have them look at your now readable acceptance tests


But Jason, this all sounds well and good...


How do I start?



In Summary


We're avoiding common pitfalls...


We're avoiding "building the wrong thing"...


It serves as documentation thats readable by all stakeholders...


It makes it easier to do sweeping refactors and maintain behavior...


... and


Me now

fit


Questions?