January 29, 2011

#41 Integration Tests With Capybara

In this post, we'll look into writing some integrations tests. An integration test will test multiple parts of our application (models, views, controllers, etc.) These tests will help to make sure that the components are interacting well together.

The Gems

As we learned in the last post, we can use Capybara to simulate a user's interaction with our app. In addition to Capybara, we will be using FactoryGirl to generate model data for our test database. (Remember to add factory_girl_rails to spec_helper. See Post 35.) The other gems, Launchy and DatabaseCleaner, will be discussed below. We keep Webrat since we relied on its matchers to write some of our view tests. Since Webrat and Capybara do similar things, including both with have some caveats (discussed at the end.)

(Note: The code for this post is up on the Github repo. Eventually, we will be writing acceptance tests with Cucumber. What we're doing in this post is to understand some of the things necessary to use Cucumber. Like everything else presented in this site, the techniques shown aren't necessarily 'best' or even 'good' practices.)

Gemfile, Terminal
# Gemfile
...
group :development, :test do
  gem 'rspec-rails', '>= 2.4.1'
  gem 'spork', '>= 0.9.0.rc2'
end

group :test do
  gem 'factory_girl_rails', '1.1.beta1'

  gem 'capybara', '0.4.1.1'
  gem 'launchy', '0.3.7'
  gem 'database_cleaner', '0.6.0'
  
  gem 'webrat', '>= 0.7.3'
end
________________________________________________________________
 
# Terminal
> bundle

Autotest

We will be putting our integration tests in spec/integration. If we want to use Autotest during the development of our integration test, we need to add the mapping in our .autotest file.

(If using Autotest's add_mapping in .autotest doesn't work, try this monkey patch to the #setup_rails_rspec2_mappings method found in the rails_rspec2.rb file of the rspec-rails gem. The entire method stays the same except that: 1) We add an extra mapping for the integration folder. 2) We change one of the existing mappings so that when anything in the support folder is updated, our integration tests will run with everything else.)

./.autotest
# Autotest.add_hook :initialize do |at|
#   at.add_mapping(%r%^spec/(integration)/.*rb$%) {|filename, _|
#     filename
#   }
# end
# If the above doesn't work, try this:

class Autotest::RailsRspec2 < Autotest::Rspec2

  def setup_rails_rspec2_mappings
    ...
    clear_mappings
    ...
    add_mapping(%r%^spec/integration/.*rb$%) { |filename, _|
      filename
    }
    add_mapping(%r%^(spec/(spec_helper|support/.*)|config/(boot|environment(s/test)?))\.rb$%) {
      files_matching %r%^spec/(integration|models|controllers|routing|views|helpers)/.*_spec\.rb$%
    }
    ...
  end
end

The Setup

For our first integration tests, we will act like a user who is not logged into our site and wants to see our listing of posts. When we start these tests, the database is empty, so we use FactoryGirl to make some posts. We put the Factory calls into let statements and cache the results on the first call of the posts. Notice that we have tagged this example with ':integration => :true'. This allows us to run 'rspec spec -t integration' to run only the tests with this tag.

spec/factories/post.rb, spec/integration/user/post_viewing_spec.rb
# post.rb
Factory.sequence :seq do |n|
  n
end

FactoryGirl.define do
  factory :post do
    sequence(:title) { |n| "#{n} Zombies!"}
    add_attribute(:sequence) { Factory.next(:seq) }
    status 'published'
  end
end
________________________________________________________________

# post_viewing_spec.rb
require "spec_helper"

describe "Meet Capybara (non admin).", :integration => true do
  context "He's crawling around our app." do
    let(:post1){ @post1 ||= Factory(:post) }
    let(:post2){ @post2 ||= Factory(:post) }

    before { post1; post2 }
I am Capybara. (Picture from http://www.mazeforge.com/Words/?p=2699)
Capy-1-300

Interacting With The Page

In general, our integration tests will interact with a page (ie. visit a page or fill in a form) and then assert that the results contain certain elements or content. For the first test, we use Capybara's #visit to get the page of interest. Once we have that page, we can use the #have_content matcher to make sure the necessary content is present.

In the second test, we have Capybara click a 'Show' link and make sure certain things are on that page. If we want to see the page that Capybara is seeing, we can use the save_and_open_page method. This method requires the use of the Launchy gem which will open a browser and show us the page of interest.

In the third test, we make sure that our visitor doesn't see an 'Edit' link since since he's not an admin. Notice that Capybara recommends not using 'page.should_not ...' and instead use 'page.should ...' with Capybara's negative expectation matcher.

spec/integration/user/post_viewing_spec.rb
...
   before { post1; post2 }

    context "When he visits the post index page," do
      before { visit posts_url }

      it "then he should see a listing of post titles" do
        page.should have_content post1.title
        page.should have_content post2.title
      end

      it "and the posts should be in div.postShow" do
        within(:xpath, '//div[@class="postShow"][1]') do
          page.should have_content(post1.title)
        end

        within("div.postShow[2]") do
          page.should have_content(post2.title)
        end
      end

      context "When Capybara clicks on the 'Show' link for the first post," do
        before do
          find(:xpath, "//a[@href=\'#{post_url(post1.sequence)}\']").click
        end

        it "then he should see the show page" do
          save_and_open_page
          page.should have_content post1.title
          current_path.should == "/posts/#{post1.sequence}"
        end

        it "and he shouldn't see an edit link" do
          page.should have_no_selector 'a', :text => 'Edit'
        end
      end
My paws are made for clickin'...
Capy2-300

Changing The DSL

We can use RSpec tags to specify helper modules for certain test. So for example, we can create a IntegrationHelper module specifically for the integration tests. If we want to avoid tagging all of our integration tests, we configure an :example_group with a :file_path to our integration tests.

Let's change our RSpec DSL a little and create some aliases using Ruby's alias_method and RSpec's define_example_method. With our alias methods, our integration test are starting to look like a certain green fruit/vegetable. (For those who don't know, Cucumber arose from RSpec's Story Runner. If these DSL tweaks appeal to you, check out the Steak gem or the Unencumbered gem. )

spec/spec_helper.rb, spec/support/integration_helper.rb, spec/integration/user/post_viewing_spec.rb
# spec_helper.rb
...
Spork.prefork do
  ...
  RSpec.configure do |config|
    ...
#  config.include MySpec::IntegrationHelper, :integration => true
   config.include MySpec::IntegrationHelper, :type => :integration, :example_group => {
     :file_path => config.escaped_path(%w[spec integration])
   }
  end
...
________________________________________________________________

# integration_helper.rb
module MySpec
  module IntegrationHelper
    def self.included(base)
      base.class_eval do
        include DSLHelper
      end
    end
  end

  module DSLHelper
    def self.included(base)
      class << base
        alias_method :Given, :describe
        alias_method :And, :describe
        alias_method :When, :describe
      end

      RSpec::Core::ExampleGroup.class_eval do
        define_example_method :Then
        define_example_method :And_
        define_example_method :Or
      end
    end
  end
end

class Object
  alias_method :Scenario, :describe
end
________________________________________________________________

# post_viewing_spec.rb
...
Scenario "Meet Capybara (non admin).", :integration => true do
  Given "He's crawling around our app." do
    ...
    When "he visits the post index page," do
      ...
      Then "he should see a listing of posts" ...
      And_ "the posts should be in div.postShow" ...

      When "Capybara clicks on the 'Show' link for the first post," do
        ...
        Then "he should see the show page" ...
        And_ "he shouldn't see an edit link" ...

Session Helper Methods

The last spec was testing a feature that only admins should be able to see (the 'Edit' link). When we spec'd the post index view, we stubbed the #admin? method in order to hide the 'Edit' link. If we generate some scaffolding now for Post, this integration test fails because there is no #admin? method yet.

What can we do if we don't want to switch gears and work on the site's administration features yet? As with the view specs, we can stub out the #admin? method in our integration tests. So let's create a SessionsHelper module to toggle #admin? from true to false depending on our needs. In the ApplicationController, we'll just declare the helper method and raise a reminder that we need to implement the code for that method.

Terminal, spec/support/integration_helper.rb, spec/integration/user/post_viewing_spec.rb
> rails g scaffold post title:string description:text sequence:integer status:string published_at:datetime -s
________________________________________________________________

# integration_helper.rb
module MySpec
  module IntegrationHelper
    def self.included(base)
      base.class_eval do
        include DSLHelper
        include SessionsHelper
      end
    end
  end

  module SessionsHelper
    def not_admin
      ApplicationController.class_eval do
        def admin?
          false
        end
      end
    end

    def sign_in_admin
      ApplicationController.class_eval do
        def admin?
          true
        end
      end
    end
    ...
________________________________________________________________

# application_controller.rb
class ApplicationController < ActionController::Base
  protect_from_forgery

  helper_method :admin?

  protected

  def admin?
    raise 'Hacka says wha? (Need to write this code.)'
  end
end
________________________________________________________________

# post_viewing_spec.rb
...
Scenario "Meet Capybara (non admin)." do
  before { not_admin }
  Given "He's crawling around our app." do
    ...
ಠ_ಠ
Capy-3-300

An Autotest-Growl Reminder

Some may frown upon our strategy for testing admin functionality because integration tests are supposed to be 'black box' tests that should not know anything about the internals of our application. Eventually when we have written our authentication features, we can define our SessionsHelper module to use Capybara methods to sign in an admin. (i.e. 'visit login_page', 'fill_in("Name", :with => 'login'), etc.)

To help remind us that we need to address this issue, we'll hack together a little Autotest-Growl reminder to show what methods we have to get back to. To simplify things, we hijack RSpec's namespace to add some methods and use RSpec's Reporter object to do our bidding.

This bit of code mimics what Autotest-Growl does to display our test results. RSpec's #notify method will display our message in the terminal as the tests are being run. When the tests are finished running, Autotest-Growl scans the terminal output and picks out our messages by their prefix. We can then growl a message in yellow.

spec/support/integration_helper.rb, ./.autotest
# integration_helper.rb
module MySpec
  module IntegrationHelper
    def self.included(base)
      base.class_eval do
        ...
        include SessionHelper
        include TodoHelper
      end
  end

  module TodoHelper
    RSpec.module_eval do
      def self.hack_alert(klass, method)
        RSpec.configuration.reporter.hack_alert(klass, method)
      end
    end

    RSpec::Core::Reporter.class_eval do
      def hack_alert(klass, method)
        notify :message, "\nDuck Punched!: #{klass.name} #{method.to_s}"
      end
    end
  end

  module SessionsHelper
    def not_admin
      ApplicationController.class_eval do
        RSpec.hack_alert(self, __method__)
        def admin?
          false
        end
      ...
________________________________________________________________

# .autotest
...
Autotest::Growl::remote_notification = true

Autotest.add_hook :ran_command do |autotest, modified|
  alerts = autotest.results.select { |line| /Duck Punched!/ =~ line }.uniq
  Autotest::Growl.growl('Todo:', alerts.join(''), 'pending', 0) unless alerts.empty?
end

Configuration For Testing JavaScript

Capybara uses a Rack strategy at baseline. If we need to test a page with JavaScript, we need to switch the driver that Capybara uses. Out of the box, we can use Selenium. We could use Selenium for all of our tests, however, using a browser-based driver will slow down our tests significantly. Therefore we use Selenium only for JavaScript tests. We can accomplish this by tagging the example groups with ':js => true' and configure RSpec to use the DatabaseCleaner gem as described here.

If we want to exclude the Selenium tests from routine runs, we can use RSpec's filter_run_excluding configuration option. Then we can put together a freedom patch to rspec-core to run the excluded tests from the command line. What we're doing here is intercepting the tags before they are included in the filtering process in RSpec's World class.

(Note: This hack is working for me with and without Autotest and Spork running.The ability to use RSpec tags while Spork is running was recently added to rspec-core, so you have to use rspec-core master at this point (or patch your local copy.) Also, the ZenTest version of Autotest doesn't allow command line options to be passed on to RSpec. However, this is supported in the grosser version of Autotest. With that, we can hack a solution to enable RSpec to take the Autotest command line configurations. Here's some extra information.)

spec/spec_helper.rb
# spec_helper.rb
...
Spork.prefork do
  ...
  RSpec.configure do |config|
    ...
    config.use_transactional_fixtures = false
  end
  #################################################
  RSpec.configure do |config|
    config.before(:suite) do
      DatabaseCleaner.strategy = :transaction
      DatabaseCleaner.clean_with :truncation
    end

    config.before(:each) do
      if example.metadata[:js]
        Capybara.current_driver = :selenium
        DatabaseCleaner.strategy = :truncation
      else
        DatabaseCleaner.strategy = :transaction
        DatabaseCleaner.start
      end
    end

    config.after(:each) do
      Capybara.use_default_driver if example.metadata[:js]
      DatabaseCleaner.clean
    end
  end
  #################################################
  RSpec.configure do |config|
    config.filter_run_excluding :js => true
  end

  # 'rspec spec' runs all tests excluding 'js'
  # 'rspec spec -t js' runs only 'js' tests
  # 'rspec spec -t all' runs all tests including 'js'
   module RSpec
    module Core
      class World
        attr_accessor :hack_run_all

        def inclusion_filter
          if @configuration && @configuration.filter && @configuration.filter.has_key?(:all)
            @configuration.filter = nil
            self.hack_run_all = true
          end
          @configuration.filter
        end

        def exclusion_filter
          @configuration.exclusion_filter.delete(:js) if self.hack_run_all || (@configuration.filter && @configuration.filter.has_key?(:js))
          @configuration.exclusion_filter
        end
      end
    end
  end
end
...
Capybara hungry!
Capy-4-300

Testing The Destroy Link

Rails implements the 'Destroy' link via JavaScript code in the rails.js file. In order to test the link, we need to handle the confirmation popup window that is shown. We do this by using #evaluate_script to return a value from the confirmation box. (We could also test the delete functionality when using RackTest driver in a non-JavaScript example by using 'page.driver.delete "/posts/1" '.)
spec/integration/admin/js/post_destroy_spec.rb
...
Scenario "Capybara is now an admin.", :js => true do
  Given "he's at the post index page," do
    let(:post1){ @post1 ||= Factory(:post) }
    let(:post2){ @post2 ||= Factory(:post) }

    before do
      post1; post2
      sign_in_admin
      visit '/posts'
    end

    Then "he can delete the post" do
      page.evaluate_script('window.confirm = function() { return true; }')
      click_link 'Destroy'
      page.should have_no_content post1.title
    end

    Or "he can delete the post, but choose not to" do
      page.evaluate_script('window.confirm = function() { return false; }')
      click_link 'Destroy'
      page.should have_content post1.title
      page.should have_link 'Destroy', :count => 2
    end

Capybara Vs. Webrat

In Post #37, we used Webrat's matchers for our view specs because Capybara's matchers don't work in the context of RSpec's view tests (although this is being worked on.) In our integration tests, RSpec includes RSpec::Rails::RequestExampleGroup (see Post #40) which will include Webrat and Capybara if both are available. This means that we can use some of Webrat's methods in our tests in addition to Capybara's. However, if a method is defined in both Webrat and Capybara, then the Capybara method will persist since Capybara was included after Webrat.

(In IRB, we can see which Webrat methods Capybara will overwrite.)

rspec-rails/lib/rspec/rails/example/request_example_group.rb, spec/integration/user/post_viewing_spec.rb, IRB
# request_example_group.rb
module RSpec::Rails
  module RequestExampleGroup
    ...
    webrat do
      include Webrat::Matchers
      include Webrat::Methods
      ...
    end

    capybara do
      include Capybara
    end
__________________________________________________

# post_viewing_spec.rb
...
      And_ "the posts should be in div.postShow" do
        ...
        # Capybara syntax
        within("div.postShow[2]") do
          page.should have_content(post2.title)
        end
       
        # Webrat syntax
        page.should have_selector '.postShow' do |div|
          div.should contain post2.title
        end
      end
      ...
        And_ "he shouldn't see an edit link" do
          # Capybara syntax
          page.has_selector?('a', :text => 'Edit').should be_false
          page.should have_no_selector 'a', :text => 'Edit'
#         page.should_not have_content 'Edit'    # Capybara recommends not doing this

          # Webrat syntax
          page.should_not have_selector("a", :count => 2)
          page.should_not have_selector 'a', :content => 'Edit'
          page.should_not contain 'Edit'
        end
        ...
__________________________________________________

# IRB
> %W(rubygems webrat capybara/dsl).each { |file| require file }
> class W; include Webrat::Matchers; include Webrat::Methods; end
> class C; include Capybara; end

# Webrat methods that are overridden by Capybara
> C.instance_methods.sort & W.instance_methods - Object.methods
  =>  ["attach_file", "check", "choose", "click_button", "click_link", "current_url",
       "field_labeled", "fill_in", "save_and_open_page", "select", "uncheck", "unselect",
       "visit", "within"]

Reference - Using A Webrat Matcher On A Capybara Object

So what's happening when we write "page.should have_selector 'a' "? When we write a 'should' statement, we are passing in some kind of matcher object as an argument to #should. In our example, when RSpec included Webrat::Matchers, the RSpec example group object picked up the #have_selector method. Calling this method creates a Webrat::Matchers::HaveSelector matcher object.

And how did 'page' get this #should method? Well, RSpec defined that method in Ruby's Kernel module. That means every object has a #should (and #should_not) method. Calling #should calls the .handle_matcher method of the RSpec::Expectations::PositiveExpectationHandler class.

Within .handle_matcher, RSpec calls #matches on our HaveSelector instance. This method is defined in Webrat::Matchers::HaveXpath class. Through this method, with help from the Webrat::XML module, Webrat will take Capybara's 'page' object and convert it to a string with 'page.body'. This string is then converted into a Nokogiri::HTML::Document object which Webrat knows how to work with.

Explicit Matcher Process
page.should( have_selector('a') )
________________________________________________________________

# webrat/core/matchers/have_selector.rb
module Webrat
  module Matchers
    class HaveSelector < HaveXpath ...

    def have_selector(name, attributes = {}, &block)
      HaveSelector.new(name, attributes, &block)
    end
________________________________________________________________

# rspec-expectations/lib/rspec/expectations/extensions/kernel.rb
module Kernel
  def should(matcher=nil, message=nil, &block)
    # 'self' here is 'page'
    RSpec::Expectations::PositiveExpectationHandler.handle_matcher(self, matcher, message, &block)
  end
________________________________________________________________

# rspec-expectations/lib/rspec/expectations/handler.rb
module RSpec
  module Expectations
    class PositiveExpectationHandler
      def self.handle_matcher(actual, matcher, message=nil, &block)
        ...
        # 'actual' is 'page'
        # 'matcher' is an instance of Webrat::Matchers::HaveSelector
        match = matcher.matches?(actual, &block)
        return match if match
________________________________________________________________

# webrat/core/matchers/have_xpath.rb
module Webrat
  module Matchers
    class HaveXpath
      def matches?(stringlike, &block)
        ...
        matched = matches(stringlike)    # 'stringlike' is 'page'
        ...
      end

      def matches(stringlike)
        nokogiri_matches(stringlike)
      end

      def nokogiri_matches(stringlike)
        ...
        @document = Webrat::XML.document(stringlike)
        @document.xpath(*@query)    # '@query' is the CSS selector we're looking for 'a'
      end
________________________________________________________________


# webrat/core/xml.rb
module Webrat
  module XML
    def self.document(stringlike)
      ...
      if ...
      elsif stringlike.respond_to?(:body)
        Nokogiri::HTML(stringlike.body.to_s)
      else
        ...

Reference - Implicit Matchers

What if we only have Capybara in our Gemfile? Capybara doesn't define have_selector in its DSL, yet we can still use it in our tests. In this case, RSpec takes advantage of method_missing.

#method_missing will pick out all references to methods that start with 'have_' and create a new RSpec::Matchers::Has object. As we saw above, this object's #matches? method will be called, and in this case, the method will change 'have_selector' into 'has_selector?'. The matcher works since has_selector? is define in Capybara.

Implicit Matcher Process
page.should( have_selector('a') )
________________________________________________________________

# rspec-expectations/lib/rspec/matchers/method_missing.rb
module RSpec
  module Matchers
    private
    def method_missing(method, *args, &block) 
      return Matchers::BePredicate.new(method, *args, &block) if method.to_s =~ /^be_/
      return Matchers::Has.new(method, *args, &block) if method.to_s =~ /^have_/
      super
    end
________________________________________________________________

# rspec-expectations/lib/rspec/matchers/has.rb
module RSpec
  module Matchers
    class Has
      ...
      def matches?(actual)    # 'actual' here is 'page'
        actual.__send__(predicate(@expected), *@args)
      end
      ...
    private
    
      def predicate(sym)
        "#{sym.to_s.sub("have_","has_")}?".to_sym
      end

    Comments

  1. It's Looks nice tute to go with Capybara..!!
    By Arun Agrawal Jan 30, 2011 00:05
  2. Excellent write-up, one of the best I have seen so far -- it covers integration testing in depth and keeps up with the RSpec Book by Chelimsky, which I happened to review while working through your examples today. Keep up the good work! --Walter
    By Walter Yu, P.E., LEED AP Apr 03, 2011 00:03

(Please login to submit comments.)



View All Posts