watir-nokogiri

At Test Automation Bazaar Aliaksandr Ikhelis gave a lightning talk Watir-Webdriver is slow (did you know that?). He talked about the speed problem he had when watir-webdriver was locating elements via regular expression on big pages. (He also reported the issue at GitHub.)

Recently I have started working on a new project that had a similar speed problem, so I remembered Aliaksandr's talk. I found a solution using Nokogiri.

For example, you have to iterate over all div elements that have id attribute starting with id. Then, you have to do something with the last element.

The page that looks like this:

...
<div id='id0'>0</div>
<div id='id1'>1</div>
...
<div id='id999'>999</div>
...

Watir code for iterating over elements and clicking the last one could look something like this.

browser.divs(id: /^id/).each do |div|
  div.click if div.id == "id999"
end

You could do exactly the same thing using Watir and Nokogiri.

Nokogiri::HTML(browser.html).css("div[id ^= 'id']").each do |div|
  browser.element(css: div.css_path).click if div["id"] == "id999"
end

The speed improvements is noticeable. For example, this script will click the last div using Nokogiri in about 0.2 seconds and using Watir in about 15 seconds.

require "watir-webdriver"
browser = Watir::Browser.new :firefox

folder = Dir.pwd
browser.goto "file://#{folder}/div.html"

require "benchmark"
require "nokogiri"

Benchmark.bm do |x|
  x.report("nokogiri") do
    Nokogiri::HTML(browser.html).css("div[id ^= 'id']").each do |div|
      if div["id"] == "id999"
       css = div.css_path
       browser.element(css: css).wd.location_once_scrolled_into_view
       browser.element(css: css).flash
      end
    end
  end
end

Benchmark.bm do |x|
  x.report("watir") do
    browser.divs(id: /^id/).each do |div|
      if div.id == "id999"
        div.wd.location_once_scrolled_into_view
        div.flash
      end
    end
  end
end

browser.close

Output:

          user       system     total    real
nokogiri  0.040000   0.010000   0.050000 (  0.219716)
watir     1.980000   0.310000   2.290000 ( 15.309858)

There is a way to speed things up using just the Watir API, but I wanted to provide an example code if Watir API is sometimes too slow for your taste.

You can find more code at watir-nokogiri.