by

Configuring VCR to work with Selenium WebDriver

If you are using VCR and Selenium Webdriver in your project, there are a couple of common errors that you might see.

"Cassette" icon by Maxim Kulikov from the Noun Project.

The quick fix

Before I go into detail, this is the configuration I'm using for VCR on my current Rails project, which fixes the issues described below. Note the comments:

 1require "vcr"
 2
 3VCR.configure do |config|
 4  config.cassette_library_dir = "vcr_cassettes"
 5  config.hook_into :webmock
 6
 7  # Avoid conflict with Selenium
 8  config.ignore_localhost = true
 9
10  # Allow downloading webdrivers for Selenium
11  driver_hosts = Webdrivers::Common.subclasses.map { |driver| URI(driver.base_url).host }
12  # Downloading the Firefox driver involves a redirect
13  driver_hosts += ["github-releases.githubusercontent.com"]
14  config.ignore_hosts(*driver_hosts)
15end

Now on to the actual issues.

Problem 1: error talking to Selenium WebDriver

The first problem manifests as an error that looks a bit like this:

 1====================================================
 2An HTTP request has been made that VCR does not know how to handle:
 3  GET http://127.0.0.1:9516/something/something
 4
 5There is currently no cassette in use. There are a few ways
 6you can configure VCR to handle this request:
 7
 8[...The error message goes on...]
 9====================================================

The important bit here is the URL. Your code is trying to access something at http://127.0.0.1:9516, most probably while running a test. This is the local address used by Selenium to communicate with the browser in order to control it.

VCR captures all your HTTP requests, including some that you may not realise are taking place and you don't want to capture. This is one such example.

Typically you don't want VCR to capture requests to local addresses, as they are normally used by things that you do want to reach. To avoid capturing these, you can use the following setting with VCR:

1  # Avoid conflict with Selenium
2  config.ignore_localhost = true

Problem 2: error downloading a Selenium driver

This is an example of a related but different problem:

 1====================================================
 2An HTTP request has been made that VCR does not know how to handle:
 3  GET https://chromedriver.storage.googleapis.com/LATEST_RELEASE_90.0.4430
 4
 5There is currently no cassette in use. There are a few ways
 6you can configure VCR to handle this request:
 7
 8[...etc...]
 9====================================================

In this case, the request is being made behind the scenes by the gem webdrivers. If you are using Ruby on Rails, this gem is installed by default as of version 6 (in older versions it was chromedriver-helper).

This gem downloads drivers that Selenium can use to control your browser. This saves you having to download and install these drivers yourself. Of course since downloading something is an HTTP request, VCR also captures it, causing this error.

To fix this, we can tell VCR to ignore (ie: let through) requests to the host from where it's trying to download the driver. In the example above, you could do the following:

1  config.ignore_request do |request|
2    # Allow downloading Chromedriver
3    next URI(request.uri).host == "chromedriver.storage.googleapis.com"
4  end

This is not all though. In this example I'm assuming that your are using Chrome with Selenium, but you could be using a different browser. For example, if you are using Firefox, you'd see an error like this:

 1====================================================
 2An HTTP request has been made that VCR does not know how to handle:
 3  GET https://github.com/mozilla/geckodriver/releases/latest
 4
 5There is currently no cassette in use. There are a few ways
 6you can configure VCR to handle this request:
 7
 8[...etc...]
 9====================================================

For each variant of this issue, the URL will be suggestive of a specific browser. For each browser its driver is downloaded from a different location, so our VCR configuration needs to allow for any download URLs that we might need. For Firefox, the following would work. It involves two checks because there's a redirect to take into account:

1  config.ignore_request do |request|
2    uri = URI(request.uri)
3
4    # Don't interfere with the Webdrivers gem downloading GeckoDriver
5    next true if uri.to_s =~ %r{https://github.com/mozilla/geckodriver/releases/}
6    next true if uri.host == "github-releases.githubusercontent.com"
7  end

If you use more than one browser, you'll need to add matchers for their respective download URLs. If you want a comprehensive configuration that covers many browsers, this can get rather tiring rather soon.

Fortunately this issue was raised with the relevant gem maintainers in 2019 (read the conversation) and a solution was offered which is documented at the Webdrivers repository. My configuration at the top of this article is based on that solution. It assumes that you are using Rails (or at least ActiveSupport), but you can read the GitHub link if that's not the case for you.

But why "no cassette in use"?

There's something else interesting here: why does VCR say that there's "no cassette in use" if our code sets up a cassette? How come that VCR doesn't allow these requests, recording them like any other?

For example, in the second case: since we are using VCR, it would be reasonable to expect that VCR handles the request, allowing the download, and storing the it in a cassette file just like with any other HTTP request.

The downside of this behaviour would be the creation of cassette files that are several megabytes in size, but otherwise it would be consistent with what we expect from VCR. Why doesn't this happen?

The reason for this not to happen is that these requests (be it to control Selenium or to download drivers) take place before your cassette is in operation. For example, if you are using these tools in a Rails system test, the Webdrivers gem performs the download before the test even starts. At that point, your calls to VCR.use_cassette haven't happened yet and there's no cassette in use, just like the error message says.