Testing for errors

Tests are written to prevent errors from happening. The experienced programmer knows that errors are inevitable, and seeks to anticipate them by writing tests that deal specifically with errors.

There are three basic cases to deal with when testing errors:

• no error is raised
• an external error (an error class not in the code under test) is raised
• an internal error (a custom error class in the code under test) is raised

There are two basic decisions to make when writing code that raises an error.

The first is whether to allow an error to be raised or to attempt to recover from it with defensive practices, such as using a rescue block or fixing inputs that could cause an error to be raised. In general, lower-level, library code should expose errors without trying to recover from them, allowing the consumer of the code to handle error cases on their own. Higher-level application code should strive to recover from errors more aggressively, allowing only truly unrecoverable errors to be raised. For example, in a web application, we may have a page that retrieves the current weather using a WeatherQuery library that retrieves weather information from an HTTP service. If the HTTP service is unavailable, we will get an error when trying to retrieve weather information from it. The WeatherQuery class should raise an error to let us know something is wrong. And to avoid showing our end user an ugly error page, our application code, most likely the controller, should recover from the error by redirecting to the home page and displaying a friendly error message to the user.

The second decision to make is to determine when to create our own custom error classes instead of relying on errors raised by other code (defined in other parts of our code, an external gem, or the Ruby standard library). One good indicator of a need for custom classes is test code that checks for multiple existing errors. Often, many existing errors can be grouped together into a custom error that is more semantically meaningful. For example, in our example weather web app, we could anticipate a number of possible network-related errors, such as an unavailable network, a request timeout, unexpected response format, or missing data in the response, each of which would be a different error class. We can group all these errors into a custom NetworkError class, allowing consumers of the WeatherQuery class to handle a single error class, instead of requiring them to know about the internal implementation details of WeatherQuery so that they can handle the various errors that could be raised. In our controller, we would then only have to recover from WeatherQuery::NetworkError.

We'll start with the following WeatherQuery module, which provides weather forecasts using the openweathermap.org API:

require 'net/http'
require 'json'
require 'timeout'

module WeatherQuery
  NetworkError = Class.new(StandardError)

  class << self
    def forecast(place)
      JSON.parse( http(place) )
    rescue JSON::ParserError
      raise NetworkError.new("Bad response")
    end

    private

    BASE_URI = 'http://api.openweathermap.org/data/2.5/weather?q='
    def http(place)
      uri = URI(BASE_URI + place)

      Net::HTTP.get(uri)
    rescue Timeout::Error
      raise NetworkError.new("Request timed out")
    rescue URI::InvalidURIError
      raise NetworkError.new("Bad place name: #{place}")
    rescue SocketError
      raise NetworkError.new("Could not reach #{uri.to_s}")
    end
  end
end

Our code anticipates network-related errors and can guarantee the user (the code that calls WeatherQuery.forecast) that either a valid response will be returned or a WeatherQuery::NetworkError will be raised. With this guarantee, the user doesn't have to worry about unexpected errors being raised by WeatherQuery due to network-related issues or problems with the openweathermap.org API service. How did we achieve this guarantee? First, we've extracted the network interactions into a private http method. Second, within this method, we've anticipated the different errors that could occur. We got this list through research as well as trial and error. Although the documentation for Net::HTTP#get states that it never raises an exception, the truth is that an exception will be raised in a number of common scenarios when the network request can't be made (for example, no response from the server, invalid URI, or network outage).

Now let's ensure that our guarantee is valid and test that, WeatherQuery.forecast will return valid data or raise a WeatherQuery::NetworkError exception.

To begin with, let's write a test to verify that a WeatherQuery::NetworkError is raised when a timeout occurs:

describe WeatherQuery do
  describe '.forecast' do
    it "raises a NetworkError instead of Timeout::Error" do
      expect(Net::HTTP).to receive(:get).and_raise(Timeout::Error)

      expected_error   = WeatherQuery::NetworkError
      expected_message = "Request timed out"

      expect{
        WeatherQuery.forecast("Antarctica")
      }.to raise_error(expected_error, expected_message)
    end
  end
end

We've used and_raise to mock the timeout and checked that the external timeout error is rescued and converted to an internal WeatherQuery::NetworkError using the raise_error matcher. That's all there is to it! This simple technique can add a new dimension to your specs and code. In fact, it is generally more important to write tests targeted at error cases than tests for the "happy path", since understanding your application's behavior in the event of exceptions is key to improving its robustness.

To test for exceptions being raised (or not raised), we used RSpec's raise_error matcher, which works with the block form of expect, as follows:

it "raises an error" do
  expect{ 1/0 }.to raise_error
  expect{ 1/0 }.to raise_error(ZeroDivisionError)
  expect{ 1/0 }.to raise_error(ZeroDivisionError, /pided/)
  expect{ 1/0 }.to raise_error(ZeroDivisionError, "pided by 0")

  expect{ 1/0 }.to raise_error do |e|
    expect(e.message).to eq("pided by 0")
  end
end

it "does not raise an error" do
  expect{
    1/1
  }.to_not raise_error
end

As you can see, you can match the expected error with as much precision as you like. For example, you can match any error, a specific class of error, an error message with a regular expression or exact match, as shown in the preceding code. You can also pass a block to raise_error for total control (in the case above, we check only for an error message regardless of class). One important point to note is that a negative assertion should only be used to assert that no error of any kind is raised. Why? Because checking that a specific kind of error has not been raised will very likely lead to false positives (see https://github.com/rspec/rspec-expectations/issues/231).

The error we've tested for the preceding code block, ZeroDivisionError, is easy to trigger. But many errors are not as easy to trigger. How would we recreate timeouts or network outages? We could try to write our own code that does this, but RSpec gives us the ability to mock errors using and_raise, as follows:

it "raises an error" do
  expect(Net::HTTP).to receive(:get).and_raise(Timeout::Error)

  # will raise Timeout::Error
  Net::HTTP.get('http://example.org')
end

The and_raise method allows us to specify the mock error with as much specificity as we like.

We'll also need a test that covers the case where JSON is not returned by the API, which commonly occurs when a server is down. Putting it all together, we have the following:

describe WeatherQuery do
  describe '.forecast' do
    context 'network errors' do
      let(:custom_error) { WeatherQuery::NetworkError }
      
      before do
        expect(Net::HTTP).to receive(:get)
                             .and_raise(err_to_raise)
      end

      context 'timeouts' do
        let(:err_to_raise) { Timeout::Error }
        
        it 'handles the error' do
          expect{
            WeatherQuery.forecast("Antarctica")
          }.to raise_error(custom_error, "Request timed out")          
        end
      end

      context 'invalud URI' do
        let(:err_to_raise) { URI::InvalidURIError }
        
        it 'handles the error' do
          expect{
            WeatherQuery.forecast("Antarctica")
          }.to raise_error(custom_error, "Bad place name: Antarctica")          
        end
      end        

      context 'socket errors' do
        let(:err_to_raise) { SocketError }
        
        it 'handles the error' do
          expect{
            WeatherQuery.forecast("Antarctica")
          }.to raise_error(custom_error, /Could not reach http:\/\//)          
        end
      end        
    end

    let(:xml_response) do
      %q(
        <?xml version="1.0" encoding="utf-8"?>
        <current>
          <weather number="800" value="Sky is Clear" icon="01n"/>
        </current>
      )
    end

    it "raises a NetworkError if response is not JSON" do
      expect(WeatherQuery).to receive(:http)
        .with('Antarctica')
        .and_return(xml_response)

      expect{
        WeatherQuery.forecast("Antarctica")
      }.to raise_error(
        WeatherQuery::NetworkError, "Bad response"
      )
    end
  end
  end

We used the and_return method, which we haven't learned about yet. We'll cover that in depth in Chapter 3, Taking Control of State with Doubles and Hooks. All we need to know here is that we need it to prevent an actual HTTP request being made. Now we've covered every possible error case. We still have some choices to make about our overall design. If we pass a place name with a space to WeatherQuery.forecast, we'll get an error because the space is not URL-encoded. Should we detect bad input and raise an error before making the API request? Or should we try to handle spaces in the input by URL-encoding the input? Should we go further and add a validator to check the input for parsable formats and raise a new kind of error (for example, WeatherQuery::UnparseablePlace) if the input can't be parsed? I've chosen to keep the code simple by doing no validation or normalization of the input. This approach works when you can expect your users to be familiar with the workings of the backend API. If that expectation is not reasonable for your use case, then you should consider validation, normalization, or both.

What about the "happy path"? In this case, we have no tests for it, and we don't really need any. Our code in this case is designed as a simple pass-through to the API. We are not changing the input or the output from the API, except to convert the JSON response into a Ruby object. What about actually hitting the API service to make sure everything works? That is a good idea but outside the scope of unit testing, the focus of this chapter. It is a very bad idea to allow any network connections in unit tests, so this type of test can be done separately in integration tests, which we will cover in detail in Chapter 5, Simulating External Services.