RSpec crashes due to bad `message` of exception
Created by: tomdalling
Subject of the issue
When the message
of a raised exception does not implement #to_s
properly, the RSpec exception formatter crashes instead of reporting a test failure.
The lack of report output is causing downstream problems in the mutant
gem.
Your environment
- Ruby version: 2.7.1
- rspec-core version: 3.9.2
Steps to reproduce
# frozen_string_literal: true
begin
require "bundler/inline"
rescue LoadError => e
$stderr.puts "Bundler version 1.10 or later is required. Please update your Bundler"
raise e
end
gemfile(true) do
source "https://rubygems.org"
gem "rspec", "3.9.0"
end
puts "Ruby version is: #{RUBY_VERSION}"
require 'rspec/autorun'
class Thing
def to_s
raise StandardError, self
end
end
RSpec.describe Thing do
it 'raises an error in #to_s' do
thing = Thing.new
expect { thing.to_s }.to raise_error(ArgumentError)
end
end
Expected behavior
The test should fail and output an error report similar to:
Fetching gem metadata from https://rubygems.org/...
Resolving dependencies...
Using bundler 2.1.4
Using diff-lcs 1.4.4
Using rspec-support 3.9.3
Using rspec-core 3.9.2
Using rspec-expectations 3.9.2
Using rspec-mocks 3.9.1
Using rspec 3.9.0
Ruby version is: 2.7.1
F
Failures:
1) Thing raises an error in #to_s
Failure/Error: expect { thing.to_s }.to raise_error(ArgumentError)
expected ArgumentError, got #<StandardError: StandardError> with backtrace:
# tmp.rb:21:in `to_s'
# tmp.rb:28:in `block (3 levels) in <main>'
# tmp.rb:28:in `block (2 levels) in <main>'
# tmp.rb:28:in `block (2 levels) in <main>'
Finished in 0.01809 seconds (files took 0.09742 seconds to load)
1 example, 1 failure
Failed examples:
rspec tmp.rb:26 # Thing raises an error in #to_s
Actual behavior
RSpec crashes without outputting an error report:
Fetching gem metadata from https://rubygems.org/...
Resolving dependencies...
Using bundler 2.1.4
Using diff-lcs 1.4.4
Using rspec-support 3.9.3
Using rspec-core 3.9.2
Using rspec-expectations 3.9.2
Using rspec-mocks 3.9.1
Using rspec 3.9.0
Ruby version is: 2.7.1
F
tmp.rb:21:in `to_s': StandardError
from /Users/tom/.gem/ruby/2.7.1/gems/rspec-core-3.9.2/lib/rspec/core/formatters/exception_presenter.rb:180:in `to_s'
from /Users/tom/.gem/ruby/2.7.1/gems/rspec-core-3.9.2/lib/rspec/core/formatters/exception_presenter.rb:180:in `message'
from /Users/tom/.gem/ruby/2.7.1/gems/rspec-core-3.9.2/lib/rspec/core/formatters/exception_presenter.rb:180:in `exception_lines'
from /Users/tom/.gem/ruby/2.7.1/gems/rspec-core-3.9.2/lib/rspec/core/formatters/exception_presenter.rb:155:in `block in failure_lines'
from /Users/tom/.gem/ruby/2.7.1/gems/rspec-core-3.9.2/lib/rspec/core/formatters/exception_presenter.rb:152:in `tap'
from /Users/tom/.gem/ruby/2.7.1/gems/rspec-core-3.9.2/lib/rspec/core/formatters/exception_presenter.rb:152:in `failure_lines'
from /Users/tom/.gem/ruby/2.7.1/gems/rspec-core-3.9.2/lib/rspec/core/formatters/exception_presenter.rb:34:in `colorized_message_lines'
from /Users/tom/.gem/ruby/2.7.1/gems/rspec-core-3.9.2/lib/rspec/core/formatters/exception_presenter.rb:243:in `formatted_message_and_backtrace'
from /Users/tom/.gem/ruby/2.7.1/gems/rspec-core-3.9.2/lib/rspec/core/formatters/exception_presenter.rb:88:in `fully_formatted_lines'
from /Users/tom/.gem/ruby/2.7.1/gems/rspec-core-3.9.2/lib/rspec/core/formatters/exception_presenter.rb:80:in `fully_formatted'
from /Users/tom/.gem/ruby/2.7.1/gems/rspec-core-3.9.2/lib/rspec/core/notifications.rb:200:in `fully_formatted'
from /Users/tom/.gem/ruby/2.7.1/gems/rspec-core-3.9.2/lib/rspec/core/notifications.rb:114:in `block in fully_formatted_failed_examples'
from /Users/tom/.gem/ruby/2.7.1/gems/rspec-core-3.9.2/lib/rspec/core/notifications.rb:113:in `each'
from /Users/tom/.gem/ruby/2.7.1/gems/rspec-core-3.9.2/lib/rspec/core/notifications.rb:113:in `each_with_index'
from /Users/tom/.gem/ruby/2.7.1/gems/rspec-core-3.9.2/lib/rspec/core/notifications.rb:113:in `fully_formatted_failed_examples'
from /Users/tom/.gem/ruby/2.7.1/gems/rspec-core-3.9.2/lib/rspec/core/formatters/base_text_formatter.rb:32:in `dump_failures'
from /Users/tom/.gem/ruby/2.7.1/gems/rspec-core-3.9.2/lib/rspec/core/reporter.rb:209:in `block in notify'
from /Users/tom/.gem/ruby/2.7.1/gems/rspec-core-3.9.2/lib/rspec/core/reporter.rb:208:in `each'
from /Users/tom/.gem/ruby/2.7.1/gems/rspec-core-3.9.2/lib/rspec/core/reporter.rb:208:in `notify'
from /Users/tom/.gem/ruby/2.7.1/gems/rspec-core-3.9.2/lib/rspec/core/reporter.rb:178:in `block in finish'
from /Users/tom/.gem/ruby/2.7.1/gems/rspec-core-3.9.2/lib/rspec/core/reporter.rb:194:in `close_after'
from /Users/tom/.gem/ruby/2.7.1/gems/rspec-core-3.9.2/lib/rspec/core/reporter.rb:174:in `finish'
from /Users/tom/.gem/ruby/2.7.1/gems/rspec-core-3.9.2/lib/rspec/core/reporter.rb:76:in `report'
from /Users/tom/.gem/ruby/2.7.1/gems/rspec-core-3.9.2/lib/rspec/core/runner.rb:115:in `run_specs'
from /Users/tom/.gem/ruby/2.7.1/gems/rspec-core-3.9.2/lib/rspec/core/runner.rb:89:in `run'
from /Users/tom/.gem/ruby/2.7.1/gems/rspec-core-3.9.2/lib/rspec/core/runner.rb:71:in `run'
from /Users/tom/.gem/ruby/2.7.1/gems/rspec-core-3.9.2/lib/rspec/core/runner.rb:45:in `invoke'
from /Users/tom/.gem/ruby/2.7.1/gems/rspec-core-3.9.2/lib/rspec/core/runner.rb:38:in `perform_at_exit'
from /Users/tom/.gem/ruby/2.7.1/gems/rspec-core-3.9.2/lib/rspec/core/runner.rb:24:in `block in autorun'