Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add code for calculating stable identifiers #13

Merged
merged 1 commit into from
Feb 6, 2018
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .rubocop.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,3 +12,5 @@ Metrics/AbcSize:
Enabled: false
Lint/RescueException:
Enabled: false
Style/MixinUsage:
Enabled: false
19 changes: 19 additions & 0 deletions AWFUL-HACKS.md
Original file line number Diff line number Diff line change
Expand Up @@ -93,3 +93,22 @@ so I've monkey-patched their build system in our Rakefile in order
to make it error properly in this case.

Can be removed when: The linked issue is fixed.


### Stable identifiers from RSpec

Another "I did terrible things to RSpec" entry, sorry. RSpec's
design here is totally reasonable and sensible and honestly
probably *is* how you should pass state around, but seems
to make it impossible to get access to the Example object from
inside an it block without actually being the definer of the
block.

See `hypothesis_stable_identifier` for details, but basically I
couldn't figure out how to get the name of a currently executing
spec in RSPec from a helper function without some fairly brutal
hacks where we extract information badly from self.inspect, because
it's there stored as a string that gets passed in for inspection.

Can be removed when: Someone shows me a better way, or a
feature is added to RSpec to make this easier.
10 changes: 8 additions & 2 deletions Rakefile
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,9 @@ require 'bundler/setup'
require 'rspec/core/rake_task'
require 'rubocop/rake_task'
require 'helix_runtime/build_task'
require 'rake/testtask'

RSpec::Core::RakeTask.new(:test)
RSpec::Core::RakeTask.new(:spec)

RuboCop::RakeTask.new

Expand All @@ -24,7 +25,12 @@ end

HelixRuntime::BuildTask.new

task test: :build
Rake::TestTask.new(minitests: :build) do |t|
t.test_files = FileList['minitests/**/test_*.rb']
t.verbose = true
end

task test: %i[build spec minitests]

task :format do
sh 'bundle exec rubocop -a'
Expand Down
56 changes: 56 additions & 0 deletions lib/hypothesis.rb
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,62 @@
require 'hypothesis/debug'

module Hypothesis
HYPOTHESIS_LOCATION = File.dirname(__FILE__)

def hypothesis_stable_identifier
# Attempt to get a "stable identifier" for any given
# call into hypothesis. We use these to create
# database keys (or will when we have a database) that
# are stable across runs, so that when a test that
# previously failed is rerun, we can fetch and reuse
# the previous examples.

# Note that essentially any answer to this method is
# "fine" in that the failure mode is that sometiems we
# just won't run the same test, but it's nice to keep
# this as stable as possible if the code isn't changing.

# Minitest makes it nice and easy to create a stable
# test identifier, because it follows the classic xunit
# pattern where a test is just a method invocation on a
# fresh test class instance and it's easy to find out
# which invocation that was.
return "#{self.class.name}::#{@NAME}" if defined? @NAME

# If we are running in an rspec example then, sadly,
# rspec take the entirely unreasonable stance that
# the correct way to pass data to a test is by passing
# it as a function argument. Honestly, what is this,
# Haskell? Ahem. Perfectly reasonable design decisions
# on rspec's part, this creates some annoying difficulties
# for us. We solve this through brute force and ignorance
# by relying on the information we want being in the
# inspect for the Example object, even if it's just there
# as a string.
begin
is_rspec = is_a? RSpec::Core::ExampleGroup
rescue NameError
is_rspec = false
end

if is_rspec
return [
self.class.description,
inspect.match(/"([^"]+)"/)[1]
].join(' ')
end

# Fallback time! We just walk the stack until we find the
# entry point into code we control. This will typically be
# where "hypothesis" was called.
Thread.current.backtrace.each do |line|
return line unless line.include?(Hypothesis::HYPOTHESIS_LOCATION)
end
# This should never happen unless something very strange is
# going on.
raise 'BUG: Somehow we have no caller!'
end

def hypothesis(options = {}, &block)
unless World.current_engine.nil?
raise UsageError, 'Cannot nest hypothesis calls'
Expand Down
12 changes: 12 additions & 0 deletions minitests/test_stable_identifier.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
# frozen_string_literal: true

require 'minitest/autorun'
require 'hypothesis'

class TestIdentifiers < Minitest::Test
include Hypothesis

def test_abc
assert_equal hypothesis_stable_identifier, 'TestIdentifiers::test_abc'
end
end
22 changes: 22 additions & 0 deletions spec/stable_identifier_spec.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
# frozen_string_literal: true

class SomeClass
include Hypothesis
def stuff
hypothesis_stable_identifier
end
end

RSpec.describe 'stable identifiers' do
it 'are the full rspec string' do
expect(hypothesis_stable_identifier).to eq(
'stable identifiers are the full rspec string'
)
end

it 'fall back to a traceback' do
ident = SomeClass.new.stuff
expect(ident).to include(__FILE__)
expect(ident).to include('6')
end
end