How to test that an element is randomly selected from a list?

Question

I'm working on a Rails application and trying to practice TDD (using RSpec). I have a file in my lib directory that contains a list of strings, and a method that will read that file and randomly select one of the strings from the list. I haven't implemented this method yet because I'm struggling with how to write a test that functionality.

There are lots of ways to randomly select an object from an array, and lots of great answered questions like this one on here that tell me how to do that (when it comes down to the implementation, I'll probably use Array#sample). But what should my expectation be? I'm thinking something like:

expect(array).to include(subject.random_select)

This will certainly assert that some expected value is returned from my method — but is it enough to assert that the method randomly returns a different string each time? What would be some alternatives, or perhaps additional tests that would ensure I've got coverage for this method? I can't really expect subject.random_select to equal a faked input, can I?

Answer 1

I would first test the non-random selection of a single string from a one-line file, then I'd test selection of a string from a multi-line file, then I'd test that the selection is random. You can't really test randomness in finite time, so the best you can do is to

• test that your method returns a value in the desired range, knowing that since your test will run many, many times over the lifetime of your app you'll probably find out if it ever returns something out of range, and
• prove that your code uses a source of randomness.

Let's say that the file doesn't exist in your test environment, or you don't know its contents, or don't want the asymmetry of having it be correct for one test and incorrect for others, so we'll need to provide a way for tests to point the class at different files.

We could write the following, writing one test at a time, making it pass and refactoring before writing the next. Here are the tests and code after the third test is written but before it's been implemented:

spec/models/thing_spec.rb

describe Thing do
  describe '.random_select' do
    it "returns a single line from a file with only one line" do
      allow(Thing).to receive(:file) { "spec/models/thing/1" }
      expect(Thing.random_select).to eq("Thing 1")
    end

    it "returns a single line from a file with multiple lines" do
      allow(Thing).to receive(:file) { "spec/models/thing/2" }
      expect(Thing.random_select).to be_in(['Thing 1', 'Thing 2'])
    end

    it "returns different lines at different times" do
      allow(Thing).to receive(:file) { "spec/models/thing/2" }
      srand 0
      thing1 = Thing.random_select
      srand 1
      thing2 = Thing.random_select
      expect(thing1).not_to eq(thing2)
    end

  end
end

app/models/thing.rb

class Thing
  def self.random_select
    "Thing 1" # this made the first two tests pass, but it'll need to change for all three to pass
  end

  def self.file
    "lib/things"
  end

end

When I wrote the second test I realized that it passed without any additional code changes, so I considered deleting it. But I deferred that decision, wrote the third test, and discovered that once the third test passes the second will have value, since the second test tests that the value comes from the file but the third test does not.

be_in is a nicer way to test that the return value is in a known set than include since it puts the actual value inside expect where RSpec expects it.

There are other ways to control the randomness so you can test that it's used. For example, if you used sample you could allow_any_instance_of(Array).to receive(:sample) and return whatever you like. But I like using srand since it doesn't require the implementation to use a specific method that uses the random number generator.

If the file can be missing or empty you'll need to test that too.

Answer 2

Here are a few ideas about how I might test such a thing. I might not do all of them for a simple function like this, but they're still good techniques for more complex cases:

1. Separate out the job of reading the entries into an array, and the job of choosing a random element from the array.

2. If you don't want to have to rely on the filesystem in your tests, you can write your method to work with a general IO object (or a stream in other languages) and then use a StringIO in your tests. You'd have a simple forwarding function that opens the file and passes the open file to the method that works with an IO. For example:

   def read_entries(file)
     File.open(file) { |io| read_entries_from_io(io) }
   end
   
   def read_entries_from_io(io)
     # ... do the work ...
   end
   
   # In your spec:
   io = StringIO.new("Entry1\nEntry2\nEntry3\n")
   expect(read_entries_from_io(io)).to eq %w[Entry1 Entry2 Entry3]
   

3. Pretty much every Ruby method that does something random (like sample and shuffle) takes an optional random: keyword argument, which allows you to supply your own random number generator. If your method follows the same convention, you can then inject a fake random number generator from your tests that returns a hard-coded sequence of fake random numbers:

   def random_select(entries, random: Random.new)
     entries.sample(random: random)
   end
   
   # In the spec:
   
   entries = %w[Entry0 Entry1 Entry2 Entry3]
   random = instance_double(Random)
   allow(random).to receive(:rand).and_return(2, 0)
   
   expect(random_select(entries, random: random)).to eq 'Entry2'
   expect(random_select(entries, random: random)).to eq 'Entry0'
   

NOTE: I wouldn't actually have two expectations in that last test; I just included it to show how you could return a sequence of random values.

Also, in this case, the test is making an assumption about how sample is using the random-number generator. That might be ok with sample, but probably not with shuffle.

Answer 3

It looks like you are on the right track. I would additionally include a fake "sample" from the array and expect the array to not include that sample.

See these docs, for example: expect(array).not_to include("fake sample")