What is the best practice when it comes to testing "infinite loops"?

Question

My basic logic is to have an infinite loop running somewhere and test it as best as possible. The reason for having an infinite loop is not important (main loop for games, daemon-like logic...) and I'm more asking about best practices regarding a situation like that.

Let's take this code for example:

module Blah
  extend self

  def run
     some_initializer_method
     loop do
        some_other_method
        yet_another_method
     end
  end
end

I want to test the method Blah.run using Rspec (also I use RR, but plain rspec would be an acceptable answer).

I figure the best way to do it would be to decompose a bit more, like separating the loop into another method or something:

module Blah
  extend self

  def run
     some_initializer_method
     do_some_looping
  end

 def do_some_looping
   loop do
     some_other_method
     yet_another_method
   end
 end
end

... this allows us to test run and mock the loop... but at some point the code inside the loop needs to be tested.

So what would you do in such a situation?

Simply not testing this logic, meaning test some_other_method & yet_another_method but not do_some_looping ?

Have the loop break at some point via a mock?

... something else?

as an aside i cannot think of one good reason to write an infinite loop. even in a game or long-living processing kernel, you have an end condition - eg, program termination, often signalled by separate component like user input or system command. we should always write our loops to terminate gracefully when this condition is met. having said that, infinite loops are certainly *possible* to write, so valid question. party on wayne. — johnny g, Apr 19 '11 at 14:26
@johnny yeah I know, you could probably say that the exit condition of this loop is when the process gets killed from another process. It's not a situation that I encounter everyday, but I'm still wondering how people would approach it — marcgg, Apr 19 '11 at 14:31
@johnny What would you say if I'd add a method that would be the exit condition (even if this method would always return false) and then mock it to get out of the loop? — marcgg, Apr 19 '11 at 14:35
haha, cool i suppose. not too familiar with Ruby stuff, so not too sure how or to what extent we are capable of "mocking" things (in C# world we are typically limited to interfaces). if this is possible, definitely give it a go. i've written similar things before - but for it to be successful, you require either a) a condition within your game loop that will trigger the mock to return true (and exit), or b) an async thread to fire on a condition external to loop that will will trigger the mock to return true (and exit). obviously, weigh cost of implementation to value of test :) — johnny g, Apr 19 '11 at 17:04
actually, similar to this whole concept of mocking, what you really want is for a) ability for end condition to be infinite in normal execution context, and b) ability for end condition to be finite (some logical condition or static definition eg 1 cycle, 3 cycles, etc). we could use principal of Inversion of Control and Dependency Injection to *pass in* our condition on class's constructor. so in a unit test, we specify our finite condition, in normal mode, we pass in infinite. ahem, pretty much what you suggest earlier i suppose :P — johnny g, Apr 19 '11 at 17:07

score 15 · Answer 1 · answered Apr 19 '11 at 14:25

15

What might be more practical is to execute the loop in a separate thread, assert that everything is working correctly, and then terminate the thread when it is no longer required.

thread = Thread.new do
  Blah.run
end

assert_equal 0, Blah.foo

thread.kill

answered Apr 19 '11 at 14:25

tadman

208,517
23
234
262

That's an interesting approach, good idea! Wouldn't it be more complicated to test the outputs of the method Blah.run if it gets ran in a new thread thought? – marcgg Apr 19 '11 at 14:28
1

I've been writing unit tests for `eventmachine` based engines and this is really the only way to do it, as an event loop is infinite by definition. Remember that threads share data, processes do not, so any class or module instance variables are easy to access. – tadman Apr 19 '11 at 15:42
@tadman Could you point me to some examples of the kind of application tests you are talking about? – rpattabi Dec 17 '11 at 14:32

score 12 · Answer 2 · answered Oct 26 '15 at 14:40

12

in rspec 3.3, add this line

allow(subject).to receive(:loop).and_yield

to your before hook will simple yield to the block without any looping

answered Oct 26 '15 at 14:40

Rundong Gao

121
1
2

score 10 · Answer 3 · answered Apr 19 '11 at 14:25

10

How about having the body of the loop in a separate method, like calculateOneWorldIteration? That way you can spin the loop in the test as needed. And it doesn’t hurt the API, it’s quite a natural method to have in the public interface.

answered Apr 19 '11 at 14:25

zoul

102,279
44
260
354

1

It'd work, but then I'd never really call the actual loop that would never get tested. It's not a big of a deal, but it's still bothering me slightly ^^ – marcgg Apr 19 '11 at 14:26
The actual loop would be just a tiny wrapper around this method, with nothing to test. But maybe this depends on your loop exit conditions? – zoul Apr 19 '11 at 14:28
the loop is just loop do {}, no exit conditions. It's just that I like my 100% code coverage, and be sure that I didn't make any mistakes even on tiny wrappers. I like @tadman idea for that matter, I'm still pondering if it's not overkill ^^ – marcgg Apr 19 '11 at 14:29
1

If I were to pick between running threads in my tests and having one `while (1) {…}` not covered, I would not hesitate for a second :) – zoul Apr 19 '11 at 14:32
It does make sense, yeah ^^! Just fmi why would you be particularly against threads in your tests? – marcgg Apr 19 '11 at 14:33
They are a source of non-deterministic behaviour, and I like my tests to be simple and predictable. Sure there are cases for [threads in tests](http://stackoverflow.com/questions/3896952), but I would not introduce them unless I really had to. – zoul Apr 19 '11 at 14:41

score 3 · Answer 4 · answered Apr 09 '12 at 00:47

You can not test that something that runs forever.

When faced with a section of code that is difficult (or impossible) to test you should:-

Refactor to isolate the difficult to test part of the code. Make the untestable parts tiny and trivial. Comment to ensure they are not later expanded to become non-trivial
Unit test the other parts, which are now separated from the difficult to test section
The difficult to test part would be covered by an integration or acceptance test

If the main loop in your game only goes around once, this will be immediately obvious when you run it.

2potatocakes · Answer 5 · 2012-04-09T01:38:02.230

I know this is a little old, but you can also use the yields method to fake a block and pass a single iteration to a loop method. This should allow you to test the methods you're calling within your loop without actually putting it into an infinite loop.

require 'test/unit'
require 'mocha'

class Something
  def test_method
    puts "test_method"
    loop do
      puts String.new("frederick")
    end
  end
end

class LoopTest < Test::Unit::TestCase

  def test_loop_yields
    something = Something.new
    something.expects(:loop).yields.with() do
      String.expects(:new).returns("samantha")
    end
    something.test_method
  end
end

# Started
# test_method
# samantha
# .
# Finished in 0.005 seconds.
#
# 1 tests, 2 assertions, 0 failures, 0 errors

score 2 · Answer 6 · answered Apr 19 '11 at 14:46

2

What about mocking the loop so that it gets executed only the number of times you specify ?

Module Object
    private
    def loop
        3.times { yield }
    end
end

Of course, you mock this only in your specs.

answered Apr 19 '11 at 14:46

Damien MATHIEU

31,924
13
86
94

cool idea. pretty obscure for someone who would lookup the tests for the first time but still cool ^^ – marcgg Apr 19 '11 at 14:49

Kelvin · Answer 7 · 2012-09-13T16:13:47.583

I almost always use a catch/throw construct to test infinite loops.

Raising an error may also work, but that's not ideal especially if your loop's block rescue all errors, including Exceptions. If your block doesn't rescue Exception (or some other error class), then you can subclass Exception (or another non-rescued class) and rescue your subclass:

Exception example

Setup

class RspecLoopStop < Exception; end

Test

blah.stub!(:some_initializer_method)
blah.should_receive(:some_other_method)
blah.should_receive(:yet_another_method)
# make sure it repeats
blah.should_receive(:some_other_method).and_raise RspecLoopStop

begin
  blah.run
rescue RspecLoopStop
  # all done
end

Catch/throw example:

blah.stub!(:some_initializer_method)
blah.should_receive(:some_other_method)
blah.should_receive(:yet_another_method)
blah.should_receive(:some_other_method).and_throw :rspec_loop_stop

catch :rspec_loop_stop
  blah.run
end

When I first tried this, I was concerned that using should_receive a second time on :some_other_method would "overwrite" the first one, but this is not the case. If you want to see for yourself, add blocks to should_receive to see if it's called the expected number of times:

blah.should_receive(:some_other_method) { puts 'received some_other_method' }

score 1 · Answer 8 · answered Mar 05 '14 at 02:39

Our solution to testing a loop that only exits on signals was to stub the exit condition method to return false the first time but true the second time, ensuring the loop is only executed once.

Class with infinite loop:

class Scheduling::Daemon
  def run
    loop do
      if daemon_received_stop_signal?
        break
      end

      # do stuff
    end
  end
end

spec testing the behaviour of the loop:

describe Scheduling::Daemon do
  describe "#run" do
    before do
      Scheduling::Daemon.should_receive(:daemon_received_stop_signal?).
        and_return(false, true)  # execute loop once then exit
    end      

    it "does stuff" do
      Scheduling::Daemon.run  
      # assert stuff was done
    end
  end
end

score 0 · Answer 9 · answered Apr 09 '12 at 12:41

0

:) I had this query a few months ago.

The short answer is there is no easy way to test that. You test drive the internals of the loop. Then you plonk it into a loop method & do a manual test that the loop works till the terminating condition occurs.

answered Apr 09 '12 at 12:41

Gishu

134,492
47
225
308

score 0 · Answer 10 · answered Dec 22 '12 at 14:07

The easiest solution I found is to yield the loop one time and than return. I've used mocha here.

require 'spec_helper'
require 'blah'

describe Blah do
  it 'loops' do
    Blah.stubs(:some_initializer_method)
    Blah.stubs(:some_other_method)
    Blah.stubs(:yet_another_method)

    Blah.expects(:loop).yields().then().returns()

    Blah.run
  end
end

We're expecting that the loop is actually executed and it's ensured it will exit after one iteration.

Nevertheless as stated above it's good practice to keep the looping method as small and stupid as possible.

Hope this helps!

What is the best practice when it comes to testing "infinite loops"?

10 Answers10

Linked