Parallel progress output from concurrent threads

Question

Having this code:

workers = (1..3).map do |n|
    Thread.new do
        puts
        print "Worker_#{n}..."
        sleep rand
        print "done (#{n})"
    end
end.each &:join
puts
puts '- ready -'

how can we get a correct output like this (lines updated dynamically):

Worker_1...done (1)
Worker_2...done (2)
Worker_3...done (3)
- ready -

and not like this:

Worker_1...
Worker_2...
Worker_3...done (3)done (1)done (2)
- ready -

Advanced option: if we redirect stdout to file — it should look fine too.

You can maybe have a monitor thread which checks the status of each of the threads and posts it as a single message. If you want to clear the previous output messages each time you post an update, you can look at some of the options here https://stackoverflow.com/questions/4963717/how-to-overwrite-a-printed-line-in-the-shell-with-ruby — max pleaner, Jan 12 '22 at 21:47
Do you want the threads to all be running at the same time, but the outputs to not be interlaced? Do you want the output just non-interlaced or also in order 1,2,3? — rogerdpack, Jan 12 '22 at 22:05
Yes, all should run and update outputs in parallel. The order of lines can be different but not mixed (progress updates should be in the correct lines). — Inversion, Jan 13 '22 at 08:34

score 2 · Answer 1 · answered Jan 12 '22 at 22:01

I'd introduce some class to manage output. It will also move the carriage 3 lines above to update the output every time (at the print method).

class StatusPrint
  def initialize
    @hash = {}
  end

  def declare_worker(name)
    @hash[name] = "Worker_#{name}"
  end

  def complete_worker(name)
    @hash[name] += "...done (#{name})"
  end

  def print(move_carriage: true)
    puts "\033[#{@hash.size + 1}A" if move_carriage
    puts @hash.values.join("\n")
  end

  def done!
    @done = true
  end

  def done?
    @done
  end
end

So your code will look like this

status_print = StatusPrint.new

workers = (1..3).map do |n|
  Thread.new do
    status_print.declare_worker(n)
    sleep rand
    status_print.complete_worker(n)
  end
end

print_loop = Thread.new do
  status_print.print(move_carriage: false)

  until status_print.done?
    sleep 0.1
    status_print.print
  end
end

workers.each &:join
status_print.done!
print_loop.join

puts '- ready -'

Also it won't work if you want to pipe your output so you'll have to add some command line option to disable updating the output and print this static text:

Worker_1...done (1)
Worker_2...done (2)
Worker_3...done (3)
- ready -

Also see https://blog.stevenocchipinti.com/2013/06/removing-previously-printed-lines.html/ for more about updating the output.

This solution partially works. But it is not a common solution and it has a bug (chance to overwrite previous lines). But it is close to what I'm looking for. Thank you! — Inversion, Jan 13 '22 at 09:35

score 1 · Accepted Answer · answered Jan 30 '22 at 15:23

Eventually, I have created a gem to solve this problem:

ParaLines Ruby gem

Output sample:

Some minimal usage sample looks like this:

require 'paralines'

plines = ParaLines.new

plines.add_static_line 'Gems latest versions'
plines.add_empty_line

gems = %w[sinatra rspec paralines rails hanami]

plines.add_static_line '- Random order:'
threads = gems.map do |name|
    Thread.new do
        plines << name.ljust(15, '.')
        res = `gem list -r #{name} -e`
        plines << 'v' + res[/\((.+)\)/, 1]
    end
end
sleep 0.2
plines.add_empty_line
plines.add_static_line '- Sorted:'
threads += gems.sort.map do |name|
    sline = plines.add_shared_line name.ljust(15, '.')
    Thread.new do
        res = `gem list -r #{name} -e`
        sline << 'v' + res[/\((.+)\)/, 1]
    end
end

threads.each &:join

The source code for the detailed sample with output as on the GIF animation can be found on the gem page on the GitHub.

Parallel progress output from concurrent threads

2 Answers2