Can I get continuous output from system calls in Ruby?

Question

When you use a system call in a Ruby script, you can get the output of that command like this:

output = `ls`
puts output

That's what this question was about.

But is there a way to show the continuous output of a system call? For example, if you run this secure copy command, to get a file from a server over SSH:

scp user@someserver:remoteFile /some/local/folder/

... it shows continuous output with the progress of the download. But this:

output = `scp user@someserver:remoteFile /some/local/folder/`
puts output

... doesn't capture that output.

How can I show the ongoing progress of the download from inside my Ruby script?

you had two orthogonal problems here, because scp only outputs to terminals by default, you need *scp -v* — tokland, Mar 04 '11 at 14:09
@tokland - that gets it to output debugging messages, but not the transfer progress that I would see if I just ran scp alone. I think those must not go to standard output, and I don't see an option for scp to send them there. — Nathan Long, Mar 04 '11 at 14:39
Apparently scp sends that progress info to "interactive terminal"? Not sure how to capture that... — Nathan Long, Mar 04 '11 at 14:45

tokland · Accepted Answer · 2014-10-31T09:19:04.347

9

Try:

IO.popen("scp -v user@server:remoteFile /local/folder/").each do |fd|
  puts(fd.readline)
end

edited Oct 31 '14 at 09:19

answered Mar 04 '11 at 14:06

tokland

66,169
13
144
170

This works for getting standard output messages, but it appears that's not where scp is sending its transfer progress. So I guess my problem is with scp now. – Nathan Long Mar 04 '11 at 14:40
This answers the question as I asked it - the fact that scp doesn't do normal output turned out to be an unexpected detail. – Nathan Long Mar 04 '11 at 18:45
See my answer below for how I was able to get that output from the Ruby standard library. – Nathan Long Mar 04 '11 at 18:51

score 3 · Answer 2 · answered Mar 04 '11 at 14:50

3

I think you would have better luck using the ruby standard library to handle SCP (as opposed to forking a shell process). The Net::SCP library (as well as the entire Net::* libraries) are full featured and used with Capistrano to handle remote commands.

Checkout http://net-ssh.rubyforge.org/ for a rundown of what is available.

answered Mar 04 '11 at 14:50

Adam

821
4
10

This looks really promising! The net-scp documentation is here: http://net-ssh.github.com/scp/v1/api/index.html Also, this tutorial shows how to output progress during the transfer: http://ruby.about.com/od/ssh/ss/netscp_4.htm – Nathan Long Mar 04 '11 at 15:14
This is what I ended up doing. I was able to show a running counter of the download progress by passing a block to to scp object. See my answer for the complete script. – Nathan Long Mar 04 '11 at 18:46

score 2 · Answer 3 · answered Mar 04 '11 at 18:49

Tokland answered the question as I asked it, but Adam's approach was what I ended up using. Here was my completed script, which does show a running count of bytes downloaded, and also a percentage complete.

require 'rubygems'
require 'net/scp'
puts "Fetching file"

# Establish the SSH session
ssh = Net::SSH.start("IP Address", "username on server", :password => "user's password on server", :port => 12345)

# Use that session to generate an SCP object
scp = ssh.scp

# Download the file and run the code block each time a new chuck of data is received
scp.download!("path/to/file/on/server/fileName", "/Users/me/Desktop/") do |ch, name, received, total|

  # Calculate percentage complete and format as a two-digit percentage
  percentage = format('%.2f', received.to_f / total.to_f * 100) + '%'

  # Print on top of (replace) the same line in the terminal
  # - Pad with spaces to make sure nothing remains from the previous output
  # - Add a carriage return without a line feed so the line doesn't move down
  print "Saving to #{name}: Received #{received} of #{total} bytes" + " (#{percentage})               \r"

  # Print the output immediately - don't wait until the buffer fills up
  STDOUT.flush
end

puts "Fetch complete!"

yeah, this is the best approach, when using good languages is only logical to rely on their libraries. — tokland, Mar 04 '11 at 20:47

score 0 · Answer 4 · answered Mar 04 '11 at 14:03

0

have you tried with IO.popen ? you should be able to read the output while the process is still running and parse it accordingly.

answered Mar 04 '11 at 14:03

ALoR

4,904
2
23
25

score 0 · Answer 5 · answered Mar 04 '11 at 15:10

Redirecting stderr to stdout may work for you:

output = `scp user@someserver:remoteFile /some/local/folder/ 2>&1`
puts output

That should capture both stderr and stdout. You can capture stderr only by throwing away stdout:

output = `scp user@someserver:remoteFile /some/local/folder/ 2>&1 >/dev/null`
puts output

You can then use IO.popen.

Can I get continuous output from system calls in Ruby?

5 Answers5