back-tick in ruby does not work with tail -f command

Question

The code below does not print the output of tail -f. Why? How can I make it work?

#myApp.rb
`ls`               #works fine
`tail -f filename`      #does not work. why?

Answer 1

By using the follow option -f on tail the executed command will not immediately terminate.

-f, --follow[={name|descriptor}]
  output appended data as the file grows;

The idea of using backticks (or the %x shortcut), as opposed to using system('...') is that these statements return the output of the executed commands. This way you could store the result in a variable:

dir_content = `ls`

tail -f spawns another process, this process keeps on writing to standard output without terminating. Hence, your statement is never finished. Without finishing the statement, a value cannot be returned. If you want to watch and output of a growing file, see the solution to the question:

Watch/read a growing log file.

Alternatively, you could run the command through system like so:

system('tail -f filename')

What is the difference here? Instead of returning the outpout of the command, it will return true (command run successfully), false (unsuccessful) or nil (command execution failed). Due to the fact, that the output of the command is not redirected to the return statement running tail -f it will print the content to standard output.

If you are fine with getting the results to standard output you can simply put it into a Thread block. This way the growing content of filename is written to standard output and you can continue to execute other Ruby code.

Thread.new { system('tail -f filename') }

If you want to have complete control, capture the output in order to store it for retrieval at another point of your script then have a look at the answer to the following question which describes such an approach:

Continuously read from STDOUT of external process in Ruby

It is based on the PTY module which creates and manages pseudo terminals. Basically you can spawn another terminal through this module. The code could then look like so:

require 'pty'
cmd = "tail -f filename" 
begin
  PTY.spawn(cmd) do |stdin, stdout, pid|
    begin
      # Do something with the output here: just printing to demonstrate it
      stdin.each { |line| print line }
    rescue Errno::EIO
      puts "Errno:EIO error, but this probably just means " +
            "that the process has finished giving output"
    end
  end
rescue PTY::ChildExited
  puts "The child process exited!"
end

Finally, there is also a socket approach. Open a socket in Bash or by using socat, pipe the output of tail -f to the socket and read in Ruby from the socket.

Answer 2

1) According to the backticks docs,

`cmd`
Returns the standard output of running cmd in a subshell.

So if you write:

puts `some command`

then ruby should print whatever some command outputs.

2) According to the tail man pages:

The tail utility displays the contents of file...to the standard
output.

But the command:

puts `tail -f some_file`

doesn't print anything despite the fact that running:

$ tail -f some_file

sends output to standard out. The tail man pages also say

The -f option causes tail to not stop when
end of file is reached, but rather to wait
for additional data to be appended to the
[the file].

What does that mean? And how is it relevant to the problem at hand?

Because running the program:

puts `tail -f some_file`

...hangs and doesn't output anything, you can infer that the backticks method must be trying to capture the entire output of the command. In other words, the backticks method does not read line by line from the command's standard output, i.e. line oriented input. Instead, the backticks method reads file oriented input, i.e. a file at a time. And because the command tail -f some_file never finishes writing to standard output, the backticks method never reads end of file, and it hangs while it waits for eof.

3) However, reading a command's output a file at a time is not the only way to read a command's output. You can also read a command's output a line at a time using popen():

my_prog.rb:

IO.popen('tail -f data.txt') do |pipe| 
  while line = pipe.gets do  #gets() reads up to the next newline, then returns
    puts line
  end
end


--output(terminal window 1):--
$ ruby my_prog.rb 
A
B
C
<hangs>

--output(terminal window 2):--
echo 'D' >> data.txt

--output(terminal window 1):--
A
B
C
D
<hangs>

echo man pages:

The echo utility writes any specified operands...followed
by a newline (`\n') character.

Answer 3

The reason it is not working is because the backticks make a separate System call to run the commands within the backticks. If you do ps aux | grep tail, you will be able to see that the tail process is in fact running.

The code is not showing any output because its waiting for tail to terminate and proceed to next statements - its not hung because it is displaying the output of tail command.

Even in normal operation, you have to do a ctrl+C to close the output of tail -fcommand. The -f is used to output appended data as the file grows, and hence tail -f keeps running. Try to do a kill -9 <pid_of_tail> using the pid from ps aux | grep tail. Then the remaining part of your ruby code will start outputting the result.

The ls, tail without -f calls work because they terminate on their own.

If you are interested in doing a tail from ruby, you can have a look at this file-tail gem http://flori.github.io/file-tail/doc/index.html