String interpolation while running shell command through ruby?

Question

Struggling with string interpolation while executing a shell command in ruby. Can you someone please help me identify what I'm missing here?

My one-liner ruby code follows redirects of any shortURL and returns the final URL. For example, this ruby code works perfectly fine.

curl -I https://bit.ly/1mJk8X7 | perl -n -e '/^Location: (.*)$/ && print "$1\n"' It prints out the final URL.

I have a .txt file with a series of short URLs from which I'd like to derive a list of the final URLs. Say, it's called shortURLs.txt. I'm using IO.foreach to loop through each line in the file, but I don't know what I'm doing wrong to bring the variable 'x' into the ruby command. This is my first time working with string interpolation, and I've tried various combinations of it, but no luck yet.

IO.foreach("shortURLs.txt") { |x| system "curl -I #{x} | perl -n -e '/^Location: (.*)$/ && print \"$1\n\"' >> finalURLs.txt" }

I get an error message around the pipe '|' symbol:

sh: -c: line 1: syntax error near unexpected token|' sh: -c: line 1: | perl -n -e '/^Location: (.*)$/ && print "https://bit.ly/1mJk8X7'

Other threads have been useful about string interpolation and running shell commands through ruby.

What is the output for `puts "curl -I #{x} | perl -n -e '/^Location: (.*)$/ && print \"$1\n\"' >> finalURLs.txt" ` ? — mabe02, Jan 20 '16 at 21:10

score 0 · Accepted Answer · answered Jan 20 '16 at 21:27

0

In order to pass two commands to a shell, you should run the system command twice (check method 8 in this post)

require 'shell'
sh = Shell.new
IO.foreach("shortURLs.txt") { |x| sh.system("curl -I #{x}")  | sh.system("perl -n -e '/^Location: (.*)$/ && print \"$1\n\" ' ") >>  "finalURLs.txt" }

answered Jan 20 '16 at 21:27

mabe02

2,676
2
20
35

Thanks mabe02. That's exactly what I was getting wrong. It's working now. To follow up, while printing "$1", I get an extra linebreak. I'm trying to add both URLs in one line separated by a comma. Currently print\"#{x},$1\n\" adds an extra line break between the two URLs. Do you know what I could be doing wrong there? – user1575454 Jan 22 '16 at 15:27

score 0 · Answer 2 · answered Jan 21 '16 at 08:30

IO.foreach yields the lines including the new line at the end so you're actually executing

curl -I https://bit.ly/1mJk8X7
| perl -n -e  ...

Which is why you get the syntax error. You could use strip to remove the new line from. I think the \n in the call to print will also get substituted before the string is passed to system. You may be interested in shellwords which has functions for escaping strings before passing them to a shell.

You could of course dodge the issue entirely and use ruby to get the redirect locations

require 'net/http'
require 'uri'

IO.foreach("shortURLs.txt") do  |url|
  puts Net::HTTP.get_response(URI.parse(url))["Location"]
end

String interpolation while running shell command through ruby?

2 Answers2