0

I want to download all the links in a webpage using a ruby script. My idea was to extract the href links if I could get the Source code, set a download location using command prompt. Is there a way for this ?

Kidambi Manoj
  • 131
  • 1
  • 2
  • 10
  • 1
    Yes.. There is a way. Then you need to use a gem called *selenium-webdriver*. – Arup Rakshit Sep 07 '13 at 14:48
  • 1
    If you want more detailed help, best thing is to start, and ask more specific questions when you get stuck. Your first try doesn't have to be the finished script. Have a play with some Ruby web automation gems, and if you cannot get one to work as you like, ask a question about that thing. There is no problem with asking *lots* of questions on Stack Overflow, as you learn about what you need to do. Just make those questions as specific as you can, with code examples, then it is easy to help you. – Neil Slater Sep 07 '13 at 15:01
  • 1
    Perhaps also take a look at http://stackoverflow.com/questions/2263540/how-do-i-download-a-binary-file-over-http for some example Ruby code to get started. You may need to use more sophisticated solution, like selenium-webdriver if the links you wish to download from are created by Javascript. – Neil Slater Sep 07 '13 at 15:04
  • @NeilSlater Very good suggestions you have given to OP. I like the way you tried to drive him in a correct way !! :) – Arup Rakshit Sep 07 '13 at 16:05

1 Answers1

1
#!/usr/bin/env ruby
#
require "net/http"
require "uri"

#past your link
gets = STDIN.gets rescue nil

url = URI.parse gets

http = Net::HTTP.start(url.host, url.port) do |resp|
  puts resp.get(url.request_uri).body
end

#your regular for href

or use http://nokogiri.org/

Roman Kiselenko
  • 43,210
  • 9
  • 91
  • 103