Ruby NET::SCP containing Wildcard

Question

I need to download a file daily from a client that I have SCP but not SSH access to.

The file name will always be /outgoing/Extract/visit_[date]-[timestamp].dat.gz'

For example yesterdays file was called visits_20130604-090003.dat.gz

I can not rely on the fact that the time stamp will always be the same, but the date should always be yesterdays date:

My set up so far:

My home directory contains to sub-directories named downloads_fullname and downloads_wildcard.

It also contains an simple ruby script named foo.rb.

The contents of foo.rb are this`

#! /usr/bin/ruby
require 'net/ssh'
require 'net/scp'
yesterday = (Time.now - 86400).strftime('%Y%m%d')

Net::SCP.start('hostname', 'username') do |scp|
  scp.download!('/outgoing/Extract/visits_' + yesterday + '-090003.dat.gz', 'downloads_fullname')
  scp.download!('/outgoing/Extract/visits_' + yesterday + '-*.dat.gz', 'downloads_wildcard')
end

When run the downloads_fullname directory contains the file, but the downloads_wildcard directory does not.

Is there any way to use wildcarding in Net::SCP? Or does anybody have any sly workarounds? I tried \*to no avail.

score 4 · Answer 1 · edited Oct 13 '14 at 20:18

Thank you Tin Man!!!

To anybody else, here is the code I ended up with following Tin Man's lead:

(Tried to post it as a comment but had formatting issues)

#! /usr/bin/ruby
require 'net/sftp'
yesterday = (Time.now - 86400).strftime('%Y%m%d')

Net::SFTP.start('hostname', 'username') do |sftp|
  sftp.dir.foreach("/outgoing/Extract") do |file|
     if file.name.include? '_' + yesterday + '-'
       sftp.download!('/outgoing/Extract/' + file.name, 'downloads/'+ file.name)
     end
  end
end

score 3 · Accepted Answer · answered Jun 05 '13 at 22:41

I don't think you can get there using scp because it expects you to know exactly which file you want, but sftp will let you get a directory listing.

You can use Net::SFTP to programmatically pick your file and request it. This is the example code:

require 'net/sftp'

Net::SFTP.start('host', 'username', :password => 'password') do |sftp|
  # upload a file or directory to the remote host
  sftp.upload!("/path/to/local", "/path/to/remote")

  # download a file or directory from the remote host
  sftp.download!("/path/to/remote", "/path/to/local")

  # grab data off the remote host directly to a buffer
  data = sftp.download!("/path/to/remote")

  # open and write to a pseudo-IO for a remote file
  sftp.file.open("/path/to/remote", "w") do |f|
    f.puts "Hello, world!\n"
  end

  # open and read from a pseudo-IO for a remote file
  sftp.file.open("/path/to/remote", "r") do |f|
    puts f.gets
  end

  # create a directory
  sftp.mkdir! "/path/to/directory"

  # list the entries in a directory
  sftp.dir.foreach("/path/to/directory") do |entry|
    puts entry.longname
  end
end

Based on that you can list the directory entries then use find or select to iterate over the returned list to find the one with the current date. Pass that filename to sftp.download! to download it to a local file.

Ruby NET::SCP containing Wildcard

2 Answers2