Running rake task on remote app, use file from local machine

Question

How can I pass the csv file or file stream or something in line of that to the rake task I'm running on the remote app via rake task arguments?

So I can get the contents of that file in the file and do something with it. It's not a big file.

Update

I tried with suggestion from Luc:

desc 'Test task'
namespace :app do
  task :pipe_file => [:environment] do |t, args|
    puts "START"

    File.open('my_temp_file', 'w') do |f2|
      while line = STDIN.gets
        f2.puts line
      end
    end

    puts "DONE"
  end
end

So when I run :

cat tst.csv | bundle exec rake app:pipe_file

Nothing happens, blank line prints

Can't you pass the path to the file to the rake task as an argument ? — Rafal, Jul 03 '14 at 13:58
You can look at http://docs.seattlerb.org/rake-remote_task/Rake/RemoteTask.html There is a PUT method which copies a file to the remote server — Rafal, Jul 03 '14 at 14:10
possible duplicate of [Execute a rake task on a remote server](http://stackoverflow.com/questions/20615329/execute-a-rake-task-on-a-remote-server) — infused, Jul 03 '14 at 17:00
@infused there is not simularity to that question other than the title — Gandalf StormCrow, Jul 03 '14 at 23:16

Luc Boissaye · Accepted Answer · 2022-10-13T14:05:09.617

7

You can pipe the content of your file to your rake task:

cat my_file | heroku run rake --no-tty my_task

Then inside your task you need to start by reading STDIN:

STDIN.binmode
tmp_file = Tempfile.new('temp_file_prefix',  Rails.root.join('tmp'))
tmp_file.write(STDIN.read)
tmp_file.close
Process tmp_file here.
puts tmp_file.path
tmp_file.unlink

Hope it helps !

edited Oct 13 '22 at 14:05

answered Jul 03 '14 at 14:44

Luc Boissaye

935
8
14

1

Hey I like your idea, is there a way to test it localy first? I tried putting your suggested code in my task and executed like this `cat my_file | bundle exec rake my_namespace:task`, but get nothing printed. What could I do wrong? – Gandalf StormCrow Jul 03 '14 at 16:39
You will have nothing printed out but a file my_temp_file should be created with the content of your file. When you run the command remotely, the file will be created where the code is executed on the remote machine. You can just use the file my_temp_file as it is a local_file after the trick. – Luc Boissaye Jul 04 '14 at 13:11
Curious, what benefit is there to writing STDIN to a file, if the file is going to be deleted after you're done processing it. Why not just save STDIN to a variable? Does reading/writing STDIN to a file use the same or less memory? – Andrew Sep 13 '21 at 21:30
Even better than saving it to a file is just reading stdin . Here I provided the code so you can keep the rest of your task as it is and read the file as if it were a local file. But from a memory perspective it will be better to just read stdin and do your processing from there. (saving only what is required in memory) – Luc Boissaye Oct 15 '21 at 14:58
1

Note that for larger files (> ~4K) you may need to pass --no-tty. See this issue: github.com/heroku/legacy-cli/issues/1409#issuecomment-243344505 – Mike Vosseller Oct 12 '22 at 13:24

Running rake task on remote app, use file from local machine

1 Answers1