1

I have an "offline processing" server which charges me very little for cpu cycles and a "media serving" server which charges me very little for bandwidth. I have a ~1.3Mb zip file which the offline processing server generates and then sends over to the media serving server to actually host for download. The way I had been doing this was by having the offline processing server make the file on himself and ftp it over to the other one, but I had to move hosts and now the offline processing server doesn't have the ability to save files to itself.

My first thought was I can make a listening php script on the media serving server and then post the contents to that script which will save it into a zip file. Unzipped it is ~4.1Mb so would it be a good idea to post this much data? The second thing I thought of was streams in php... but I didn't know what would be best.

What is the best way for me to move this 4.1Mb of data from one server to the other when I can't save it as a file in the server which generates the data?

hackartist
  • 5,172
  • 4
  • 33
  • 48
  • How often do you need to do this processing? – corsiKa Jul 14 '12 at 02:59
  • about once a week and it is on a cron job because of this -- as my underlying dataset changes, this makes a summary for download on a mobile client. – hackartist Jul 14 '12 at 02:59
  • Off the top of my head, it seems like it's easily small enough to fit in memory to send to the content server... but I'll note that I'm not a big php guy. – corsiKa Jul 14 '12 at 03:01
  • 1
    Why do you have to use PHP? Rsync or FTP are well suited for this kind of task. – Eric J. Jul 14 '12 at 03:04
  • if you have to use php , you can post such amount of data. it's fine. but make sure your media server settings allow it and php doesn't time out: http://stackoverflow.com/questions/2364840/what-is-the-size-limit-of-a-post-request – Dreaded semicolon Jul 14 '12 at 03:05
  • I have to use php because it is a php script talking to the database and doing the processing so the data is in php's memory and can not make it to a file. Otherwise I would just FTP like I used to but part of the question was specifically that php could not make any files. – hackartist Jul 14 '12 at 03:14

1 Answers1

2

If your host supports it, I might try something like:

$fp = fopen('php://memory', 'w+');
fwrite($fp, "Hello, World!"); // create your file
fseek($fp, 0, SEEK_SET);

// connect to ftp
ftp_fput($ftp, '/foo', $fp, FTP_BINARY);   

fclose($fp);
Matthew
  • 47,584
  • 11
  • 86
  • 98
  • Very very interesting -- I had never heard of the php://memory before but I will look into that. – hackartist Jul 14 '12 at 03:13
  • 1
    Worst case, you could put everything into a variable (string) and write a thin stream wrapper over it. That would definitely work. (For an ugly example: http://php.net/manual/en/stream.streamwrapper.example-1.php) – Matthew Jul 14 '12 at 03:16
  • Thank you so much for your answer. It worked very well but I also have a [follow up question](http://stackoverflow.com/questions/11488752/string-to-zipped-stream-in-php) you might be able to solve since it is related – hackartist Jul 15 '12 at 01:56