9

I have this code, which writes a zip file to disk, reads it back, uploads to s3, then deletes the file:

compressed_file = some_temp_path

Zip::ZipOutputStream.open(compressed_file) do |zos|
  some_file_list.each do |file|
    zos.put_next_entry(file.some_title)
    zos.print IO.read(file.path)
  end
end # Write zip file

s3 = Aws::S3.new(S3_KEY, S3_SECRET)
bucket = Aws::S3::Bucket.create(s3, S3_BUCKET)
bucket.put("#{BUCKET_PATH}/archive.zip", IO.read(compressed_file), {}, 'authenticated-read')

File.delete(compressed_file)

This code works already but what I want is to not create the zip file anymore, to save a few steps. I was wondering if there is a way to export the zipfile data directly to s3 without having to first create a tmpfile, read it back, then delete it?

shytikov
  • 9,155
  • 8
  • 56
  • 103
index
  • 3,697
  • 7
  • 36
  • 55

1 Answers1

10

I think I just found the answer to my question.

It's Zip::ZipOutputStream.write_buffer. I'll check this out and update this answer when I get it working.

Update

It does work. My code is like this now:

compressed_filestream = Zip::ZipOutputStream.write_buffer do |zos|
  some_file_list.each do |file|
    zos.put_next_entry(file.some_title)
    zos.print IO.read(file.path)
  end
end # Outputs zipfile as StringIO

s3 = Aws::S3.new(S3_KEY, S3_SECRET)
bucket = Aws::S3::Bucket.create(s3, S3_BUCKET)

compressed_filestream.rewind
bucket.put("#{BUCKET_PATH}/archive.zip", compressed_filestream.read, {}, 'authenticated-read')

The write_buffer returns a StringIO and needs to rewind the stream first before reading it. Now I don't need to create and delete the tmpfile.

I'm just wondering now if write_buffer would be more memory extensive or heavier than open? Or is it the other way around?

index
  • 3,697
  • 7
  • 36
  • 55
  • What is `some_file_list` ? – Trip Nov 07 '13 at 06:16
  • I'm guessing its something like.. `some_file_list = Zip::ZipFile.open(zipped_file)` – Trip Nov 07 '13 at 06:26
  • 2
    For those confused why you're getting errors like `NameError: uninitialized constant Zip::ZipOutputStream`, it's because nowadays the class is called `Zip::OutputStream`. – fatuhoku Mar 07 '17 at 18:48