Compress and upload to AWS S3

Question

I have users uploading files to my server generally large HTML (4-8 MB) files. Server then pushses it to S3 and serve the files to users. What I need is to compress files before pushing them to S3 and serve the compressed files to users. How can I achieve this ? I have APACHE, PHP running in my back end.

Alex · Answer 1 · 2015-04-17T18:48:53.357

2

You could compress your files using PHP and push the compressed files to S3.

// Name of the file to compress
$file = "yourfile.txt";

// Name of compressed gz file 
$gzfile = "test.gz";

// Open the gz file (w9 is the highest compression)
$fp = gzopen ($gzfile, 'w9');

// Compress the file
gzwrite ($fp, file_get_contents($file));

//close the file after compression is done
gzclose($fp);

edited Apr 17 '15 at 18:48

answered Apr 17 '15 at 18:46

Alex

21,273
10
61
73

And push the .gz file to S3? – vipulsodha Apr 17 '15 at 18:47
Yes, assuming it is okay to server your users with .gz files – Alex Apr 17 '15 at 18:49
how can we serve .gz file to user browser and decompress it ? back to html? Will adding header Content Type help? – vipulsodha Apr 17 '15 at 18:51
You need to set your content type to application/x-gzip. Please note that you cannot upload a gz file then extract it once its in S3. – Alex Apr 17 '15 at 18:55
How can one add stream context to set metadata such as Content-Type? With php's fopen function there is 4th parameter $context for that, but gzopen does not. ? – RayOnAir Jun 29 '16 at 23:58

score 2 · Answer 2 · edited May 23 '17 at 12:06

You have the example in other answer how to compress to gzip
Copy to s3 is possible via aws console as:

aws s3 cp /path/test.gz s3://your_bucket/path/

you will need to put key/paid credentials to ~/.aws/config file

Here is some answer how to serve gzipped content with Apache apache .gz gzip content handler for Linux documentation /usr/share/doc and localhost/doc/

I belive you will need to integrate s3fs-fuse(https://code.google.com/p/s3fs/) file system to your web server to read gzipped content

Also I have some experience with s3fs and won't recommend you try write directly to mounted s3fs filesystem from server, it can bring troubles and system crash. The best approach is copy content via aws console, but use mounted filesystem for read only access

Update: More details about s3fs problem and how to use aws s3 upload api you can see at me other answer here How could I store uploaded images to AWS S3 on PHP

Compress and upload to AWS S3

2 Answers2