9

This has gone around a number of times but I'm still a bit bewildered. A lot of answers only copncern themselves with talking about upload progress bars and not getting the actual upload progress from an S3 upload.

I have read a number of questions and found a number of software pieces but I am still no closer to understanding the fundamental question of S3 uploads.

Is there a way of uploading to S3 while understanding the progress of that upload without having to use my own app servers resources to store the file in the temp directory first?

Do I have to store the file on my own server first and then push to S3? I have heard people talk about streaming bit at a time to S3 (one chunk at a time) but I am unsure what this involves. Imagine I was uploading from client side in a HTML page how would I stream the file chunk by chunk from a multi-part from as it comes in? (I don't see any example of that only an example of when the file is already on your system and you know the chunks which is kinda useless tbh).

There is of course an example of upload progress in the API docs but again that is assuming the file is on your server first and is not coming from another computer supplied by the user.

EDIT: My original thought was to make a PHP script that could ping AWS once every so oftern getting upload progress. I am looking through the API to see if they do support something like that but atm no luck. Let me know if there is something...

Further EDIT: Is flash the only way to go with this?

Thanks,

hakre
  • 193,403
  • 52
  • 435
  • 836
Sammaye
  • 43,242
  • 7
  • 104
  • 146

5 Answers5

7

YES, it is possible in AWS PHP SDK v3.

$client = new S3Client(/* config */);

$result = $client->putObject([
    'Bucket'     => 'bucket-name',
    'Key'        => 'bucket-name/file.ext',
    'SourceFile' => 'local-file.ext',
    'ContentType' => 'application/pdf',
    'params' => [
            '@http' => [
                'progress' => function ($downloadTotalSize, $downloadSizeSoFar, $uploadTotalSize, $uploadSizeSoFar) {
                    printf(
                        "%s of %s downloaded, %s of %s uploaded.\n",
                        $downloadSizeSoFar,
                        $downloadTotalSize,
                        $uploadSizeSoFar,
                        $uploadTotalSize
                    );
                }
            ]
    ]
]);

This is explained in the AWS docs - S3 Config section. It works by exposing GuzzleHttp's progress property-callable, as explained in this SO answer.

zvi
  • 3,677
  • 2
  • 30
  • 48
The Onin
  • 5,068
  • 2
  • 38
  • 55
0

After a lot of searching around I found the simple answer to this question is NO.

Basically the best way to do this, as I have found out from Amazons own coding (their little uploading widget on the AWS S3 console) it is only really possible to get upload progress by sending to another server first which PHP can read for progress information.

This creates a 'pass-through' policy using your own app servers /tmp directory as the stepping stone. This seems to be the preferred method.

Sammaye
  • 43,242
  • 7
  • 104
  • 146
  • 1
    Do you perhaps have any examples still available? I'll probably have to do a similar thing.. – trainoasis Aug 24 '16 at 09:18
  • Only very old code which I do not think is valid with their new API, basically you upload as normal to our own server and then copy either in it's full form or per chunk to S3 (using a multiple part form you can get the chunks before it is fully uploaded in PHP from the stream) – Sammaye Aug 24 '16 at 09:21
  • I managed to upload to S3 from JS via my PHP proxy. But how would I go about keeping progress bar status in JS ? no info about that anywhere.. – trainoasis Aug 24 '16 at 13:06
  • You cannot really do a progress bar in JS, you need to create an AJAX poller to the server side which responds with the progression – Sammaye Aug 24 '16 at 14:06
  • Indeed - but how do I get progress from putObject PHP call for example? (so I can pass the data on ..) – trainoasis Aug 25 '16 at 06:26
  • You don't you o multipart uploads https://docs.aws.amazon.com/aws-sdk-php/v3/guide/service/s3-multipart-upload.html you take the file size and break it into chunks and then use that to determine progress – Sammaye Aug 25 '16 at 08:46
  • Thanks, will get into it – trainoasis Aug 25 '16 at 09:12
  • No progress example here though? It looks pretty similar to putObject ... – trainoasis Aug 25 '16 at 10:13
  • You won't find an example, you gotta put it together yourself. This function allows you to upload in chunks to S3, if you split the file down in PHP and use this function then you can get progress while uploading the chunks you have – Sammaye Aug 25 '16 at 10:37
0

AWS PHP SDK has a code example where they show upload progress while uploading to S3. I am not sure if its valid for small files but for streaming it sure works.

You can check more out at - http://aws.amazon.com/sdkforphp/

In a way if they are showing progress - we surely can get that in the form of a progress bar with the help of a Javascript framework like jquery etc.

I shall be working on it very soon and will surely post a tutorial. Cheers!

foxybagga
  • 4,184
  • 2
  • 34
  • 31
  • I would like that very much, this question was written about 4 versions ago of the sdk so I would be very much interested if things have changed and AWS does have a good way of dong this – Sammaye Feb 03 '13 at 16:05
  • 2
    It would be best if you could link to the specific page with the code example and not just the sdk homepage. – tim peterson Dec 28 '13 at 22:35
  • Did you post a tutorial on this? – trainoasis Aug 25 '16 at 06:28
0

Yes you can via multipart uploading: http://docs.aws.amazon.com/AmazonS3/latest/dev/UsingMPphpAPI.html

Multipart uploading is a three-step process: You initiate the upload, you upload the object parts, and after you have uploaded all the parts, you complete the multipart upload. Upon receiving the complete multipart upload request, Amazon S3 constructs the object from the uploaded parts, and you can then access the object just as you would any other object in your bucket.

http://docs.aws.amazon.com/AmazonS3/latest/dev/mpuoverview.html

I will be doing this personally very soon but this is what you need to start.

Michael J. Calkins
  • 32,082
  • 15
  • 62
  • 91
  • You still have to upload this to your own server to understand a progression since you must have the total file size on your server before you can do anything with it. Effectively you will be uploading it to your sertver first as normal and then multi part uploading to Amazon to get the upload progression which means you will actually be judging the progression of upload on your server only. I suppose you could rewrite the core PHP scripts to release upload parts that you can then upload to Amazon, howeer the PHP core does not currently do this – Sammaye Feb 27 '13 at 08:28
  • @Sammaye I'm actually in the middle of doing this atm and I'm having to upload to server then to S3. Until I figure out to go straight to S3 I was going write a little hack or write to a file temporarily to send feedback for my users. As long as you can see or use this multipart function you should be able to hook in, in one way or another. – Michael J. Calkins Feb 27 '13 at 19:01
  • 1
    @MichaelCalkins did you ever get this accomplished? 2014 is almost done and I still can't see a clear cut answer to do this? – turntwo Nov 21 '14 at 07:20
  • Any examples perhaps? I'm using JS to send to browser but credentials are plaintext, so I'll have to go with JS->proxy->S3 where proxy will be my server script. Any help appreciated – trainoasis Aug 24 '16 at 09:17
-2

Yes,you can do this by modifying s3.php like this,

<?php
class S3Withprogressbar {
// ACL flags
const ACL_PRIVATE = 'private';
const ACL_PUBLIC_READ = 'public-read';
const ACL_PUBLIC_READ_WRITE = 'public-read-write';
private static $__accessKey; // AWS Access key
private static $__secretKey; // AWS Secret key
public static $_uploadProgressFileName="" ;//uploadProgressFileName
public function __construct($accessKey = null, $secretKey = null) {
if ($accessKey !== null && $secretKey !== null)
self::setAuth($accessKey, $secretKey);
}
public static function setAuth($accessKey, $secretKey) {
self::$__accessKey = $accessKey;
self::$__secretKey = $secretKey;
}
public static function listBuckets($detailed = false) {
$rest = new S3Request('GET', '', '');
$rest = $rest->getResponse();
if ($rest->error === false && $rest->code !== 200)
$rest->error = array('code' => $rest->code, 'message' => 'Unexpected HTTP status');
if ($rest->error !== false) {
trigger_error(sprintf("S3Withprogressbar::listBuckets(): [%s] %s", $rest->error['code'], $rest->error['message']), E_USER_WARNING);
return false;
}
$results = array(); //var_dump($rest->body);
if (!isset($rest->body->Buckets)) return $results;
if ($detailed) {
if (isset($rest->body->Owner, $rest->body->Owner->ID, $rest->body->Owner->DisplayName))
$results['owner'] = array(
'id' => (string)$rest->body->Owner->ID, 'name' => (string)$rest->body->Owner->ID
);
$results['buckets'] = array();
foreach ($rest->body->Buckets->Bucket as $b)
$results['buckets'][] = array(
'name' => (string)$b->Name, 'time' => strtotime((string)$b->CreationDate)
);
} else
foreach ($rest->body->Buckets->Bucket as $b) $results[] = (string)$b->Name;
return $results;
}
public static function getBucket($bucket, $prefix = null, $marker = null, $maxKeys = null) {
$rest = new S3Request('GET', $bucket, '');
if ($prefix !== null && $prefix !== '') $rest->setParameter('prefix', $prefix);
if ($marker !== null && $prefix !== '') $rest->setParameter('marker', $marker);
if ($maxKeys !== null && $prefix !== '') $rest->setParameter('max-keys', $maxKeys);
$response = $rest->getResponse();
if ($response->error === false && $response->code !== 200)
$response->error = array('code' => $response->code, 'message' => 'Unexpected HTTP status');
if ($response->error !== false) {
trigger_error(sprintf("S3Withprogressbar::getBucket(): [%s] %s", $response->error['code'], $response->error['message']), E_USER_WARNING);
return false;
}
$results = array();
$lastMarker = null;
if (isset($response->body, $response->body->Contents))
foreach ($response->body->Contents as $c) {
$results[(string)$c->Key] = array(
'name' => (string)$c->Key,
'time' => strToTime((string)$c->LastModified),
'size' => (int)$c->Size,
'hash' => substr((string)$c->ETag, 1, -1)
);
$lastMarker = (string)$c->Key;
//$response->body->IsTruncated = 'true'; break;
}
if (isset($response->body->IsTruncated) &&
(string)$response->body->IsTruncated == 'false') return $results;
// Loop through truncated results if maxKeys isn't specified
if ($maxKeys == null && $lastMarker !== null && (string)$response->body->IsTruncated == 'true')
do {
$rest = new S3Request('GET', $bucket, '');
if ($prefix !== null) $rest->setParameter('prefix', $prefix);
$rest->setParameter('marker', $lastMarker);
if (($response = $rest->getResponse(true)) == false || $response->code !== 200) break;
if (isset($response->body, $response->body->Contents))
foreach ($response->body->Contents as $c) {
$results[(string)$c->Key] = array(
'name' => (string)$c->Key,
'time' => strToTime((string)$c->LastModified),
'size' => (int)$c->Size,
'hash' => substr((string)$c->ETag, 1, -1)
);
$lastMarker = (string)$c->Key;
}
} while ($response !== false && (string)$response->body->IsTruncated == 'true');
return $results;
}
public function putBucket($bucket, $acl = self::ACL_PRIVATE) {
$rest = new S3Request('PUT', $bucket, '');
$rest->setAmzHeader('x-amz-acl', $acl);
$rest = $rest->getResponse();
if ($rest->error === false && $rest->code !== 200)
$rest->error = array('code' => $rest->code, 'message' => 'Unexpected HTTP status');
if ($rest->error !== false) {
//trigger_error(sprintf("S3Withprogressbar::putBucket({$bucket}): [%s] %s", $rest->error['code'], $rest->error['message']), E_USER_WARNING);
return false;
}
return true;
}
public function deleteBucket($bucket = '') {
$rest = new S3Request('DELETE', $bucket);
$rest = $rest->getResponse();
if ($rest->error === false && $rest->code !== 204)
$rest->error = array('code' => $rest->code, 'message' => 'Unexpected HTTP status');
if ($rest->error !== false) {
trigger_error(sprintf("S3Withprogressbar::deleteBucket({$bucket}): [%s] %s",
$rest->error['code'], $rest->error['message']), E_USER_WARNING);
return false;
}
return true;
}
public static function inputFile($file, $md5sum = true) {
if (!file_exists($file) || !is_file($file) || !is_readable($file)) {
trigger_error('S3Withprogressbar::inputFile(): Unable to open input file: '.$file, E_USER_WARNING);
return false;
}
return array('file' => $file, 'size' => filesize($file),
'md5sum' => $md5sum !== false ? (is_string($md5sum) ? $md5sum :
base64_encode(md5_file($file, true))) : '');
}
public static function inputResource(&$resource, $bufferSize, $md5sum = '') {
if (!is_resource($resource) || $bufferSize <= 0) {
trigger_error('S3Withprogressbar::inputResource(): Invalid resource or buffer size', E_USER_WARNING);
return false;
}
$input = array('size' => $bufferSize, 'md5sum' => $md5sum);
$input['fp'] =& $resource;
return $input;
}
// Modified Function with one more  parameter $uploadProgressFileName
public static function putObject($input, $bucket, $uri, $acl = self::ACL_PRIVATE, $metaHeaders = array(), $contentType = null,$uploadProgressFileName) {
if ($input == false) return false;
$rest = new S3Request('PUT', $bucket, $uri);
// Set public variable value($uploadProgressFileName) in S3Request class  
$rest->uploadProgressFileName = $uploadProgressFileName;
if (is_string($input)) $input = array(
'data' => $input, 'size' => strlen($input),
'md5sum' => base64_encode(md5($input, true))
);
// Data
if (isset($input['fp']))
$rest->fp =& $input['fp'];
elseif (isset($input['file']))
$rest->fp = @fopen($input['file'], 'rb');
elseif (isset($input['data']))
$rest->data = $input['data'];
// Content-Length (required)
if (isset($input['size']) && $input['size'] > 0)
$rest->size = $input['size'];
else {
if (isset($input['file']))
$rest->size = filesize($input['file']);
elseif (isset($input['data']))
$rest->size = strlen($input['data']);
}
// Content-Type
if ($contentType !== null)
$input['type'] = $contentType;
elseif (!isset($input['type']) && isset($input['file']))
$input['type'] = self::__getMimeType($input['file']);
else
$input['type'] = 'application/octet-stream';
// We need to post with the content-length and content-type, MD5 is optional
if ($rest->size > 0 && ($rest->fp !== false || $rest->data !== false)) {
$rest->setHeader('Content-Type', $input['type']);
if (isset($input['md5sum'])) $rest->setHeader('Content-MD5', $input['md5sum']);
$rest->setAmzHeader('x-amz-acl', $acl);
foreach ($metaHeaders as $h => $v) $rest->setAmzHeader('x-amz-meta-'.$h, $v);
$rest->getResponse();
} else
$rest->response->error = array('code' => 0, 'message' => 'Missing input parameters');
if ($rest->response->error === false && $rest->response->code !== 200)
$rest->response->error = array('code' => $rest->response->code, 'message' => 'Unexpected HTTP status');
if ($rest->response->error !== false) {
//trigger_error(sprintf("S3Withprogressbar::putObject(): [%s] %s", $rest->response->error['code'], $rest->response->error['message']), E_USER_WARNING);
return false;
}
return true;
}
// Modified Function with one more  parameter $uploadProgressFileName
public static function putObjectFile($file, $bucket, $uri, $acl = self::ACL_PRIVATE, $metaHeaders = array(), $contentType = null,$uploadProgressFileName) {
return self::putObject(S3Withprogressbar::inputFile($file), $bucket, $uri, $acl, $metaHeaders, $contentType,$uploadProgressFileName);
}
public function putObjectString($string, $bucket, $uri, $acl = self::ACL_PRIVATE, $metaHeaders = array(), $contentType = 'text/plain') {
return self::putObject($string, $bucket, $uri, $acl, $metaHeaders, $contentType);
}
public static function getObject($bucket = '', $uri = '', $saveTo = false) {
$rest = new S3Request('GET', $bucket, $uri);
if ($saveTo !== false) {
if (is_resource($saveTo))
$rest->fp =& $saveTo;
else
if (($rest->fp = @fopen($saveTo, 'wb')) == false)
$rest->response->error = array('code' => 0, 'message' => 'Unable to open save file for writing: '.$saveTo);
}
if ($rest->response->error === false) $rest->getResponse();
if ($rest->response->error === false && $rest->response->code !== 200)
$rest->response->error = array('code' => $rest->response->code, 'message' => 'Unexpected HTTP status');
if ($rest->response->error !== false) {
trigger_error(sprintf("S3Withprogressbar::getObject({$bucket}, {$uri}): [%s] %s",
$rest->response->error['code'], $rest->response->error['message']), E_USER_WARNING);
return false;
}
$rest->file = realpath($saveTo);
return $rest->response;
}
public static function getObjectInfo($bucket = '', $uri = '', $returnInfo = true) {
$rest = new S3Request('HEAD', $bucket, $uri);
$rest = $rest->getResponse();
if ($rest->error === false && ($rest->code !== 200 && $rest->code !== 404))
$rest->error = array('code' => $rest->code, 'message' => 'Unexpected HTTP status');
if ($rest->error !== false) {
trigger_error(sprintf("S3Withprogressbar::getObjectInfo({$bucket}, {$uri}): [%s] %s",
$rest->error['code'], $rest->error['message']), E_USER_WARNING);
return false;
}
return $rest->code == 200 ? $returnInfo ? $rest->headers : true : false;
}
public static function setBucketLogging($bucket, $targetBucket, $targetPrefix) {
$dom = new DOMDocument;
$bucketLoggingStatus = $dom->createElement('BucketLoggingStatus');
$bucketLoggingStatus->setAttribute('xmlns', 'http://s3.amazonaws.com/doc/2006-03-01/');
$loggingEnabled = $dom->createElement('LoggingEnabled');
$loggingEnabled->appendChild($dom->createElement('TargetBucket', $targetBucket));
$loggingEnabled->appendChild($dom->createElement('TargetPrefix', $targetPrefix));
// TODO: Add TargetGrants
$bucketLoggingStatus->appendChild($loggingEnabled);
$dom->appendChild($bucketLoggingStatus);
$rest = new S3Request('PUT', $bucket, '');
$rest->setParameter('logging', null);
$rest->data = $dom->saveXML();
$rest->size = strlen($rest->data);
$rest->setHeader('Content-Type', 'application/xml');
$rest = $rest->getResponse();
if ($rest->error === false && $rest->code !== 200)
$rest->error = array('code' => $rest->code, 'message' => 'Unexpected HTTP status');
if ($rest->error !== false) {
trigger_error(sprintf("S3Withprogressbar::setBucketLogging({$bucket}, {$uri}): [%s] %s",
$rest->error['code'], $rest->error['message']), E_USER_WARNING);
return false;
}
return true;
}
public static function getBucketLogging($bucket = '') {
$rest = new S3Request('GET', $bucket, '');
$rest->setParameter('logging', null);
$rest = $rest->getResponse();
if ($rest->error === false && $rest->code !== 200)
$rest->error = array('code' => $rest->code, 'message' => 'Unexpected HTTP status');
if ($rest->error !== false) {
trigger_error(sprintf("S3Withprogressbar::getBucketLogging({$bucket}): [%s] %s",
$rest->error['code'], $rest->error['message']), E_USER_WARNING);
return false;
}
if (!isset($rest->body->LoggingEnabled)) return false; // No logging
return array(
'targetBucket' => (string)$rest->body->LoggingEnabled->TargetBucket,
'targetPrefix' => (string)$rest->body->LoggingEnabled->TargetPrefix,
);
}
public static function setAccessControlPolicy($bucket, $uri = '', $acp = array()) {
$dom = new DOMDocument;
$dom->formatOutput = true;
$accessControlPolicy = $dom->createElement('AccessControlPolicy');
$accessControlList = $dom->createElement('AccessControlList');
// It seems the owner has to be passed along too
$owner = $dom->createElement('Owner');
$owner->appendChild($dom->createElement('ID', $acp['owner']['id']));
$owner->appendChild($dom->createElement('DisplayName', $acp['owner']['name']));
$accessControlPolicy->appendChild($owner);
foreach ($acp['acl'] as $g) {
$grant = $dom->createElement('Grant');
$grantee = $dom->createElement('Grantee');
$grantee->setAttribute('xmlns:xsi', 'http://www.w3.org/2001/XMLSchema-instance');
if (isset($g['id'])) { // CanonicalUser (DisplayName is omitted)
$grantee->setAttribute('xsi:type', 'CanonicalUser');
$grantee->appendChild($dom->createElement('ID', $g['id']));
} elseif (isset($g['email'])) { // AmazonCustomerByEmail
$grantee->setAttribute('xsi:type', 'AmazonCustomerByEmail');
$grantee->appendChild($dom->createElement('EmailAddress', $g['email']));
} elseif ($g['type'] == 'Group') { // Group
$grantee->setAttribute('xsi:type', 'Group');
$grantee->appendChild($dom->createElement('URI', $g['uri']));
}
$grant->appendChild($grantee);
$grant->appendChild($dom->createElement('Permission', $g['permission']));
$accessControlList->appendChild($grant);
}
$accessControlPolicy->appendChild($accessControlList);
$dom->appendChild($accessControlPolicy);
$rest = new S3Request('PUT', $bucket, '');
$rest->setParameter('acl', null);
$rest->data = $dom->saveXML();
$rest->size = strlen($rest->data);
$rest->setHeader('Content-Type', 'application/xml');
$rest = $rest->getResponse();
if ($rest->error === false && $rest->code !== 200)
$rest->error = array('code' => $rest->code, 'message' => 'Unexpected HTTP status');
if ($rest->error !== false) {
trigger_error(sprintf("S3Withprogressbar::setAccessControlPolicy({$bucket}, {$uri}): [%s] %s",
$rest->error['code'], $rest->error['message']), E_USER_WARNING);
return false;
}
return true;
}
public static function getAccessControlPolicy($bucket, $uri = '') {
$rest = new S3Request('GET', $bucket, $uri);
$rest->setParameter('acl', null);
$rest = $rest->getResponse();
if ($rest->error === false && $rest->code !== 200)
$rest->error = array('code' => $rest->code, 'message' => 'Unexpected HTTP status');
if ($rest->error !== false) {
trigger_error(sprintf("S3Withprogressbar::getAccessControlPolicy({$bucket}, {$uri}): [%s] %s",
$rest->error['code'], $rest->error['message']), E_USER_WARNING);
return false;
}
$acp = array();
if (isset($rest->body->Owner, $rest->body->Owner->ID, $rest->body->Owner->DisplayName)) {
$acp['owner'] = array(
'id' => (string)$rest->body->Owner->ID, 'name' => (string)$rest->body->Owner->DisplayName
);
}
if (isset($rest->body->AccessControlList)) {
$acp['acl'] = array();
foreach ($rest->body->AccessControlList->Grant as $grant) {
foreach ($grant->Grantee as $grantee) {
if (isset($grantee->ID, $grantee->DisplayName)) // CanonicalUser
$acp['acl'][] = array(
'type' => 'CanonicalUser',
'id' => (string)$grantee->ID,
'name' => (string)$grantee->DisplayName,
'permission' => (string)$grant->Permission
);
elseif (isset($grantee->EmailAddress)) // AmazonCustomerByEmail
$acp['acl'][] = array(
'type' => 'AmazonCustomerByEmail',
'email' => (string)$grantee->EmailAddress,
'permission' => (string)$grant->Permission
);
elseif (isset($grantee->URI)) // Group
$acp['acl'][] = array(
'type' => 'Group',
'uri' => (string)$grantee->URI,
'permission' => (string)$grant->Permission
);
else continue;
}
}
}
return $acp;
}
public static function deleteObject($bucket = '', $uri = '') {
$rest = new S3Request('DELETE', $bucket, $uri);
$rest = $rest->getResponse();
if ($rest->error === false && $rest->code !== 204)
$rest->error = array('code' => $rest->code, 'message' => 'Unexpected HTTP status');
if ($rest->error !== false) {
trigger_error(sprintf("S3Withprogressbar::deleteObject(): [%s] %s", $rest->error['code'], $rest->error['message']), E_USER_WARNING);
return false;
}
return true;
}
public static function __getMimeType(&$file) {
$type = false;
// Fileinfo documentation says fileinfo_open() will use the
// MAGIC env var for the magic file
if (extension_loaded('fileinfo') && isset($_ENV['MAGIC']) &&
($finfo = finfo_open(FILEINFO_MIME, $_ENV['MAGIC'])) !== false) {
if (($type = finfo_file($finfo, $file)) !== false) {
// Remove the charset and grab the last content-type
$type = explode(' ', str_replace('; charset=', ';charset=', $type));
$type = array_pop($type);
$type = explode(';', $type);
$type = array_shift($type);
}
finfo_close($finfo);
// If anyone is still using mime_content_type()
} elseif (function_exists('mime_content_type'))
$type = mime_content_type($file);
if ($type !== false && strlen($type) > 0) return $type;
// Otherwise do it the old fashioned way
static $exts = array(
'jpg' => 'image/jpeg', 'gif' => 'image/gif', 'png' => 'image/png',
'tif' => 'image/tiff', 'tiff' => 'image/tiff', 'ico' => 'image/x-icon',
'swf' => 'application/x-shockwave-flash', 'pdf' => 'application/pdf',
'zip' => 'application/zip', 'gz' => 'application/x-gzip',
'tar' => 'application/x-tar', 'bz' => 'application/x-bzip',
'bz2' => 'application/x-bzip2', 'txt' => 'text/plain',
'asc' => 'text/plain', 'htm' => 'text/html', 'html' => 'text/html',
'xml' => 'text/xml', 'xsl' => 'application/xsl+xml',
'ogg' => 'application/ogg', 'mp3' => 'audio/mpeg', 'wav' => 'audio/x-wav',
'avi' => 'video/x-msvideo', 'mpg' => 'video/mpeg', 'mpeg' => 'video/mpeg',
'mov' => 'video/quicktime', 'flv' => 'video/x-flv', 'php' => 'text/x-php'
);
$ext = strToLower(pathInfo($file, PATHINFO_EXTENSION));
return isset($exts[$ext]) ? $exts[$ext] : 'application/octet-stream';
}
public static function __getSignature($string) {
return 'AWS '.self::$__accessKey.':'.base64_encode(extension_loaded('hash') ?
hash_hmac('sha1', $string, self::$__secretKey, true) : pack('H*', sha1(
(str_pad(self::$__secretKey, 64, chr(0x00)) ^ (str_repeat(chr(0x5c), 64))) .
pack('H*', sha1((str_pad(self::$__secretKey, 64, chr(0x00)) ^
(str_repeat(chr(0x36), 64))) . $string)))));
}
}
final class S3Request {
private $verb, $bucket, $uri, $resource = '', $parameters = array(),
$amzHeaders = array(), $headers = array(
'Host' => '', 'Date' => '', 'Content-MD5' => '', 'Content-Type' => ''
);
public $fp = false, $size = 0, $data = false, $response;
public $uploadProgressFileName;
function __construct($verb, $bucket = '', $uri = '') {
$this->verb = $verb;
$this->bucket = strtolower($bucket);
$this->uri = $uri !== '' ? '/'.$uri : '/';
if ($this->bucket !== '') {
$this->bucket = explode('/', $this->bucket);
$this->resource = '/'.$this->bucket[0].$this->uri;
$this->headers['Host'] = $this->bucket[0].'.s3.amazonaws.com';
$this->bucket = implode('/', $this->bucket);
} else {
$this->headers['Host'] = 's3.amazonaws.com';
if (strlen($this->uri) > 1)
$this->resource = '/'.$this->bucket.$this->uri;
else $this->resource = $this->uri;
}
$this->headers['Date'] = gmdate('D, d M Y H:i:s T');
$this->response = new STDClass;
$this->response->error = false;
$this->response->body  = false; 
$CI =& get_instance();
$CI->load->helper('file');
}
public function setParameter($key, $value) {
$this->parameters[$key] = $value;
}
public function setHeader($key, $value) {
$this->headers[$key] = $value;
}
public function setAmzHeader($key, $value) {
$this->amzHeaders[$key] = $value;
}
//:callback function for upload progress.
public function progressCallback_new_curl($new,$download_size, $downloaded_size, $upload_size, $uploaded_size )
{
ob_start();
static $previousProgress = 0;
if ( $upload_size == 0 )
$progress = 0;
else
$progress = round( $uploaded_size * 100 / $upload_size );
if ( $progress > $previousProgress)
{
flush();
$previousProgress = $progress;
$new_file_path = FCPATH.'upload/'.$this->uploadProgressFileName.'.txt'; // modify this line to point to the actual location of the file
write_file($new_file_path, $progress."\n");
// Check for Cancel upload text file and if exists return 1 to interrupt curl upload process.
$cancel_path=FCPATH.'upload/cancel_'.$this->uploadProgressFileName.'.txt';
if(file_exists($cancel_path)){
unlink($cancel_path);
if(file_exists($new_file_path)){
unlink($new_file_path);
}
return 1;
}
}        
ob_flush();
flush();                         
}
public function progressCallback_old_curl($download_size, $downloaded_size, $upload_size, $uploaded_size )
{
ob_start();
static $previousProgress = 0;
if ( $upload_size == 0 )
$progress = 0;
else
$progress = round( $uploaded_size * 100 / $upload_size );
if ( $progress > $previousProgress)
{
flush();
$previousProgress = $progress;
$new_file_path = FCPATH.'upload/'.$this->uploadProgressFileName.'.txt'; // modify this line to point to the actual location of the file
write_file($new_file_path, $progress."\n");
// Check for Cancel upload text file and if exists return 1 to interrupt curl upload process.
$cancel_path=FCPATH.'upload/cancel_'.$this->uploadProgressFileName.'.txt';
if(file_exists($cancel_path)){
unlink($cancel_path);
if(file_exists($new_file_path)){
unlink($new_file_path);
}
return 1;
}  
}
ob_flush();
flush(); 
}
public function getResponse() {
$query = '';
if (sizeof($this->parameters) > 0) {
$query = substr($this->uri, -1) !== '?' ? '?' : '&';
foreach ($this->parameters as $var => $value)
if ($value == null || $value == '') $query .= $var.'&';
else $query .= $var.'='.$value.'&';
$query = substr($query, 0, -1);
$this->uri .= $query;
if (isset($this->parameters['acl']) || !isset($this->parameters['logging']))
$this->resource .= $query;
}
$url = (extension_loaded('openssl')?'https://':'http://').$this->headers['Host'].$this->uri;
//var_dump($this->bucket, $this->uri, $this->resource, $url);
// Basic setup
$curl = curl_init();
curl_setopt($curl, CURLOPT_USERAGENT, 'S3/php');
curl_setopt($curl, CURLOPT_SSL_VERIFYHOST, 0);
curl_setopt($curl, CURLOPT_SSL_VERIFYPEER, 0);
curl_setopt($curl, CURLOPT_URL, $url);
//:callback function call for upload progress.
curl_setopt($curl, CURLOPT_NOPROGRESS, false);
if (!defined('PHP_VERSION_ID') || PHP_VERSION_ID < 50500) {
curl_setopt($curl, CURLOPT_PROGRESSFUNCTION,  array($this, 'progressCallback_old_curl'));
} else {
curl_setopt($curl, CURLOPT_PROGRESSFUNCTION,  array($this, 'progressCallback_new_curl'));
}
// Headers
$headers = array(); $amz = array();
foreach ($this->amzHeaders as $header => $value)
if (strlen($value) > 0) $headers[] = $header.': '.$value;
foreach ($this->headers as $header => $value)
if (strlen($value) > 0) $headers[] = $header.': '.$value;
foreach ($this->amzHeaders as $header => $value)
if (strlen($value) > 0) $amz[] = strToLower($header).':'.$value;
$amz = (sizeof($amz) > 0) ? "\n".implode("\n", $amz) : '';
// Authorization string
$headers[] = 'Authorization: ' . S3Withprogressbar::__getSignature(
$this->verb."\n".
$this->headers['Content-MD5']."\n".
$this->headers['Content-Type']."\n".
$this->headers['Date'].$amz."\n".$this->resource
);
curl_setopt($curl, CURLOPT_HTTPHEADER, $headers);
curl_setopt($curl, CURLOPT_HEADER, false);
curl_setopt($curl, CURLOPT_RETURNTRANSFER, false);
curl_setopt($curl, CURLOPT_WRITEFUNCTION, array(&$this, '__responseWriteCallback'));
curl_setopt($curl, CURLOPT_HEADERFUNCTION, array(&$this, '__responseHeaderCallback'));
// Request types
switch ($this->verb) {
case 'GET': break;
case 'PUT':
if ($this->fp !== false) {
curl_setopt($curl, CURLOPT_PUT, true);
curl_setopt($curl, CURLOPT_INFILE, $this->fp);
if ($this->size > 0)
curl_setopt($curl, CURLOPT_INFILESIZE, $this->size);
} elseif ($this->data !== false) {
curl_setopt($curl, CURLOPT_CUSTOMREQUEST, 'PUT');
curl_setopt($curl, CURLOPT_POSTFIELDS, $this->data);
if ($this->size > 0)
curl_setopt($curl, CURLOPT_BUFFERSIZE, $this->size);
} else
curl_setopt($curl, CURLOPT_CUSTOMREQUEST, 'PUT');
break;
case 'HEAD':
curl_setopt($curl, CURLOPT_CUSTOMREQUEST, 'HEAD');
curl_setopt($curl, CURLOPT_NOBODY, true);
break;
case 'DELETE':
curl_setopt($curl, CURLOPT_CUSTOMREQUEST, 'DELETE');
break;
default: break;
}
// Execute, grab errors
if (curl_exec($curl))
$this->response->code = curl_getinfo($curl, CURLINFO_HTTP_CODE);
else
$this->response->error = array(
'code' => curl_errno($curl),
'message' => curl_error($curl),
'resource' => $this->resource
);
@curl_close($curl);
// Parse body into XML
if ($this->response->error === false && isset($this->response->headers['type']) &&
$this->response->headers['type'] == 'application/xml' && isset($this->response->body)) {
$this->response->body = simplexml_load_string($this->response->body);
// Grab S3 errors
if (!in_array($this->response->code, array(200, 204)) &&
isset($this->response->body->Code, $this->response->body->Message)) {
$this->response->error = array(
'code' => (string)$this->response->body->Code,
'message' => (string)$this->response->body->Message
);
if (isset($this->response->body->Resource))
$this->response->error['resource'] = (string)$this->response->body->Resource;
unset($this->response->body);
}
}
// Clean up file resources
if ($this->fp !== false && is_resource($this->fp)) fclose($this->fp);
return $this->response;
}
private function __responseWriteCallback(&$curl, &$data) {
if ($this->response->code == 200 && $this->fp !== false)
return fwrite($this->fp, $data);
else
$this->response->body .= $data;
return strlen($data);
}
private function __responseHeaderCallback(&$curl, &$data) {
if (($strlen = strlen($data)) <= 2) return $strlen;
if (substr($data, 0, 4) == 'HTTP')
$this->response->code = (int)substr($data, 9, 3);
else {
list($header, $value) = explode(': ', trim($data));
if ($header == 'Last-Modified')
$this->response->headers['time'] = strtotime($value);
elseif ($header == 'Content-Length')
$this->response->headers['size'] = (int)$value;
elseif ($header == 'Content-Type')
$this->response->headers['type'] = $value;
elseif ($header == 'ETag')
$this->response->headers['hash'] = substr($value, 1, -1);
elseif (preg_match('/^x-amz-meta-.*$/', $header))
$this->response->headers[$header] = is_numeric($value) ? (int)$value : $value;
}
return $strlen;
}
}
?>