If your task is to upload a huge data-set or process it on the server, while updating progress to the server you should consider going with some sort of jobs architecture, where you initiate the job and do it with some other script running on the server (for example scaling / processing images etc). In this you do one thing at a time, thus forming a pipeline of tasks where there is an input and a final processed output.
At each step of pipeline the status of task is updated inside the database which can then be sent to the user by any server-push mechanism which exists these days. Running a single script which handles uploads and updates puts load on your server and also restricts you (what if the browser closes, what if some other error occurs). When process is divided into steps you can resume a failed task from the point where it succeeded last.
There exists many ways to do it. But the overall process flow looks like this
The following method is what I did for a personal project and this script held good for uploading and processing thousands of high resolution image to my server which then were scaled down into multiple versions and uploaded to amazon s3 while recognizing objects inside them. (My original code was in python)
Step 1 :
Initiate the transport or task
First Upload your content and then return a transaction id or uuid for this transaction immediately via a simple POST request. If you are doing multiple files or multiple things in the task, you may also want to handle that logic in this step
Step 2:
Do the job & Return the progress.
Once you have figured out how transactions occur you can then use any server side push technology to send update packet. I would choose WebSocket or Server Sent Events whichever applicable falling back to Long Polling on un-supported browsers. A simple SSE method would look like this.
function TrackProgress(upload_id){
var progress = document.getElementById(upload_id);
var source = new EventSource('/status/task/' + upload_id );
source.onmessage = function (event) {
var data = getData(event); // your custom method to get data, i am just using json here
progress.setAttribute('value', data.filesDone );
progress.setAttribute('max', data.filesTotal );
progress.setAttribute('min', 0);
};
}
request.post("/me/photos",{
files: files
}).then(function(data){
return data.upload_id;
}).then(TrackProgress);
On the server side, you will need to create something which keeps a track of the tasks, a simple Jobs architecture with job_id and progress sent to the db shall suffice. I would leave the job scheduling to you and the routing as well, but after that the conceptual code (for simplest SSE which will suffice the above code) is as follows.
<?php
header('Content-Type: text/event-stream');
header('Cache-Control: no-cache');
/* Other code to take care of how do you find how many files are left
this is really not required */
function sendStatusViaSSE($task_id){
$status = getStatus($task_id);
$json_payload = array('filesDone' => $status.files_done,
'filesTotal' => $status.files_total);
echo 'data: ' . json_encode( $json_payload ) . '\n\n';
ob_flush();
flush();
// End of the game
if( $status.done ){
die();
}
}
while( True ){
sendStatusViaSSE( $request.$task_id );
sleep(4);
}
?>
A good tutorial on SSE can be found here http://html5doctor.com/server-sent-events/
and you can read more about pushing updates from the server on this question Pushing updates from server
The above was a conceptual explanation, there are other ways to achieve this but this was the solution that took care of a fairly huge task in my case.
