0

Now before anyone tries consider this question a duplicate, I'd like to say that I've looked at all the similar questions on SO and other forums to absolutely no avail. I tried all the solutions posted and nothing worked. With that being said, here's the problem:

I have a large set of jpg/jpeg images (6000-20000), each of maximum 10 KB in size, each of dimensions 512x512 pixels, being uploaded on my Azure web + MySQL setup. I figured out how to upload that many files without breaking the code or our service and everything is cool.

What I do with these images once they are uploaded is that I build a tiling pyramid of sorts. If there are 5000 images, I tile 'em together in batches of 4 images at a time to form a single image, resample the resultant image using PHP to half its size, and store it as a new tile in a folder representing another level in this pyramid. Hence if all the base files (that were uploaded) are stored in Folder-1 then the new images being created are stored in Folder-2, then Folder-3, and so on until there aren't any images left to tile. At this point all the images have been stitched together and sampled down to 1 image. Now for those of you familiar with deep zoom concepts on the web, you would understand why this is necessary.

Now, here's the real problem. All of this works perfectly well on localhost. I ported this code to a personal server of mine which is CentOS running LAMP and it worked perfectly fine. However, on the company server which is a free tier Azure Web + MySQL instance, the process return an internal server error.

To be more specific, this pyramid functionality is invoked by jQuery AJAX once the upload function returns a success. As part of the process on PHP, I move the uploaded PHP files from the designated temporary location to its actual upload location and begin processing the images for tiling. Usually, the entire pyramid generation takes between 5-15 minutes depending on the number of base images I uploaded. These statistics are based on how it worked on a localhost. On the server, I'm hitting some sort of a limit or a timeout that seems to be killing my PHP after a few minutes. The images stop being processed after maybe two levels of creation. This process works well when the total images are less than 2000.

Sometimes, the AJAX fails. But, when I check the uploads folder via Filezilla, the images continue being processed since I see them being created by my script. At this point if my script doesn't complete in an unknown bit of time, it dies too. I arrived at this conclusion with the fact that no more images/folders for subsequent processing were being created.

Before you say it, my PHP.ini settings have been modified and I've restarted the server countless times before posting here. These are my current settings:

max_execution_time = 9999
memory_limit = 3G
post_max_size = 10G
upload_max_filesize = 1G
max_input_time = 9999

phpinfo() reflects this, too. .htaccess has no role here so trying it made no difference. It wasn't recognized. web.config seems to work and I've also added the line to it:

<system.web>
<httpRuntime executionTimeout="9999" maxRequestLength="2097151"/>
</system.web>

This makes no difference, too. I know the web.config file works because I tried to play around with other settings in it and the changes do reflect in the web app.

So, what is it that I'm missing? I tried raising a support ticket but Azure doesn't let me don that unless I'm on the paid package. I can post my code if required but I'm quite certain that it isn't necessary since it works perfectly well on my localhost and on my personal Linux server.

I've spent way too much time on this problem. Any help would be appreciated. Thanks in advance!

Anoop Santhanam
  • 85
  • 1
  • 10
  • I take it you have a server large enough to handle this. – Millard Feb 27 '17 at 14:20
  • 2
    Have you checked the error logs on the server? – Jay Blanchard Feb 27 '17 at 14:21
  • Yes. From what Azure shows me, it has 1 core, 1.75 GB of RAM, and 50GB of storage. From what I've measured, the image processing doesn't take more than 10 MB. Plus, the processing is sequential; I'm not sure why it would blow past any RAM limits. – Anoop Santhanam Feb 27 '17 at 14:23
  • `post_max_size` is part of `memory_limit` So a larger `post_max_size` than a `memory_limit` could be causing you issues. Its no good just throwing large numbers at these settings, you need to think what you are doing as well – RiggsFolly Feb 27 '17 at 14:24
  • @JayBlanchard I should have added in the original question that I'm Azure noob and I pretty much dived into all of this coding. I do believe that the error logging is on, but I have no idea where to access it from. I hunted all over using FileZilla but I didn't find what I was looking for. – Anoop Santhanam Feb 27 '17 at 14:24
  • @RiggsFolly Let me change this and see if it makes any difference. – Anoop Santhanam Feb 27 '17 at 14:25
  • 3
    You should be looking at the error logs for the web server you're using. – Jay Blanchard Feb 27 '17 at 14:25
  • _Before you say it, my PHP.ini settings have been modified_ Can we assume these are the same on both TEST and LIVE server? – RiggsFolly Feb 27 '17 at 14:26
  • _I figured out how to upload that many files without breaking the code or our service and everything is cool_ So how did you do this? – RiggsFolly Feb 27 '17 at 14:27
  • @RiggsFolly Yep they are the same. – Anoop Santhanam Feb 27 '17 at 14:28
  • @RiggsFolly I used Chrome's webkitdirectory feature. This let me access any number of files contained within the folder. Then, to post it, I sent the images one by one. Our end customers have low internet speeds (less than 1 MBPS), so I had to make sure all the images get uploaded without anything missing. Plus, I threw in an additional functionality to enable them to continue uploading in case the process was interrupted. The files are named in a certain pattern. – Anoop Santhanam Feb 27 '17 at 14:31
  • @JayBlanchard Tried the logs. PHP error log shows no error. – Anoop Santhanam Feb 27 '17 at 17:33
  • @JayBlanchard At this point, I'm not even sure where the problem is happening. I know it's not in PHP because there's nothing in the logs. – Anoop Santhanam Feb 27 '17 at 17:34

1 Answers1

1

Currently, there is a 230 second (i.e. a little less than 4 mins) timeout for requests that are not sending any data back from Azure app service. So I think that raises your issue. You can check my earlier post from details.

Community
  • 1
  • 1
Aaron Chen
  • 9,835
  • 1
  • 16
  • 28
  • I'm going to try this and confirm. – Anoop Santhanam Feb 28 '17 at 06:05
  • Unfortunately for me, you are right, Aaron. I figured out a way to let the API respond even before it starts processing my data. And then I use AJAX to poll the image directory to check for completion. This works quite well. There's no way I can change Azure's behaviour. – Anoop Santhanam Mar 01 '17 at 10:48