3

We have a web app using Andrew Valums ajax file uploader, if we kick off 5 - 10 image uploads at once, more often then not at least 2 or 3 will result in the same gd error "Corrupt JPEG data"

Warning: imagecreatefromjpeg() [function.imagecreatefromjpeg]:
         gd-jpeg, libjpeg: recoverable error: Corrupt JPEG data:
         47 extraneous bytes before marker 0xd9 in ....

However this did not happen on our old test server, or local development box's, only on our new production server.

The file size on the server is the same as the original on my local machine, so it completes the upload but I think the data is being corrupted by the server.

I can "fix" the broken files by deleting them and uploading again, or manually uploading via FTP

We had a shared host on Godaddy and just have started to have this issue on a new box (that I set up, so probably explains a lot :) CentOS 5.5+, Apache 2.2.3, PHP 5.2.10

You can see some example good and bad picture here. http://174.127.115.220/temp/pics.zip

When I BinDiffed them I see a consistent pattern the corruption is always 64 byte blocks, and while the distance between corrupted blocks is not constant the number 4356 comes up a lot.

I really think we can rule out the Internet as error checking and retransmission with TCP is pretty reliable, further there seems to be no difference between browser versions, or if I turn anti-virus and firewalls off.

So I'm picking configuration of Apache / PHP?

Blender
  • 289,723
  • 53
  • 439
  • 496
dave.zap
  • 491
  • 4
  • 13
  • 1
    I probably can't help you here, but this is a good question. Nice work. – rockerest Jul 10 '11 at 23:10
  • This might not be a solution, but have you tried a different webserver? There's always `nginx` and `lighttpd`. – Blender Jul 10 '11 at 23:13
  • Yes I'm sure a different server would solve the problem, as it only started on the new server, this is why I feel it is a Apache/PHP config issue. – dave.zap Jul 11 '11 at 03:23
  • When I get time today I might follow the ideas in this thread https://bugs.php.net/bug.php?id=19263. My other thought is to check if the data is corrupt before php copies it from tmp. – dave.zap Jul 11 '11 at 03:24
  • the idea in the forum thread above did not help. I also tried disabling the session_write_close() that allowed concurrent uploads, no dice. I also switched the JSUploader from XMLHttpRequest() uploading to older iframe form post method, still the same result. – dave.zap Jul 11 '11 at 22:29
  • Thanks for writing this up, I'm going to try a couple of the thing you mention for troubleshooting-- I think I'm having the same issue w/ trinidad and nginx, on engineyard but not my local machine! http://stackoverflow.com/questions/9236143/image-corruption-on-upload-to-s3-production-only-carrierwave-engineyard – Peter Ehrlich Feb 23 '12 at 07:34

5 Answers5

2

Some cameras will append some data inside the file that will get interpreted incorrectly (most likely do to character encoding with in the headers).

A solution I found was to read the file in binary mode like so

$fh = fopen('test.jpg', 'rb');
$str = '';
while($fh !== false && !feof($fh)){
    $str .= fread($fh, 1024);
}

$test = @imagecreatefromstring($str);

imagepng($test,'save.png');
MANCHUCK
  • 2,424
  • 1
  • 16
  • 22
1

Well, i think the problem is jpeg-header data, and as far as i know there is nothing to do with it by PHP, i think the problem is your fileuploader, maybe there are some configuration for it that you are missing.

  • The problem is the jpeg-header data, however I disagree that it is the fault of the JS Uploader, or even the PHP script side of the uploader as we had this running on two separate servers (Godaddy, and localhost on laptop) and were not able to reproduce the error, it only occurs on our new production server. – dave.zap Jul 11 '11 at 22:27
  • If i where you, i would test simple upload script to make sure everything is fine about server works. you may want to solve this problem by creating jpeg-header manually (i don't say this is the right thing to do), take a look at this: [link](http://en.wikipedia.org/wiki/JPEG#Syntax_and_structure) –  Jul 12 '11 at 07:59
0

This can be solved with:

ini_set ('gd.jpeg_ignore_warning', 1);
حسین عبایی
  • 19
  • 2
0

I had this problem with GoDaddy hosting. I had created the database on GoDaddy using their cPanel interface. It was created as "latin collation" (or something like that). The database on the development server was UTF8. I've tried all solutions on this page, to no avail. Then I converted the database to UTF8, and it worked.

Database encoding shouldn't affect BLOB data (or so I would think). BLOB stands for BINARY Large Object (something...), to my knowledge!

Also, strangely, the data was copied from the dev to production server while the database was still "latin", and it was not corrupted at all. It's only when inserting new images that the problem appeared. So I guess the image data was being fed to MySQL as text data, and I think there is a way (when using SQL) of inserting binary data, and I did not follow it.

Edit: just took a look at the MySQL export script, here it is:
INSERT INTO ... VALUES (..., _binary 0xFFD8FF ...

Anyway, hope this will help someone. The OP did not indicate what solved his problem...

Rolf
  • 5,550
  • 5
  • 41
  • 61
0

Hmm - a 64 byte corruption?...or did you mean 64 bit?

I'm going to suggest that the issue is in fact as a result of the PHP script. the problem that regularly comes up here is that the script inserts CRLFs into the data stream being uploaded, and is caused by differences between the Window/*nix standards.

Solution is to force the php script to upload in binary mode (use the +b switch for ALL fopen() commands in the php upload). It is safe to upload a text file in binary mode as at least you can still see the data.

Read here for more information on this issue:

http://us2.php.net/manual/en/function.fopen.php

Peter Edmond
  • 1