4

Images from our website do not display in Safari for some Mac users and they report seeing either no image or a black image. Here is an example:

http://s3-eu-west-2.amazonaws.com/bp18.boxcleverpress.com/Boxclever_logo_chartreuse.png

What I have discovered is:

  • Images display on PC
  • Images display on SOME Macs (I have an older one that is OK)
  • Images display on iPhones and iPads
  • Images are PNG
  • I have optimised the images with pngtastic
  • When images are copied to the Mac and opened with Adobe Photoshop they give the error: the file format module cannot parse the file
  • When I tried to open a pngtastic optimised file in Photoshop Elements on Windows I also get that error
  • When I tried to open the optimised file in Photoshop on Windows I get the error IDAT: incorrect data check

I will replace the optimised images with unoptimised ones but I am not sure if this problem is with pngtastic or Adobe image libraries or something else.

Kevin Sadler
  • 2,306
  • 25
  • 33
  • Did replacing solve the issue? I also get complaints from Safari users about not seeing PNGs, but these were created with `imagecreatetruecolor` in PHP (standard GD library), not 'optimized' in any way. – Marten Koetsier May 05 '17 at 09:43
  • @Marten - yes replacing with the unoptimised images allowed them to be viewed. I hope to get some time to research the problem further and shall report back what I find. – Kevin Sadler May 09 '17 at 14:48

2 Answers2

4

The problem lies in Zopfli.java, included by pngtastic.

It uses this Java code to calculate the Adler-32 checksum:

/**
 * Calculates the adler32 checksum of the data
 */
private static int adler32(byte[] data) {
    int s1 = 1;
    int s2 = 1 >> 16;
    int i = 0;
    while (i < data.length) {
        int tick = Math.min(data.length, i + 1024);
        while (i < tick) {
            s1 += data[i++];
            s2 += s1;
        }
        s1 %= 65521;
        s2 %= 65521;
    }

    return (s2 << 16) | s1;
}

However, bytes in Java are always signed, and so it may return a wrong checksum value for some data inputs. Also, the bare int declarations for s1 and s2 cause further complications.

With (my C version of) the same code and data explicitly declared as signed char and both s1 and s2 as signed int, I get a wrong checksum FFFF9180 – exactly the one in your damaged PNG.

If I change the declaration to use unsigned char and unsigned int, it returns the correct checksum 1BCD6EB2 again.


The original C code for the Adler-32 checksum in zopfli uses unsigned types throughout, so it's just the Java implementation that suffers from this.

Jongware
  • 22,200
  • 8
  • 54
  • 100
  • 1
    Thank you for your investigations! I have reported the issue to the pngtastic project: https://github.com/depsypher/pngtastic/issues/13 – Kevin Sadler Jan 10 '18 at 13:18
  • Thanks for tracking this issue down! I've pushed a new pngtastic release with a fix based on these findings. – depsypher Jan 11 '18 at 02:23
  • It would be nice, if the upstream projects would be notified about problems like this. Indeed, code should look like this: `s1 += data[i++] & 0xFF;`. Fixed and thoroughly tested. – Eugene Kliuchnikov May 16 '18 at 09:24
1

The problem appears to be due to the use of the zopfli compression in the PNGs that I optimised using pngtastic. The workaround is to use a different pngtastic compression option and the PNGs are then readable in Photoshop.

Using a different compression algorithm will result in less optimisation.

I am not sure why the zopfli compression is a problem, it could be that there is a fault in my code (although the same code works fine when only the zopli option is changed), in pngtastic, or that MacOS and Adobe don't support zopfli.

@usr2564301 has done some investigation and it appears the Adler-32 checksum on the compressed data in my example image is incorrect. usr2564301 has also tested the pngtastic code and found it to produce the correct checksum. The problem might be in how I handle the bytestream out of pngtastic.

The code below performs the PNG optimisation using pngtastic (com.googlecode.pngtastic.core)

public static final String OPT_ZOPFLI = "zopfli";
public static final String OPT_DEFAULT = "default";
public static final String OPT_IMAGEOPTIM = "imageoptim";

private String optimization = OPT_ZOPFLI;

public void optimizePng(File infile, String out) {

     final InputStream in;
     try {
         in = new BufferedInputStream(new FileInputStream(infile));
         final PngImage image = new PngImage(in);

         // optimize
         final PngOptimizer optimizer = new PngOptimizer();

         optimizer.setCompressor(optimization, 1);
         final PngImage optimizedImage = optimizer.optimize(image, false, 9);

         // export the optimized image to a new file
         final ByteArrayOutputStream optimizedBytes = new ByteArrayOutputStream();
         optimizedImage.writeDataOutputStream(optimizedBytes);
         optimizedImage.export(out, optimizedBytes.toByteArray());

       } catch (FileNotFoundException e) {
         e.printStackTrace();
     } catch (IOException e) {
         e.printStackTrace();
     }
 }
Kevin Sadler
  • 2,306
  • 25
  • 33
  • 1
    Very interesting. Do these images pass `pngcheck`? – Jongware Jan 05 '18 at 09:59
  • 1
    Good point. That was the clue that lead me to my workaround. When I run pngcheck on the file I get the error: "zlib: inflate error = -3" which suggested to me I try a different compression, but I don't know what the problem with zopfli is, or why some machines or applications can read it. – Kevin Sadler Jan 05 '18 at 13:30
  • 1
    It is reason for concern – and not only for PNG users. It is claimed that "...Zopfli is bit-stream compatible with compression used in gzip, Zip, PNG, ..." (in less words: Flate) and your experience shows it is not. Mark Adler is a regular here and he may well be interested. – Jongware Jan 05 '18 at 15:15
  • I see what you mean. I'd be honoured to provide assistance to Mark Adler if he wanted it. The png file linked in the OP is still there to analyse. It does occur to me that the problem could be caused by my own code, but as it is just handling the data in and out of pngtastic, and creates valid PNGs when there is different compression option (change a parameter in the method call) it's hard to see how it could be (hopefully I am not being blinded by my own awesomeness ;) ) – Kevin Sadler Jan 05 '18 at 21:40
  • 1
    It took me some time to drill down to the core but the error in the sample file lies in the Adler-32 checksum at the end of the `zlib` stream. Can that possibly be caused by your part of the code? I verified `zopfli`s adler32 function at https://github.com/google/zopfli/blob/master/src/zopfli/zlib_container.c and it returns the correct value `1BCD6EB2`, the same as Python (which I used to determine the problem). – Jongware Jan 06 '18 at 17:11
  • What I do is create a PngOptimizer with either "zopfli" or "imageoptim" and generate a PngImage object. I then use the image's writeDataOutputStream() to write into a ByteArrayOutputStream and then use the image's export() method to create the file from the stream. I will see if I can identify where in this flow things are going wrong. I will add my code above. – Kevin Sadler Jan 09 '18 at 11:24