How to respect EXIF orientations when creating ID2D1Bitmap1 from stream

Question

When I construct a bitmap from a random access stream, I want to have it rotated automatically according to its EXIF orientation. Let's extend the following code snippet:

ComPtr<IWICImagingFactory2> wicFactory2; // Details of factory creation omitted
ComPtr<IWICBitmapDecoder> wicBitmapDecoder;
wicFactory2->CreateDecoderFromStream(
    stream.Get(), // stream is a valid ComPtr<IStream>
    nullptr,
    WICDecodeMetadataCacheOnDemand,
    &wicBitmapDecoder
    );

ComPtr<IWICBitmapFrameDecode> wicFrameDecode;
wicBitmapDecoder->GetFrame(
    0,
    &wicFrameDecode
    );

ComPtr<IWICFormatConverter> wicFormatConverter;
wicFactory2->CreateFormatConverter(
    &wicFormatConverter
    );

wicFormatConverter->Initialize(
    wicFrameDecode.Get(),
    GUID_WICPixelFormat32bppPBGRA,
    WICBitmapDitherTypeNone,
    NULL,
    0.f,
    WICBitmapPaletteTypeMedianCut
    );

// <-- What code to insert here to respect EXIF orientation???

ComPtr<ID2D1Bitmap1> bitmap;
m_d2dContext->CreateBitmapFromWicBitmap(
    wicFormatConverter.Get(),
    &bitmap
    );

The behavior I'm looking for is similar to calling BitmapDecoder.GetPixelDataAsync with ExifOrientationMode set to RespectExifOrientation. However, I cannot use this function because the bitmap shall serve as input to a Direct2D effect, so there is no need to access the raw pixel data by CPU.

In the worst case you can write our own code to handle the EXIT orientation. IWICBitmapFrameDecode has functions for reading metadata fields like EXIF orientation. And IWICBitmapFlipRotator provides functionality for flipping WIC image. — Anton Angelov, Mar 01 '15 at 18:59

Kristian Andaker · Answer 1 · 2016-12-21T05:14:46.707

I had this same problem. I couldn't figure out a way to have WIC or DirectX apply the EXIF orientation instructions automatically when creating a Texture2D directly from a WIC bitmap without passing through a byte[] as with GetPixelDataAsync. So instead I read EXIF property 0x112 and figure out the orientations to apply as suggested in one of the answer to Problem reading JPEG Metadata (Orientation). Then I insert a IWICBitmapFlipRotator on top of the IWICFormatConverter you have in the code above, and specify the BitmapTransformOptions to match the EXIF instructions.

I'm writing my code in C# with SharpDX, so the syntax is a little different than your C++ above. But it should be straightforward to translate:

Getting the metadata:

        using (MetadataQueryReader metadata = bitmapObjects.frame.MetadataQueryReader)
        {
            EXIForientation = (Nullable<UInt16>)metadata.TryGetMetadataByName("/app1/{ushort=0}/{ushort=274}"); //0x0112
        }

Converting the EXIF orientation info to the variables I need (copy/pasted and a little modified from the other article I linked to. I found a bug in that other article's code - for case '5'):

            // If the EXIF orientation specifies that the image needs to be flipped or rotated before display, set that up to happen
        BitmapTransformOptions bitmapTransformationOptions = BitmapTransformOptions.Rotate0;
        bool flipHeightWidth = false;
        if (EXIForientation != null)
        {
            switch (EXIForientation)
            {
                case 1:
                    // No rotation required.
                    break;
                case 2:
                    bitmapTransformationOptions = BitmapTransformOptions.Rotate0 | BitmapTransformOptions.FlipHorizontal;
                    break;
                case 3:
                    bitmapTransformationOptions = BitmapTransformOptions.Rotate180;
                    break;
                case 4:
                    bitmapTransformationOptions = BitmapTransformOptions.Rotate180 | BitmapTransformOptions.FlipHorizontal;
                    break;
                case 5:
                    bitmapTransformationOptions = BitmapTransformOptions.Rotate270 | BitmapTransformOptions.FlipHorizontal;
                    flipHeightWidth = true;
                    break;
                case 6:
                    bitmapTransformationOptions = BitmapTransformOptions.Rotate90;
                    flipHeightWidth = true;
                    break;
                case 7:
                    bitmapTransformationOptions = BitmapTransformOptions.Rotate90 | BitmapTransformOptions.FlipHorizontal;
                    flipHeightWidth = true;
                    break;
                case 8:
                    bitmapTransformationOptions = BitmapTransformOptions.Rotate270;
                    flipHeightWidth = true;
                    break;
            }
            if (flipHeightWidth)
            {
                originalWidth = bitmapObjects.frame.Size.Height;
                originalHeight = bitmapObjects.frame.Size.Width;
            }
        }

The bitmap creation I do (with slightly different transforms than the ones you have plugged into that pipeline):

        bitmapObjects.scaler.Initialize(bitmapObjects.frame, (int)bitmapRectangle.Width, (int)bitmapRectangle.Height, SharpDX.WIC.BitmapInterpolationMode.Linear);
        bitmapObjects.pixelFormatConverter.Initialize(bitmapObjects.scaler, SharpDX.WIC.PixelFormat.Format32bppPBGRA);
        bitmapObjects.bitmapFlipRotator.Initialize(bitmapObjects.pixelFormatConverter, bitmapTransformationOptions);
        bitmapObjects.bitmap = SharpDX.Direct2D1.Bitmap1.FromWicBitmap(D2DObjectCollection.D2Dcontext2,
            bitmapObjects.bitmapFlipRotator,
            new BitmapProperties1(
                new SharpDX.Direct2D1.PixelFormat(
                    Format.B8G8R8A8_UNorm,
                    SharpDX.Direct2D1.AlphaMode.Ignore),
                96,
                96,
                BitmapOptions.Target));

As an aside, I'm keeping all the objects used to render the bitmap around in the 'bitmapObjects' collection, so I can dispose them in the reverse order of how they were initialized. I've found that if I don't do that, I get very rare instances of a bizarre memory access violation from deep within a Windows Codecs DLL.

Edit Since implementing this, I realized something to watch out for with this solution: When the scaler is first in the pipeline, it (in my app scenario) reduces the size of the decoded bitmap significantly for most of today's large jpg images from modern cameras. That's great for performance. However, to make the rotation and flip apply correctly (unlike what's in the code segment I shared above), I have to put that at the start of the pipeline, which causes decoding into a bitmap the size of the resolution in the image file. To fix this as best I could, I changed my code to only apply the bitmapFlipRotator when a particular file actually has orientation instructions in it. Otherwise, the bitmapFlipRotator isn't used.

How to respect EXIF orientations when creating ID2D1Bitmap1 from stream

1 Answers1