Converting System.Drawing Bitmap to Dlib Array2D

Question

In this case, a grayscale Array2D for ShapePredictor.

Here is what I am trying, without much success.

using DlibDotNet;
using Rectangle = System.Drawing.Rectangle;
using System.Runtime.InteropServices;

public static class Extension
{
    public static Array2D<byte> ToArray2D(this Bitmap bitmap)
    {
        var bits = bitmap.LockBits(new Rectangle(0, 0, bitmap.Width, bitmap.Height), ImageLockMode.ReadOnly, PixelFormat.Format32bppPArgb);
        var length = bits.Stride * bits.Height;
        var data = new byte[length];

        Marshal.Copy(bits.Scan0, data, 0, length);
        bitmap.UnlockBits(bits);

        var array = new Array2D<byte>(bitmap.Width, bitmap.Height);

        for (var x = 0; x < bitmap.Width; x++)
            for (var y = 0; y < bitmap.Height; y++)
            {
                var offset = x * 4 + y * bitmap.Width * 4;

                array[x][y] = data[offset];
            }

        return array;
    }

I've searched and have not yet found a clear answer.

You are accessing the data as 32bpp, but trying to put it in an Array2D of `Byte`. You will probably need `Aray2D` instead, and read per 4 bytes. — Nyerguds, Apr 19 '18 at 00:10
Notice that I am using offset `offset = x * 4 + y * bitmap.Width` for that same reason. — O. Coder, Apr 19 '18 at 07:26
Yes, but isn't your return type wrong if it only has one byte in each entry of your 2D array? You'll need a full `Int32` (possibly `UInt32`) for 32bpp pixels. I don't know ShapePredictor, so I don't know what kind of data it expects, but you can't put four bytes of data into one byte. — Nyerguds, Apr 19 '18 at 07:28
The return type is fine. ShapePredictor works with grayscale (8-bit). An `Array2D array = Dlib.LoadPng("file.png")` loaded from file works as expected, but what I really need is to be able to convert a Bitmap into Array2D, so that I can work with data that is already loaded in memory. — O. Coder, Apr 19 '18 at 07:51
You need more than that. You need code to _actually convert your image to grayscale_ too then. You have no such thing in your code; you're accessing it as 32bpp alpha-capable 16 million colour data. Such conversions are better performed _before_ accessing it as bytes. — Nyerguds, Apr 19 '18 at 07:53
For now, I would be satisfied with using just one colour channel, in order to have a working solution. Later, I may write a robust version. — O. Coder, Apr 19 '18 at 08:01
I think I found your error. You remarked that you are using `offset = x * 4 + y * bitmap.Width`, but in fact _you aren't_. You're multiplying _that_ by 4 _again_. That said, doing multiple multiplications inside the deepest loop is pretty inefficient (and, to be honest, looping over the data per column is pretty weird in itself), so I advise you use the code I provided, which iterates over the image data in the order of the actual image data. — Nyerguds, Apr 19 '18 at 08:28

Nyerguds · Accepted Answer · 2018-04-19T10:25:27.863

As noted before, you first need to convert your image to grayscale. There are plenty of answers here on StackOverflow to help you with that. I advise the ColorMatrix method used in this answer:

A: Convert an image to grayscale

I'll be using the MakeGrayscale3(Bitmap original) method shown in that answer in my code below.

Typically, images are looped through line by line for processing, so for clarity, you should put your Y loop as outer loop. It also makes the calculation of the data offsets a lot more efficient.

As for the actual data, if the image is grayscale, the R, G and B bytes should all be the same. The order "ARGB" in 32-bit pixel data refers to one UInt32 value, but those are little-endian, meaning the actual order of the bytes is [B, G, R, A]. This means that in each loop iteration we can just take the first of the four bytes, and it'll be the blue component.

public static Array2D<Byte> ToArray2D(this Bitmap bitmap)
{
    Int32 stride;
    Byte[] data;
    // Removes unnecessary getter calls.
    Int32 width = bitmap.Width;
    Int32 height = bitmap.Height;
    // 'using' block to properly dispose temp image.
    using (Bitmap grayImage = MakeGrayscale(bitmap))
    {
        BitmapData bits = grayImage.LockBits(new Rectangle(0, 0, width, height), ImageLockMode.ReadOnly, PixelFormat.Format32bppPArgb);
        stride = bits.Stride;
        Int32 length = stride*height;
        data = new Byte[length];
        Marshal.Copy(bits.Scan0, data, 0, length);
        grayImage.UnlockBits(bits);
    }
    // Constructor is (rows, columns), so (height, width)
    Array2D<Byte> array = new Array2D<Byte>(height, width);
    Int32 offset = 0;
    for (Int32 y = 0; y < height; y++)
    {
        // Offset variable for processing one line
        Int32 curOffset = offset;
        // Get row in advance
        Array2D<Byte>.Row<Byte> curRow = array[y];
        for (Int32 x = 0; x < width; x++)
        {
            curRow[x] = data[curOffset]; // Should be the Blue component.
            curOffset += 4;
        }
        // Stride is the actual data length of one line. No need to calculate that;
        // not only is it already given by the BitmapData object, but in some situations
        // it may differ from the actual data length. This also saves processing time
        // by avoiding multiplications inside each loop.
        offset += stride;
    }
    return array;
}

Your code is more lean, though I noticed a few typos, easily fixed. The conversion still does not work. The main problem is that initialization of Array2D data seems to be undocumented, which is why I am doing this in an array loop. I have seen lots of people asking how to do it, but no definitive answer as of yet. — O. Coder, Apr 19 '18 at 08:42
@O.Coder I think I found it. The DlibDotNet source code uses `rows` and `columns`, in that order. That means _height_ and _width_. You switched those everywhere. The multidimensional array is, as I already noted, a lot more logical if seen per row, so the access is also `array[y][x]`. — Nyerguds, Apr 19 '18 at 10:21
It also seems every retrieve of a row does a function call, so it's a lot more efficient to retrieve it outside the X loop. I edited the code to show that. — Nyerguds, Apr 19 '18 at 10:30

Converting System.Drawing Bitmap to Dlib Array2D

1 Answers1