I am trying to get multiple images and put them through an algorithm. The format that I am using is YUV_420_8888
like this:
streamImageReader = ImageReader.newInstance(
imageSize.width,
imageSize.height,
ImageFormat.YUV_420_888,
30
)
After that, I need to convert the result android.media.Image
instance (the result image) into an RGB byte array. I am doing this with this extension function called with image.planes
:
fun Array<Plane>.YUV_420_888_toRGB(
context: Context,
width: Int,
height: Int
): ByteArray {
// Get the three image planes
val planes: Array<Plane> = this
var buffer: ByteBuffer = planes[0].buffer
buffer.rewind()
val y = ByteArray(buffer.capacity())
buffer.get(y)
buffer = planes[1].buffer
val u = ByteArray(buffer.capacity())
buffer.get(u)
buffer = planes[2].buffer
val v = ByteArray(buffer.capacity())
buffer.get(v)
// get the relevant RowStrides and PixelStrides
// (we know from documentation that PixelStride is 1 for y)
val yRowStride = planes[0].rowStride
val uvRowStride =
planes[1].rowStride // we know from documentation that RowStride is the same for u and v.
val uvPixelStride =
planes[1].pixelStride // we know from documentation that PixelStride is the same for u and v.
// rs creation just for demo. Create rs just once in onCreate and use it again.
val rs = RenderScript.create(context)
//RenderScript rs = MainActivity.rs;
val mYuv420 = ScriptC_yuv420888(rs)
// Y,U,V are defined as global allocations, the out-Allocation is the Bitmap.
// Note also that uAlloc and vAlloc are 1-dimensional while yAlloc is 2-dimensional.
val typeUcharY: Type.Builder = Type.Builder(rs, Element.U8(rs))
typeUcharY.setX(yRowStride).setY(height)
val yAlloc = Allocation.createTyped(rs, typeUcharY.create())
yAlloc.copy1DRangeFrom(0, y.size, y)
mYuv420.set_ypsIn(yAlloc)
val typeUcharUV: Type.Builder = Type.Builder(rs, Element.U8(rs))
// note that the size of the u's and v's are as follows:
// ( (width/2)*PixelStride + padding ) * (height/2)
// = (RowStride) * (height/2)
typeUcharUV.setX(u.size)
val uAlloc = Allocation.createTyped(rs, typeUcharUV.create())
uAlloc.copyFrom(u)
mYuv420._uIn = uAlloc
val vAlloc = Allocation.createTyped(rs, typeUcharUV.create())
vAlloc.copyFrom(v)
mYuv420._vIn = vAlloc
// handover parameters
mYuv420._picWidth = width.toLong()
mYuv420._uvRowStride = uvRowStride.toLong()
mYuv420._uvPixelStride = uvPixelStride.toLong()
val outBitmap = Bitmap.createBitmap(width, height, Bitmap.Config.ARGB_8888)
val outAlloc = Allocation.createFromBitmap(
rs,
outBitmap,
Allocation.MipmapControl.MIPMAP_NONE,
Allocation.USAGE_SCRIPT
)
val lo = LaunchOptions()
lo.setX(
0,
width
) // by this we ignore the y’s padding zone, i.e. the right side of x between width and yRowStride
lo.setY(0, height)
mYuv420.forEach_doConvert(outAlloc, lo)
outAlloc.copyTo(outBitmap)
return outBitmap.toByteArray()
}
The problem I am having is that this resulted byte array is rotated. I tried fixing it with this suggestion: Rotate an YUV byte array on Android
BUT, the resulted byte array after the algorithm above has half as many items as the first one (some loss here).
I was thinking that the easiest solution for this would be to rotate directly the Image, instead of the byte array.