3

My goal is to create a scanner app by scratch. To do this, I am using the CameraX library in android in java. Assume that I have already detected the corners of the page. After the image is captured, how to I perform the transformation matrix multiplication on the image? (I am following the algorithm mentioned here except trying to do it in java) I would need the image in a matrix so that i can multipy each pixel position say [i, j, 1] by my 3x3 transformation matrix to get the new position. Then, i would have another matrix where i record the color of pixel that was transformed if it is inside the area that I want to show. Finally, i would convert the final matrix into an image.

So basically, my question(s) are:

  1. how do I do perform operations with the captured image in camerax in java
  2. how do I convert a image to a matrix (to do the stuff mentioned above) then a matrix back into an image

I want it so that when the user clicks the capture button, the latest coordinates of the corners of the document are recorded and used to transform the captured image using the transformation matrix.

Ak01
  • 362
  • 3
  • 19

1 Answers1

1

android.graphics.Matrix is the 3x3 perspective transformation matrix in Android. Example:

val matrix = Matrix()
matrix.setPolyToPoly(docVertexes, 0, bitmapVertexes, 0, 4)
Bitmap transformedBitmap = Bitmap.createBitmap(
            sourceBitmap,
            0,
            0,
            sourceBitmap.getWidth(),
            sourceBitmap.getHeight(),
            quadToRect,
            true);
Xi 张熹
  • 10,492
  • 18
  • 58
  • 86
  • but according to [this](https://stackoverflow.com/questions/34620327/select-an-area-on-bitmap-with-4-points-using-matrix-setpolytopoly) post, setPolyToPoly doesn't work on some cases. Plus, I want to implement the perspective warping and transforming myself and I already know how to do that. What I need is to convert image to and from Matrix and to do this with the camerax image capture specifically – Ak01 Jul 30 '20 at 17:58
  • The bug is not in Matrix.setPolyToPoly() but in Canvas. The root cause IIRC, is related to CPU rasterizer in the Skia graphic library used by Android. I have updated my sample in a way that won't trigger the bug. Do you mind explain why do you prefer to DIY? Doing it efficiently and correctly usually involves native C++ and/or GPU. It's a lot of work even you know all the math. – Xi 张熹 Jul 30 '20 at 22:00
  • I just want to DIY for a few reasons. One is that I gain an understanding of the math and code behind the process. Second, if I do it myself, it reduces the apk size considerably. When I first started this, I researched how others did this and they all used OpenCV and other Img. Proc. Libraries which reduced the code but made the app very large by adding redundant code. I will try to implement it with the method you mentioned, but do you know how to use this with camerax? I am fairly new to android dev and since most of the code for camerax is in kotlin, I can't find how to use this – Ak01 Jul 31 '20 at 07:48
  • 1
    Please take a look at ImageCapture#takePicture(Executor, OnImageCapturedCallback). In OnImageCapturedCallback, you can get a ImageProxy which can be converted to a Bitmap. see: https://stackoverflow.com/questions/41775968/how-to-convert-android-media-image-to-bitmap-object Then you can apply the sample above from there. – Xi 张熹 Jul 31 '20 at 13:49
  • Hi @Ak01, were you able to do this without using OpenCV or other image processing library? How cumbersome was that? Yes, the APK increase size problem is real and I am also trying to explore how this can be done with tools available by default in Android sDK. – amar Jul 14 '22 at 03:10