5

Everyone,

I am working with SurfaceTexture in Android but I am not able to understand Its API: getTransformMatrix(float[] mtx), the API doc is as follows:

/**
 * Retrieve the 4x4 texture coordinate transform matrix associated with the texture image set by
 * the most recent call to updateTexImage.
 *
 * This transform matrix maps 2D homogeneous texture coordinates of the form (s, t, 0, 1) with s
 * and t in the inclusive range [0, 1] to the texture coordinate that should be used to sample
 * that location from the texture.  Sampling the texture outside of the range of this transform
 * is undefined.
 *
 * The matrix is stored in column-major order so that it may be passed directly to OpenGL ES via
 * the glLoadMatrixf or glUniformMatrix4fv functions.
 *
 * @param mtx the array into which the 4x4 matrix will be stored.  The array must have exactly
 *     16 elements.
 */

After reading it, I really don't know how to use it. I want to crop texture image in SurfaceTexture and provide the SurfaceTexture obj to a EGLSurface object.

For example, the original image is 320*720(w*h) and I expect the new image to be 320*240(w*h)

What should I do in order to achieve this function? Can the 4*4 matrix help me? What should I do with the 4*4 matrix?

JasonMArcher
  • 14,195
  • 22
  • 56
  • 52
dragonfly
  • 1,151
  • 14
  • 35
  • Matrices are used to apply transformations to points, such as translations. http://en.wikipedia.org/wiki/Matrix_%28mathematics%29#Linear_transformations – Machinarius Jun 02 '15 at 12:47

1 Answers1

6

It's a GLES-compatible transformation matrix that may be used to rotate or flip an image. Generally speaking you just pass it through to whatever is going to render the GLES texture that SurfaceTexture creates.

You can see examples of it used in Grafika; search for it in ContinuousCaptureActivity.java, CameraCaptureActivity.java, and TextureMovieEncoder.java.

The matrix can be used to perform various affine transformations, such as rotation and scaling, but cannot be used to clip an image.

The matrix is included in the API because there are times when the input source is in the "wrong" orientation. Rather than spending CPU or GPU cycles to rearrange the pixels, the system just sends a matrix along with each frame. If the frame contents are correct as-is, the matrix will be identity. If it's upside-down, a matrix with an appropriate correction will be provided.

fadden
  • 51,356
  • 5
  • 116
  • 166
  • Many thanks to you! your explanation is excellent. I am touched that you asked my question at 5:24 am. – dragonfly Jun 04 '15 at 07:01
  • Could you take a look at this: http://stackoverflow.com/questions/32217249/how-to-make-textureview-play-video-with-round-corners-and-bubble-effect – dragonfly Aug 26 '15 at 03:08
  • I have read about the way of drawing video frame to eglsurface from your Grafika, you get a matrix through SurfaceTexture.getTransformMatrix, and you pass it to the method drawFrame of FullFrameRect. But I want to draw the first frame at first(a jpg on the disk) to the EGLsurface and then draw video frame. I managed to create a texture id mapped a bitmap and I actually draw the bitmap to the eglsurface successfully but upside-down. I also use drawFrame method of FullFrameRect, I set the input matrix a identity matrix which is not right. But How to set the matrix, Is there an appropriate way? – dragonfly Sep 19 '15 at 05:46
  • 2
    You can modify the matrix to flip the image. For an example, see http://bigflake.com/mediacodec/ExtractMpegFramesTest.java.txt `drawFrame(SurfaceTexture st, boolean invert)` and note how it modifies entries 5 and 13 in the matrix. – fadden Sep 19 '15 at 20:05
  • Dear fadden, I use opengl to draw video frame to screen. I use the way of ContinuousCaptureActivity.java. My data source is MediaPlayer rather than Camera which makes no difference. Now I want to rotate each video frame with 270 degrees. How to modify the 4*4 matrix? I use Matrix.rotateM method but video is corrupted when playing. Could you help me to get the proper matrix? – dragonfly Nov 18 '15 at 05:33
  • I don't know why applying a rotation would corrupt the video output. You should file a new question, with a code snippet and a screen grab showing the problem. – fadden Nov 18 '15 at 06:11
  • I have created a new question, please have a look. http://stackoverflow.com/questions/33773770/use-rotatem-of-matrix-to-rotate-matrix-from-surfacetexture-but-corrupt-the-vid – dragonfly Nov 18 '15 at 07:08