5

I am working on an Android NDK app that needs access to the pixels of an OpenGL ES 2.0 texture.

OpenGL extensions allow doing this as described e.g. here, here, and here.
All of these approaches require using the GraphicBuffer class which is not a part of the NDK (see e.g. here). Apparently, this requires accessing a private C++ Android API (from the AOSP-Android Open Source Project).

First, if I actually build and link with the AOSP repo, will my app run on the same Android devices as the "pure"-NDK version?
Can anyone provide (ref. to) detailed instruction for how to do this and the pros/cons of doing this?

Second, Android's "Graphics architecture" document states:

ANativeWindow

The public Surface class is implemented in the Java programming language. The equivalent in C/C++ is the ANativeWindow class, semi-exposed by the Android NDK. You can get the ANativeWindow from a Surface with the ANativeWindow_fromSurface() call. Just like its Java-language cousin, you can lock it, render in software, and unlock-and-post.

To create an EGL window surface from native code, you pass an instance of EGLNativeWindowType to eglCreateWindowSurface(). EGLNativeWindowType is just a synonym for ANativeWindow, so you can freely cast one to the other.

and according to this GraphicBuffer is a "simple wrapper" over ANativeWindowBuffer.

So the second question is: Is it possible to create an OpenGL texture compatible surface like GraphicBuffer by using only EGL and NDK functions (without requiring AOSP)? If so how?

Community
  • 1
  • 1
Adi Shavit
  • 16,743
  • 5
  • 67
  • 137

1 Answers1

2

Starting by your second question, I don't think there is. If you want to keep your code only on the native side, you have to use GraphicBuffers. It would be great to have functionality like this exposed (iOS has core video pixel buffers), but I doubt it will ever happen.

You don't really need to build AOSP to work with GraphicBuffers. Just get a few of the headers where the structures are defined and used them in your code. Then use dlopen() and dlsym() to get the pointers to the functions that you need, and use them. As your last link points out, check system/core/include/system/window.h and frameworks/native/include/ui/GraphicBuffer.h The mangled function names make the code a pain to write, but it works. There is a good usage example in Firefox sources.

user1906
  • 2,310
  • 2
  • 20
  • 37
  • I don't mind calling out to Java via the JNI if that works. Does Java have a `GraphicBuffer` equivalent that isn't private? – Adi Shavit Dec 12 '14 at 06:23
  • You can render to a Surface in Java and then pass that Surface to the native side and get the ANativeWindow, and read the pixels. There's an example here: https://vec.io/posts/how-to-render-image-buffer-in-android-ndk-native-code. – user1906 Dec 12 '14 at 06:40
  • Yes. I know this link. What I wonder is if I can create the surface in (or through) Java - preferably from the JNI itself, and once I have it as an `ANativeWindow` both render to it using OpenGL *and* access the pixels. – Adi Shavit Dec 12 '14 at 06:47
  • You can't create just a Surface, but you could create a SurfaceView, and then get the Surface from it. It'd be a bit convoluted. I've opted to go the GraphicBuffer route in the last few projects I've worked on. Renderscript has NDK support now, it may be another solution? – user1906 Dec 12 '14 at 22:26
  • My app is already very widely installed so I don't want to risk private APIs (nor do I look forward to a complicated build process). Do you think that the SurfaceView route is more convoluted than demangling function names etc.? Can you point me to some docs or examples of using SurfaceView in a similar way ( i.e. to JNI to OpenGL texture)? – Adi Shavit Dec 12 '14 at 22:32
  • As far as public APIs go, Graphic Buffer has worked for a few years already, and it works on Lollipop too. I don't think a project such as Firefox would use it if it was constantly changing. The problem with SurfaceView is that it's an actual view that has to be on the UI. I've used SurfaceView->JNI to render things to screen, but as in the link above, you can also read from that Surface. – user1906 Dec 14 '14 at 01:17
  • OK, thanks. Can you provide a direct link to the Firefox code that shows this? – Adi Shavit Dec 14 '14 at 08:44