0

I'm developing an app in android, and I would like to stream the camera preview and my 3D model at the same time....now I'm streaming just the camera preview using a PreviewCallback to get the frames...is it possible to integrate the 3D model into the preview?

Wooble
  • 87,717
  • 12
  • 108
  • 131
  • I believe this is a legitimate technical question and I have no objections regarding its wording. It may look opinion-based to people far from the specific area of mobile AR. Please unhold. – Alex Cohn Oct 06 '14 at 03:47

1 Answers1

0

Usually, in AR applications we cannot steam the camera preview directly to the screen because we want to keep these frames in sync with the 3D model. That's why we render the data received from onPreviewFrame() callback in OpenGL as a texture on a simple rectangular mesh, with the 3D model before it.

Alex Cohn
  • 56,089
  • 9
  • 113
  • 307
  • So I need to merge everything in OpenGL and then can I stream all the scenario? –  Oct 06 '14 at 00:57
  • It's not exactly "merge", the video stream is not altered or projected, you draw it on "background" as is. You will ask, why not simply show SurfaceTexture instead? The answer is two-fold: you still need the preview callback for your AR calculations, because copy pixels from texture (GPU) to byte array (CPU) may be very slow; and second, even short delay between the 2D and 3D scenes on the screen looks very unnatural, while such delay between the screen and the real objects in front of you is acceptable. – Alex Cohn Oct 06 '14 at 04:21
  • Thank you Alex....do you know where I can find more information about it? because I was looking for an answer or examples in many pages but the information is not relevant or useless. –  Oct 06 '14 at 07:04
  • What kind of information are you looking for? No, I don't know an ultimate source for AR development on Android. You can start with this short tutorial, though: http://code.tutsplus.com/tutorials/augmented-reality-getting-started-on-android--mobile-4457. There are quite a few frameworks more or less available so that you don't need to reinvent all the wheels, see http://stackoverflow.com/questions/1939318/augmented-reality-framework – Alex Cohn Oct 06 '14 at 11:47