5

So I know about setSurface, and have no problem using it as an overlay or whatever - its on a surfacecontrol. That said, I am stumped about getting pixel data

1) I've tried everything I can think of (the control, the root, etc) to use the drawing cache functions to get the bits for the camera surface. Yah, no. The cached bitmap is always zerod out.

2) I've used both SurfaceView and GLSurfaceView successfully as a setSurface taget. I cannot use any other class, such as TextureView.

3) I've investigated the C API and I see the camera exposes connectOnFrameAvailable, which will give me access to the pixels

My guess is that the internal tango logic is just using the surface in java to gain access to the underlying bit transfer channel - in the C API it requires a texture ID, which makes me suspect at the end of the day, the camera data is shipped in to the GPU pretty quickly, and I bet that CUDA lib operates on it - given the state of things, I can't see how to get the bits on the Java side without rooting the device - just cause I have a texture or simple surface view rendering raw bits on the screen doesn't mean I can get to them.

I don't want to peel the image data back out of the GPU. I'd need to switch my busy animation from a watch to a calendar for that.

Before I dive down into the C API, is there any way I can get the camera bits in Java ? I really want to be able to associate them with a specific pose, but right now I can't even figure out how to get them at all. I really want to know the location and color of a 3D point. Camera intrinsics, the point cloud, and the 2d image that generated the point cloud are all I need. But I can't do anything if I can't get the pixels, and the more questionable the relationship between an image and a (pose and a pointcloud) the sketchier any efforts will become.

If I do dive into C, will the connectOnFrameAvailable give me what I need ? How well synced is it with the point cloud generation ? Oh, and have I got this right ? Color camera is used for depth, fisheye is used for pose ?

Can I mix Java and C, i.e. create a Tango instance in Java and then just use C for the image issue ? Or am I going to have to re-realize everything in C and stop using the tango java jar ?

Mark Mullin
  • 1,340
  • 1
  • 9
  • 21

1 Answers1

0
  • will the connectOnFrameAvailable give me what I need?

Yes, it indeed returns the YUV byte buffer.

  • How well synced is it with the point cloud generation?

The Tango API itself doesn't provide synchronization between the color image and depth point cloud, however, it does provide the timestampe which allow you to sync at the application level.

  • Color camera is used for depth, fisheye is used for pose?

Yes, you are right.

  • Can I mix Java and C (i.e. create a Tango instance in Java and then just use C for the image issue) Starting two Tango instance is really not the way Tango supported, even though it works, it will be extremely hacky..

As the temp walk-around, you could probably try to use the drawing cache of the view?

xuguo
  • 1,816
  • 1
  • 11
  • 15
  • nope - i ran that puppy right into the ground - canna get the bits in that path unless I want to root the device. However, I'm now running with the C++ stuff. TangoService_connectOnFrameAvailable is firing regularly. Unfortunately, it seems that when it is firing, the image updates that happen as a result of connect texture cease. But at least I've got my hands on the bits! – Mark Mullin Jan 16 '15 at 02:54