ARCore - playing YouTube video over augmented image surface

Question

I'm trying to play a video from YouTube over an augmented image target surface in an android app built with Google ARCore library. With a local video file I have no problem, I use the MediaPlayer object as below:

MediaPlayer mediaPlayer = MediaPlayer.create(context, R.raw.my_vid_local_file);
mediaPlayer.setSurface(texture.getSurface());
mediaPlayer.setLooping(true);
if (!mediaPlayer.isPlaying()) {
  mediaPlayer.start();

  // Wait to set the renderable until the first frame of the  video becomes available.
  // This prevents the renderable from briefly appearing as a black quad before the video
  // plays.
  texture.getSurfaceTexture().setOnFrameAvailableListener(
        (SurfaceTexture surfaceTexture) -> {
          videoPlayerTexture.getNow(null).getMaterial().setExternalTexture("videoTexture", texture);
          videoPlayerTexture.getNow(null).getMaterial().setFloat4("keyColor", CHROMA_KEY_COLOR);
          mediaPlayerNode.setRenderable(videoPlayerTexture.getNow(null));
          texture.getSurfaceTexture().setOnFrameAvailableListener(null);
        });
} else {
  mediaPlayerNode.setRenderable(videoPlayerTexture.getNow(null));
}

All works fine, but I would love get the video data source from YouTube so I can change it without download again my app from store.

I read this thread: Android Java Youtube API v3.0 get video RTSP link about taking an RTSP link within YouTube API v3 but when I put it as my MediaPlayer object datasource I get an Exception.

I modified the code as below:

MediaPlayer mediaPlayer = new MediaPlayer();
try {
  mediaPlayer.setDataSource(context, Uri.parse("https://r8---sn-nx5cvox-hpay.googlevideo.com/videoplayback?expire=...and so on"));
  mediaPlayer.setOnPreparedListener(new MediaPlayer.OnPreparedListener() {
    @Override
    public void onPrepared(MediaPlayer mp) {
      if (!mediaPlayer.isPlaying()) {
        mediaPlayer.start();

        // Wait to set the renderable until the first frame of the  video becomes available.
        // This prevents the renderable from briefly appearing as a black quad before the video
        // plays.
        texture.getSurfaceTexture().setOnFrameAvailableListener(
                (SurfaceTexture surfaceTexture) -> {
                  videoPlayerTexture.getNow(null).getMaterial().setExternalTexture("videoTexture", texture);
                  videoPlayerTexture.getNow(null).getMaterial().setFloat4("keyColor", CHROMA_KEY_COLOR);
                  mediaPlayerNode.setRenderable(videoPlayerTexture.getNow(null));
                  texture.getSurfaceTexture().setOnFrameAvailableListener(null);
                });
      } else {
        mediaPlayerNode.setRenderable(videoPlayerTexture.getNow(null));
      }
    }
  });
  mediaPlayer.prepare();
}catch(Exception e) {
  e.printStackTrace();
  Log.e(TAG, e.getMessage());
}

And the error is:

D/MediaHTTPConnectionEx: [response code] STATUS CODE:403
I/native: tracking.cc:3027 no history. confidence : 1.0
D/MediaHTTPConnectionEx: [ResponseMessage]:Forbidden
E/MediaHTTPConnectionEx: MediaHTTPConnectionEx
com.lge.media.MediaHTTPConnectionEx 411 seekTo exception toString: java.io.IOException
D/MediaHTTPConnectionEx: connect
D/MediaHTTPConnectionEx: disconnecting
D/MediaHTTPConnectionEx: disconnected
D/MediaHTTPConnectionEx: [seekToEx] offset:0/mCurrentOffset:-1
D/MediaHTTPConnectionEx: proxy null port 0
I/native: tracking.cc:3027 no history. confidence : 1.0
D/MediaHTTPConnectionEx: [response code] STATUS CODE:403
D/MediaHTTPConnectionEx: [ResponseMessage]:Forbidden
E/MediaHTTPConnectionEx: MediaHTTPConnectionEx com.lge.media.MediaHTTPConnectionEx 411 seekTo exception toString: java.io.IOException
D/MediaHTTPConnectionEx: connect
D/MediaHTTPConnectionEx: disconnecting
D/MediaHTTPConnectionEx: disconnected
...

There would be a way to use YouTube API KEY or am I completely wrong approach?

Thank you in advice

Carlo Tassi

hi is this still active? I tried your code, just want to know what is the mediaPlayerNode? — Joe, Jul 10 '23 at 06:36

score 1 · Accepted Answer · answered Jun 11 '19 at 09:15

1

YouTube is designed to be played either using its own webpage, on another webpage via embedded HTML5 or using the YouTube mobile API's.

The key common factor is that they all retain the YouTube branding and look and feel.

There are some examples of ways to get the raw video URL so you can play it directly (e.g. https://stackoverflow.com/a/9740907/334402) but if you use the raw video URL and MediaPlayer, you lose the Youtube look and feel and generally this is not something they support.

If you just want to ability to play a video that you have some other source for or that you can host yourself, then your approach looks good.

answered Jun 11 '19 at 09:15

Mick

24,231
1
54
120

Probably the best approach is hosting myself source video files... I don't know neither if playing from YouTube raw source video URL is within their terms of usage of service :) – Carlo Tassi Jun 12 '19 at 10:21
1

@CarloTassi, do you have any example for a video URL with ARCore ? – Idris Stack Jul 01 '20 at 12:34
@CarloTassi, did you use the `AugmentedImageDatabase()` in android to add the video URLs plus the images, or you used something else. Can you please share any resource for that – Lutaaya Huzaifah Idris Jul 02 '20 at 22:52

ARCore - playing YouTube video over augmented image surface

1 Answers1