0

I'm developing a mobile app that must implement Twilio Programmable Video. I have developed a backend with Twilio SDK 5.24.2 in Laravel PHP Framework and that works well.

In the mobile app, I have installed the twilio-video 2.0.0-beta4 package through npm and developed a page based in the example in "video-quickstart-js-1.x". The problem is when Twilio tries to connect to the room, the error in the console is as follows (at the end of the log):

INFO in [connect #1]: Connecting to a Room
DEBUG in [connect #1]: Options: Object
INFO in [NTSIceServerSource #1]: Created a new NTSIceServerSource
DEBUG in [NTSIceServerSource #1]: ECS server: https://ecs.us1.twilio.com
INFO in [connect #1]: LocalTracks were not provided, so they will be acquired automatically before connecting to the Room. LocalTracks will be released if connecting to the Room fails or if the Room is disconnected
WARN in [createLocalTracks #1]: Call to getUserMedia failed: DOMException: Could not start audio source
INFO in [connect #1]: Error while connecting to a Room: DOMException: Could not start audio source

My code in Ionic is as follows:

import { Component, ViewChild } from '@angular/core';
import { IonicPage, NavController, NavParams, LoadingController } from 'ionic-angular';
import { ApiServiceProvider } from '../../providers/api-service';
import TwilioVideo from 'twilio-video';

export class VideoPage
{
  public roomSid: string;
  public activeRoom: any;
  public twilioAccessToken: string;

  @ViewChild('remoteMedia') remoteMedia: any;
  @ViewChild('localMedia') localMedia: any;
  @ViewChild('buttonLeave') buttonLeave: any;

  constructor(
    public navCtrl: NavController, 
    public navParams: NavParams,
    private apiServiceProvider: ApiServiceProvider
  ) {
    this.roomSid = navParams.get('room_sid');
    this.joinToRoom(this.roomSid);
  }

  joinToRoom(roomSid: string) {
    let videoCallData = {
      room_sid: roomSid
    };

    this.apiServiceProvider.joinVideoRoom(videoCallData)
    .then(
      (result) => {
        this.twilioAccessToken = result.access_token;
        this.generateVideoCall();
      },
      (error) => {
        console.log(error);
      }
    )
    .catch((error) => {
      console.log(error);
    });
  }

  generateVideoCall() {
    let connectOptions = {
      name: this.roomSid,
      video: true,
      audio:true
    };

    connectOptions['logLevel'] = 'debug';

    TwilioVideo.connect(this.twilioAccessToken, connectOptions)
    .then(this.roomJoined, function(error) {
      console.log('Could not connect to Twilio: ' + error.message);
    });
  }

  roomJoined(room: any) {
    this.activeRoom = room;

    // Attach LocalParticipant's Tracks, if not already attached.
    var previewContainer = this.localMedia;
    if (!previewContainer.querySelector('video')) {
      this.attachParticipantTracks(this.activeRoom.localParticipant, previewContainer);
    }

    // Attach the Tracks of the Room's Participants.
    this.activeRoom.participants.forEach(function(participant) {
      console.log("Already in Room: '" + participant.identity + "'");
      let previewContainer = this.remoteMedia;
      this.attachParticipantTracks(participant, previewContainer);
    });

    // When a Participant joins the Room, log the event.
    this.activeRoom.on('participantConnected', function(participant) {
      console.log("Joining: '" + participant.identity + "'");
    });

    // When a Participant adds a Track, attach it to the DOM.
    this.activeRoom.on('trackAdded', function(track, participant) {
      console.log(participant.identity + " added track: " + track.kind);
      var previewContainer = this.remoteMedia;
      this.attachTracks([track], previewContainer);
    });

    // When a Participant removes a Track, detach it from the DOM.
    this.activeRoom.on('trackRemoved', function(track, participant) {
      console.log(participant.identity + " removed track: " + track.kind);
      this.detachTracks([track]);
    });

    // When a Participant leaves the Room, detach its Tracks.
    this.activeRoom.on('participantDisconnected', function(participant) {
      console.log("Participant '" + participant.identity + "' left the room");
      this.detachParticipantTracks(participant);
    });

    // Once the LocalParticipant leaves the room, detach the Tracks of all Participants, including that of the LocalParticipant.
    this.activeRoom.on('disconnected', function() {
      console.log('Left room');

      if (this.previewTracks) {
        this.previewTracks.forEach(function(track) {
          track.stop();
        });
        this.previewTracks = null;
      }

      this.detachParticipantTracks(this.activeRoom.localParticipant);
      this.activeRoom.participants.forEach(this.detachParticipantTracks);
    });
  }

  // Attach the Participant's Tracks to the DOM.
  attachParticipantTracks(participant, container) {
    var tracks = Array.from(participant.tracks.values());
    this.attachTracks(tracks, container);
  }

  // Attach the Tracks to the DOM.
  attachTracks(tracks, container) {
    tracks.forEach(function(track) {
      container.appendChild(track.attach());
    });
  }

  // Detach the Tracks from the DOM.
  detachTracks(tracks) {
    tracks.forEach(function(track) {
      track.detach().forEach(function(detachedElement) {
        detachedElement.remove();
      });
    });
  }

  // Detach the Participant's Tracks from the DOM.
  detachParticipantTracks(participant) {
    var tracks = Array.from(participant.tracks.values());
    this.detachTracks(tracks);
  }

  // Leave Room.
  leaveRoomIfJoined() {
    if (this.activeRoom) {
      this.activeRoom.disconnect();
    }
  }
}

The HTML has the next structure:

<ion-content padding>
  <div #remoteMedia></div>

  <div #localMedia></div>

  <div id="buttonContainer">
    <button ion-button icon-only round #buttonLeave (click)="leaveCall($event)">
      <ion-icon name="close"></ion-icon>
    </button>
  </div>
</ion-content>

I have installed the plugin cordova-plugin-media-capture and assigned this permissions in config.xml

<config-file parent="/*" target="AndroidManifest.xml">
    <uses-permission android:name="android.permission.RECORD_AUDIO" />
    <uses-permission android:name="android.permission.MODIFY_AUDIO_SETTINGS" />
</config-file>

What I am missing?

Thanks in advance.

DCC
  • 36
  • 7
  • Twilio Video uses `getUserMedia` in order to capture the video and audio. The `cordova-plugin-media-captuire` plugin appears to provide a different interface. Could that be why this is failing? – philnash Dec 05 '18 at 05:31
  • Hi @philnash, thanks for your comment. I removed the plugin cordova-plugin-media-capture, and reinstall all the project (removing folders node_modules, platforms & plugins); but the same error continues. Another information is that the video and audio HTML tags are not created by Twilio. Any suggestions? – DCC Dec 06 '18 at 00:08
  • @philnash Twilio customer service told me that Ionic and Twilio are not compatible, is that true? – DCC Dec 06 '18 at 18:24
  • I don't see why they wouldn't be compatible. One thing you seem to be missing are permissions for the Camera. You might find this answer about getting `getUserMedia` to work in Ionic of some help: https://stackoverflow.com/questions/52557767/does-getusermedia-works-in-ionic-webview-in-android-app/53080000#53080000 – philnash Dec 09 '18 at 07:16
  • Thanks @philnash for your hekp. I decided to user OpenTok instead of Twilio. Twilio's customer support service told me this: _Unfortunately Twilio JS SDK does not work with Ionic because it relies on browser specific WebRTC APIs. The JS SDK only works with major browser (except Edget). There has been some consideration given to supporting a hybrid development platform in the future, but it is likely going to be React Native instead of Ionic._ – DCC Dec 10 '18 at 12:48
  • Oh man, the underlying browser is Chrome where this all works. You’re welcome to choose a different platform but I’m going to follow up to make sure we’re all on the same side. – philnash Dec 10 '18 at 14:22

0 Answers0