1

I'm developing an AR game. In a particular situation, I want to play a video, which user selected from phone gallery, in a part of scene in unity(for example on a Plane). I test unity Video Player, but that would lag so much when video size is more than 100 Mbyte and even textures doesn't display, just I can hear video sound.

What should I do now? Should I write a java native plugin which stream video in java and set textures in unity?

Thanks and Sorry for bad English.

    videoPlayer.playOnAwake = false;
    videoPlayer.source = VideoSource.Url;
    videoPlayer.url = url;
    videoPlayer.Prepare();

    //Wait until video is prepared
    while (!videoPlayer.isPrepared)
    {
        Debug.Log("Preparing Video");
        yield return null;
    }

    //Assign the Texture from Video to RawImage to be displayed
    image.texture = videoPlayer.texture;

    //Play Video
    videoPlayer.Play();

I also assigned AudioSource in editor. All things works fine just if video size is lower than 10 Mbyte.

Programmer
  • 121,791
  • 22
  • 236
  • 328
MehDi
  • 504
  • 6
  • 20
  • @Programmer I didn't expose any code because I'm using default codes for making Unity VideoPlayer component run (and use WWW for get data). here is a link if you want to know: https://stackoverflow.com/a/41154599/5112333 I'm playing video from android local storage(not unity asset folder). – MehDi May 25 '18 at 09:14
  • @Programmer oh sorry I said wrong, I didn't use WWW. I get video path in native java, and use like "file:///..." as video url. I posted my unity code – MehDi May 25 '18 at 10:07
  • Ah I know it. Can I write a short native plugin for handle this? Streaming and make texture in java and then put it on a Plane in Unity, is it possible?and is it fast enough? Thanks for your attention, and I'm looking for your answer. @Programmer – MehDi May 25 '18 at 13:41
  • Do not transcode the video file in Unity. – Naresh Bisht Nov 10 '22 at 07:33

1 Answers1

5

Unity's VideoPlayer seems to be lagging on some Android devices when the video file is big. One possible solution on the Android device is to read the video in chunks then host it on the Android device with HttpListener.

Connect to that host you created on the device with the VideoPlayer API by using VideoSource.Url; and setting the VideoPlayer.url to that host url on the local device then play the video.

Host the video with HttpListener(Must change filePath to point to the actual path of the video:

//Server url
string url = "http://127.0.0.1:8080/";


//Types of media supported(Can be extended)
private static IDictionary<string, string> mimeDic =
       new Dictionary<string, string>(StringComparer.InvariantCultureIgnoreCase)
       {
            {".asf", "video/x-ms-asf"},
            {".asx", "video/x-ms-asf"},
            {".avi", "video/x-msvideo"},
            {".flv", "video/x-flv"},
            {".jpeg", "image/jpeg"},
            {".jpg", "image/jpeg"},
            {".mng", "video/x-mng"},
            {".mov", "video/quicktime"},
            {".mp3", "audio/mpeg"},
            {".mpeg", "video/mpeg"},
            {".mp4", "video/mp4" },
            {".mpg", "video/mpeg"},
            {".ra", "audio/x-realaudio"},
            {".swf", "application/x-shockwave-flash"},
            {".wbmp", "image/vnd.wap.wbmp"},
            {".wmv", "video/x-ms-wmv"},
       };

List<HttpListenerResponse> httpResponses = new List<HttpListenerResponse>();
Thread listeningThread;

void Awake()
{
    Application.runInBackground = true;

    //Your Video Path
    string filePath = Path.Combine(Application.persistentDataPath, "Cater2U.mp4");

    Debug.Log(filePath.Replace("/", "\\"));

    //Create Server
    StartHttpServer(filePath);
}

void StartHttpServer(string dataPath)
{
    listeningThread = new Thread(new ParameterizedThreadStart(ListenToClient));
    listeningThread.IsBackground = true;
    listeningThread.Start(dataPath);
}

void StopHttpServer()
{
    //Stop thread
    if (listeningThread != null && listeningThread.IsAlive)
    {
        listeningThread.Abort();
        Debug.LogWarning("Listening Thread Stopped!");
    }
}

void DisconnectClients()
{
    //Disconnect from each connected client
    for (int i = 0; i < httpResponses.Count; i++)
    {
        if (httpResponses[i] != null)
        {
            httpResponses[i].StatusDescription = "Server done";
            httpResponses[i].OutputStream.Close();
            Debug.LogWarning("Disconnected Client!");
        }
    }
}

void ListenToClient(object path)
{
    //Get the param
    string dataPath = (string)path;

    HttpListener listener = new HttpListener();
    listener.Prefixes.Add(url);
    listener.Start();

    Debug.Log("Listening to Client");

    while (true)
    {
        HttpListenerContext context = listener.GetContext();
        Debug.LogWarning("New Client Connected: " + context.Request.RemoteEndPoint.ToString());

        //Construct param that will be sent to the Thread
        ServerParamData serverData = new ServerParamData(context, dataPath);
        ThreadPool.QueueUserWorkItem(new WaitCallback(RunInNewThread), serverData);
    }
}

private void RunInNewThread(object ctx)
{
    //Get the param
    ServerParamData serverData = (ServerParamData)ctx;

    //Open the file and start sending it to the client
    WriteFile(serverData.context, serverData.path);
}

void WriteFile(HttpListenerContext ctx, string path)
{
    HttpListenerResponse response = ctx.Response;
    httpResponses.Add(response);

    using (FileStream fs = File.OpenRead(path))
    {
        string filename = Path.GetFileName(path);
        string mime;

        //Set the type of media to play
        if (!mimeDic.TryGetValue(Path.GetExtension(filename), out mime))
            mime = "application/octet-stream";


        ctx.Response.ContentType = mime;
        response.ContentLength64 = fs.Length;

        //Stream the File
        response.SendChunked = true;

        //Enable Media Seek(Rewind/Fastforward)
        response.StatusCode = 206;
        response.AddHeader("Content-Range", "bytes 0-" + (fs.Length - 1) + "/" + fs.Length);
        //According to Content Range
        //https://greenbytes.de/tech/webdav/rfc7233.html#header.content-range


        //Send data to the connected client
        byte[] buffer = new byte[64 * 1024];
        int read;
        using (BinaryWriter bw = new BinaryWriter(response.OutputStream))
        {
            while ((read = fs.Read(buffer, 0, buffer.Length)) > 0)
            {
                bw.Write(buffer, 0, read);
                bw.Flush(); //seems to have no effect
            }
            bw.Close();
        }

        response.StatusCode = (int)HttpStatusCode.OK;
        response.StatusDescription = "OK";
        response.OutputStream.Close();
    }
}

void OnDisable()
{
    //Clean Up
    StopHttpServer();
    DisconnectClients();
}

//Holds multiple params sent to a function in another Thread
public class ServerParamData
{
    public HttpListenerContext context;
    public string path;

    public ServerParamData(HttpListenerContext context, string path)
    {
        this.context = context;
        this.path = path;
    }
}

Play the hosted video with VideoPlayer API (This is a minor modification of the code from this post) :

//Raw Image to Show Video Images [Assign from the Editor]
public RawImage image;

private VideoPlayer videoPlayer;
private VideoSource videoSource;

//Audio
private AudioSource audioSource;


//Server url
string url = "http://127.0.0.1:8080/";

// Use this for initialization
void Start()
{
    Application.runInBackground = true;
    StartCoroutine(playVideo());
}

IEnumerator playVideo()
{
    //Add VideoPlayer to the GameObject
    videoPlayer = gameObject.AddComponent<VideoPlayer>();

    //Add AudioSource
    audioSource = gameObject.AddComponent<AudioSource>();

    //Disable Play on Awake for both Video and Audio
    videoPlayer.playOnAwake = false;
    audioSource.playOnAwake = false;

    //We want to play from url
    videoPlayer.source = VideoSource.Url;
    videoPlayer.url = url;

    //Set Audio Output to AudioSource
    videoPlayer.audioOutputMode = VideoAudioOutputMode.AudioSource;

    //Assign the Audio from Video to AudioSource to be played
    videoPlayer.EnableAudioTrack(0, true);
    videoPlayer.SetTargetAudioSource(0, audioSource);

    //Prepare Audio to prevent Buffering
    videoPlayer.Prepare();

    //Wait until video is prepared
    while (!videoPlayer.isPrepared)
    {
        Debug.Log("Preparing Video");
        yield return null;
    }

    Debug.Log("Done Preparing Video");

    //Assign the Texture from Video to RawImage to be displayed
    image.texture = videoPlayer.texture;

    //Play Video
    videoPlayer.Play();

    //Play Sound
    audioSource.Play();

    Debug.Log("Playing Video");
    while (videoPlayer.isPlaying)
    {
        Debug.LogWarning("Video Time: " + Mathf.FloorToInt((float)videoPlayer.time));
        yield return null;
    }
    Debug.Log("Done Playing Video");
}

This works for me but may work on your device. If that's the case then you have to abandon the VideoPlayer API for now and use the MediaPlayer with OpenGL ES to make your own video player for Android. This and this posts should get you started on the plugin.

Programmer
  • 121,791
  • 22
  • 236
  • 328
  • Thanks in Advance! I found old Vuforia videoplayback for unity sample and use their component for my needed, I don't know why do they remove this in their new samples(do you know?) but It work nice in my phone. But your answer is so cool and I'm trying to learn it haha, let me ask some question later if I didn't get it:) So much thankful! – MehDi May 27 '18 at 13:39
  • 1
    They removed it because Unity's VideoPlayer API is now released so need need to use a plugin. Many people won't be happy to download and use `Vuforia` so I think you should try this answer to verify if this solves your problem. You don't have to use it, just try it to see if it works. – Programmer May 27 '18 at 15:13
  • I tested it, it doesn't meet our requirement. It doesn't stream file (doesn't send as chunks) and even I didn't see any texture or didn't hear any sound during playing, but it seems somethings was happening, I understand it via my network speed. during playing network speed show 11KB/s. Maybe it try to download whole of video @Programmer – MehDi Jun 01 '18 at 11:44