RawImage flicker when changing from Texture2D to VideoPlayer texture

Question

Using Unity 2017.3

I have a RawImage into which I display a sequence of images loaded as Texture2D's. Works perfectly and seamlessly.

I then show a video into the same RawImage, using the sample VideoPlayer code, and assigning

rawImage.texture = videoPlayer.texture;

The video plays perfectly well, but as part of switching from the still images to the video, there is a noticeable flicker in the RawImage, as if there's a frame or two of black displayed. The first frame of the video matches the last static image I displayed, so I had expected the transition to be pretty seamless.

Note that the video has been "Prepared" prior to all this - my code yields until videoPlayer.isPrepared returns true, and only then tell the video to play and set the texture.

I thought maybe there was an issue with the texture not being quite ready, but I tried yielding once or twice after calling Play and before setting the texture, but that didn't have any effect on the flicker.

I saw this item: https://answers.unity.com/questions/1294011/raw-image-flickering-when-texture-changed.html which suggests that this is something to do with material instances being set up. I don't fully understand the solution presented in that answer, nor do I understand how I could adapt it to my own case, but maybe it means something to those more skilled in Unity than I.

Any suggestions on how to get rid of that flickery frame?

EDIT: Here's the code

public class VideoAnimation : MonoBehaviour, IAnimation {
    private VideoPlayer _videoPlayer;
    private UnityAction _completeAction;
    private bool _prepareStarted;

    public void Configure(VideoClip clip, bool looping)
    {
        _videoPlayer = gameObject.AddComponent<VideoPlayer>();
        _videoPlayer.playOnAwake = false;
        _videoPlayer.isLooping = looping;
        _videoPlayer.source = VideoSource.VideoClip;
        _videoPlayer.clip = clip;
        _videoPlayer.skipOnDrop = true;
    }

    public void Prepare()
    {
        _prepareStarted = true;
        _videoPlayer.Prepare();
    }

    public void Begin(RawImage destImage, UnityAction completeAction)
    {
        _completeAction = completeAction;
        _videoPlayer.loopPointReached += OnLoopPointReached;
        StartCoroutine(GetVideoPlaying(destImage));
    }

    private IEnumerator GetVideoPlaying(RawImage destImage)
    {
        if (!_prepareStarted)
        {
            _videoPlayer.Prepare();
        }

        while(!_videoPlayer.isPrepared)
        {
            yield return null;
        }

        _videoPlayer.Play();
        destImage.texture = _videoPlayer.texture;
    }

    public void OnLoopPointReached(VideoPlayer source)
    {
        if (_completeAction != null)
        {
            _completeAction();
        }
    }

    public void End()
    {
        _videoPlayer.Stop();
        _videoPlayer.loopPointReached -= OnLoopPointReached;
    }

    public class Factory : Factory<VideoAnimation>
    {
    }
}

In the specific case I'm dealing with, Configure and Prepare are called ahead of time, while the RawImage is showing the last static image before the video. Then when it's time to show the video, Begin is called. Thus, _prepareStarted is already true when Begin is called. Inserting log messages shows that isPrepared is returning true by the time I get around to calling Begin, so I don't loop there either.

I've tried altering the order of the two line

    _videoPlayer.Play();
    destImage.texture = _videoPlayer.texture;

but it doesn't seem to change anything. I also thought that maybe the VideoPlayer was somehow outputting a black frame ahead of the normal video, but inserting a yield or three after Play and before the texture set made no difference.

None of the samples I've seen have a Texture in the RawImage before the VideoPlayer's texture is inserted. So in those, the RawImage is starting out black, which means that an extra black frame isn't going to be noticeable.

EDIT #2:

Well, I came up with a solution and, I think, somewhat of an explanation.

First, VideoPlayer.frame is documented as "The frame index currently being displayed by the VideoPlayer." This is not strictly true. Or, maybe it is somewhere in the VideoPlayer's pipeline, but it's not the frame that's observable by code using the VideoPlayer's texture.

When you Prepare the VideoPlayer, at least in the mode I'm using it, the VideoPlayer creates an internal RenderTexture. You would think that, once the player has been prepared, that texture would contain the first frame of the video. It doesn't. There is a very noticeable delay before there's anything there. Thus, when my code set the RawImage texture to the player's texture, it was arranging for a texture that was, at least at that moment, empty to be displayed. This perfectly explains the black flicker, since that's the color of the background Canvas.

So my first attempt at a solution was to insert the loop here:

    _videoPlayer.Play();
    while(_videoPlayer.frame < 1)
    {
        yield return null;
    }
    destImage.texture = _videoPlayer.texture;

between Play and the texture set.

I figured that, despite the documentation, maybe frame was the frame about to be displayed. If so, this should result in the first (0th) frame already being in the buffer, and would get rid of the flicker. Nope. Still flickered. But when I changed to

    _videoPlayer.Play();
    while(_videoPlayer.frame < 2)
    {
        yield return null;
    }
    destImage.texture = _videoPlayer.texture;

then the transition was seamless. So my initial attempt where I inserted yields between the two was the right approach - I just didn't insert quite enough. One short, as a matter of fact. I inserted a counter in the loop, and it showed that I yielded 4 times in the above loop, which is what I would expect, since the video is 30fps, and I'm running at 60fps on my computer. (Sync lock is on.)

A final experiment showed that:

    _videoPlayer.Play();
    while(_videoPlayer.frame < 1)
    {
        yield return null;
    }
    yield return null;
    destImage.texture = _videoPlayer.texture;

also did not result in a flicker. (Or, at least, not one that I could see.) So once the VideoPlayer was reporting that it was displaying the second frame (the numbers are 0-based according to the docs), it took one additional game frame before the transition was seamless. (Unless there was a 60-th of a second flicker that my eyes can't see.) That game frame might have something to do with Unity's graphic pipeline or VideoPlayer pipeline - I don't know.

So, the bottom line is that there is a noticeable delay from the time you call Play until there is actually anything in the VideoPlayer's texture that will make it to the screen, and unless you wait for that, you'll be displaying "nothing" (which, in my case, resulted in black background flickering through.)

It occurs to me that since the VideoPlayer is producing a RenderTexture, it might also be possible to blit the previous static texture to the VideoPlayer's texture (so that there would be something there right away) and then do the switch immediately. Another experiment to run...

Answer 1

Hmm, let's try use shaders, maybe it helps you.

First we must create custom shader and it must work like a standard UI shader. You can download all build-in shaders in this link.

Take UI-Default.shader and modificate it. I modificate it for you!

Just create shader in unity and paste this code:

Shader "Custom/CustomShaderForUI"
{
Properties
{
    //[PerRendererData] _MainTex ("Sprite Texture", 2D) = "white" {}
    _CustomTexture ("Texture", 2D) = "white" {} // <--------------- new property
    _Color ("Tint", Color) = (1,1,1,1)

    _StencilComp ("Stencil Comparison", Float) = 8
    _Stencil ("Stencil ID", Float) = 0
    _StencilOp ("Stencil Operation", Float) = 0
    _StencilWriteMask ("Stencil Write Mask", Float) = 255
    _StencilReadMask ("Stencil Read Mask", Float) = 255

    _ColorMask ("Color Mask", Float) = 15

    [Toggle(UNITY_UI_ALPHACLIP)] _UseUIAlphaClip ("Use Alpha Clip", Float) = 0
}

SubShader
{
    Tags
    {
        "Queue"="Transparent"
        "IgnoreProjector"="True"
        "RenderType"="Transparent"
        "PreviewType"="Plane"
        "CanUseSpriteAtlas"="True"
    }

    Stencil
    {
        Ref [_Stencil]
        Comp [_StencilComp]
        Pass [_StencilOp]
        ReadMask [_StencilReadMask]
        WriteMask [_StencilWriteMask]
    }

    Cull Off
    Lighting Off
    ZWrite Off
    ZTest [unity_GUIZTestMode]
    Blend SrcAlpha OneMinusSrcAlpha
    ColorMask [_ColorMask]

    Pass
    {
        Name "Default"
    CGPROGRAM
        #pragma vertex vert
        #pragma fragment frag
        #pragma target 2.0

        #include "UnityCG.cginc"
        #include "UnityUI.cginc"

        #pragma multi_compile __ UNITY_UI_CLIP_RECT
        #pragma multi_compile __ UNITY_UI_ALPHACLIP

        struct appdata_t
        {
            float4 vertex   : POSITION;
            float4 color    : COLOR;
            float2 texcoord : TEXCOORD0;
            UNITY_VERTEX_INPUT_INSTANCE_ID
        };

        struct v2f
        {
            float4 vertex   : SV_POSITION;
            fixed4 color    : COLOR;
            float2 texcoord  : TEXCOORD0;
            float4 worldPosition : TEXCOORD1;
            UNITY_VERTEX_OUTPUT_STEREO
        };

        fixed4 _Color;
        fixed4 _TextureSampleAdd;
        float4 _ClipRect;

        v2f vert(appdata_t v)
        {
            v2f OUT;
            UNITY_SETUP_INSTANCE_ID(v);
            UNITY_INITIALIZE_VERTEX_OUTPUT_STEREO(OUT);
            OUT.worldPosition = v.vertex;
            OUT.vertex = UnityObjectToClipPos(OUT.worldPosition);

            OUT.texcoord = v.texcoord;

            OUT.color = v.color * _Color;
            return OUT;
        }

        //sampler2D _MainTex;
        sampler2D _CustomTexture; // <---------------------- new property

        fixed4 frag(v2f IN) : SV_Target
        {
            //half4 color = (tex2D(_MainTex, IN.texcoord) + _TextureSampleAdd) * IN.color;
            half4 color = (tex2D(_CustomTexture, IN.texcoord) + _TextureSampleAdd) * IN.color; // <- using new property

            #ifdef UNITY_UI_CLIP_RECT
            color.a *= UnityGet2DClipping(IN.worldPosition.xy, _ClipRect);
            #endif

            #ifdef UNITY_UI_ALPHACLIP
            clip (color.a - 0.001);
            #endif

            return color;
        }
    ENDCG
    }
}
}

Next, create a material and set your RenderTexture to texture field in shader (not in RawImage component).

Hope this helps!

Answer 2

Try hiding the RawImage gameObject while the video is loading. This should fix any flickering caused by the VideoPlayer not being fully loaded.

public VideoPlayer _videoPlayer;
public RawImage _videoImage;

private void PlayClip(VideoClip videoClip)
{
    StartCoroutine(PlayClipCoroutine(videoClip));
}

private IEnumerator PlayClipCoroutine(VideoClip clip) 
{
  _videoImage.gameObject.SetActive(false);
  _videoPlayer.clip = clip;
  _videoPlayer.Prepare();

  while (!_videoPlayer.isPrepared) 
  {
      yield return null;
  }

  _videoPlayer.Play();
  _videoImage.texture = _videoPlayer.texture;
  _videoImage.gameObject.SetActive(true);
}