2

right now I have a client and server. The server is making screenshots and sending them to my client in byte.


Here I'm getting the bytes and putting them into my texture:

private IEnumerator RenderImage(byte[] values)
{
    var texture = new Texture2D(128, 128);
    texture.LoadImage(values); // <-- is running on main thread -> lag
    texture.Apply();
    RawImage.texture = texture;

    yield return null;
}

This works but its causes a short freeze on the hololens, for maybe 1/2 second.

1. I've read a lot of times that LoadImage will run on main thread and you cant move it somewhere else.
2. Using Texture2D.LoadRawTextureData should work faster and fix my problem. But using it, it throws the exception that Im not providing enough data. Writing something like
var texture = new Texture2D(128, 128, TextureFormat.RGBA32, false); wont fixe the problem.

Does sb know how encode the bytes in a more efficient way or a way that the main thread doesn't get stuck having to decode a image?

--EDIT-- I forgot to say that I'm using this Main-Thread-Dispatcher, but its not helping:

private void OnMessageReceived(byte[] values)
{
    UnityMainThreadDispatcher.Instance().Enqueue(RenderImage(values));
}

private IEnumerator RenderImage(byte[] values)
{
    var texture = new Texture2D(128, 128, TextureFormat.RGBA32, false);
    texture.LoadImage(values); // <-- is running on main thread -> lag
    texture.Apply();
    RawImage.texture = texture;

    yield return null;
}
Perazim
  • 1,501
  • 3
  • 19
  • 42
  • 2
    Possible duplicate of [Non-blocking loading and copying of large Texture2D's in C# for Unity](https://stackoverflow.com/questions/40450867/non-blocking-loading-and-copying-of-large-texture2ds-in-c-sharp-for-unity) – derHugo Feb 18 '19 at 14:26
  • 1
    or maybe better link using the Job-System: https://gamedev.stackexchange.com/q/113096/99708 .. the problem is quite old and unfortunately Unity so far didn't provide any better async encoding/decoding of bytes to/from Textures .. only way arround seems to be a Native Plugin – derHugo Feb 18 '19 at 14:27
  • the problem is that they are just using url that are referenced to an image. But I'm connected to an server, that screenshots the screen and sends me the screenshot in bytes. So that means I have to rewrite some methodes to get a link to the screenshot... But right now Im gonna try the job system. Maybe this will work better then the dispatcher I already tried. – Perazim Feb 19 '19 at 08:35
  • 1
    afaik there is unfortunetaly no way for creating a texture from bytes in another thread than the mainthread. If you find a solution I'm sure a lot of poeple would be very interrested ;) – derHugo Feb 19 '19 at 08:46
  • 1
    I'd focus on trying to solve the LoadRawTextureData exception: https://forum.unity.com/threads/loadrawtexturedata-not-enough-data-provided.439760/ – ickydime Feb 28 '19 at 15:14
  • Right now I'm working on another solution. I'm gonna create a mp4-stream and access it with unitys web player. I tried other mp4-streams and it worked without any lags. @ickydime I saw this post and tried the same way, but still got an exception...if the mp4-stream wont work I will come back to this – Perazim Mar 01 '19 at 08:21

1 Answers1

1

Ok, I fixed the problem by using an Server and this Nativ Render Plugin. The Render Plugin uses Direct3D and the RenderAPI for setting the texture.

Perazim
  • 1,501
  • 3
  • 19
  • 42
  • Could you please elaborate on how you passed the byte[] values to the Native Render plugin to create the texture? – charis Oct 02 '19 at 11:14