Can you write to a Unity texture, from an iOS C plugin, "on the spot"?

Question

Say you have a low-level Unity plugin for iOS,

So in c#

using System.Runtime.InteropServices;
using AOT;

public class Teste: MonoBehaviour {

    [DllImport("__Internal")] private static extern
       void set_texture_from_unity(System.IntPtr texture, int w, int h);

have a texture, .Apply() it, and then send the pointer to the native iOS plugin:

public void ClickPassTexture() {

    tex = new Texture2D(256, 256, TextureFormat.ARGB32, false);
    tex.filterMode = FilterMode.Point;

    tex.Apply(); // ACTUALLY UPLOAD TO GPU

    someMaterial.mainTexture = tex;

    set_texture_from_unity(
       tex.GetNativeTexturePtr(), tex.width, tex.height);
}

Now the C code.

Make a texture with a few colored lines, and write it to the Unity texture.

#include <OpenGLES/ES2/gl.h>
...
void set_texture_from_unity(void *g_TextureHandle,  int w, int h)) {
    
    // make a texture with a few gray rows
    unsigned char* data = malloc( g_TextureWidth * 4 * g_TextureHeight );
    for (int i = 0; i < 1000; ++i) { data[i] = 100; }

    // now, write that to the texture pointer given to us from Unity
    GLuint gltex = (GLuint)(size_t)(g_TextureHandle);
    glBindTexture(GL_TEXTURE_2D, gltex);
    glTexSubImage2D(GL_TEXTURE_2D, 0, 0, 0,
      g_TextureWidth, g_TextureHeight, GL_RGBA, GL_UNSIGNED_BYTE, data);

    free(data);

So, in the C code, we have seemingly successfully modified the texture tex.

However.

It does not appear on screen.

Image shows complete failure to achieve anything at all.

The fine Unity doco docs.unity3d.com/Manual/NativePluginInterface.html

gives an example where the C texture change gets called by a callback in the Unity rendering chain.

However.

Can we simply make Unity take the new texture "now"?

When we want to ?

I thought this might simply work:

    set_texture_from_unity( tex.GetNativeTexturePtr(),
          tex.width, tex.height);
    // the plugin has changed the texture...
    // perhaps an Apply will change it on screen?!
    tex.Apply();

But no! (It makes no difference if you wait a few frames before the .Apply.)

You'd think .Apply would send the texture up to the gpu again, but it seems to not work.

In short, what to do to the presented code, to make it show the new texture "there and then", after the C plugin runs??

When you know a Texture has been modified by a C plugin, this on iOS, how to get Unity to display the change???

I think `set_texture_from_unity( tex.GetNativeTexturePtr(), tex.width, tex.height); //... tex.Apply();` would work if you returned the texture inside the C plugin, instead of returning void. so `tex = set_texture_from_unity(...); tex.Apply();`as Unity doesn't currently know what it is applying. (though i believe `glTexSubImage2D` should already Apply it in your C part...) — Remy, Jan 15 '19 at 08:15
hmm, it's surely the same texture - you pass the pointer, you know ? g_TextureHandle — Fattie, Jan 15 '19 at 11:02
If you create a texture in Unity via code.. you can change it in your native plugin.. However, there doesn't seem to be a way to change an existing texture (one loaded by Unity from your assets). — Brandon, Jan 16 '19 at 04:01
@Brandon - question - have you ever been able to actually achieve this !!? (ie, as you say, create in Unity, then change in the plugin ...) We just can not get it to work! Have you ever been able to do that? thanks... — Fattie, Jan 16 '19 at 11:55
@Fattie; Yes I just did it. I did: `Texture2D tex = new Texture2D(128, 128, TextureFormat.RGBA32, true)`.. Then I pass it to my plugin like so: `set_texture_from_unity(tex. GetNativeTexturePtr(), tex.width, tex.height)` and when I display it on a RawImage it changed from White to Grey/Gray.. Please note that in your code, you are doing: `g_TextureWidth` instead of using `w` parameter.. That's the only thing I changed (same for height).. — Brandon, Jan 16 '19 at 13:56
@remy_rm ! BTW don't miss the utterly critical post by Master T. - see the link I edited in to my answer! It's been there for 6+ months, sadly I had not noticed it. There it is. — Fattie, Jan 16 '19 at 14:42
Master T. is beyond good eh? And he is unbelievably helpful - the bloke fixed a small bug in that one, literally in a couple hours. Amazing ! — Fattie, Jan 16 '19 at 14:52

score 3 · Accepted Answer · edited Jun 20 '20 at 09:12

Solution in the "new" Unity

The (legendary) Keijiro Takahashi from Unity Japan, has posted probably the single most important thing ever posted from Unity:

https://github.com/keijiro/TextureUpdateExample

An actual example of the (new) frame-wise style texture-updating from plugin.

AND IT WORKS WITH IOS

Again, all that can be said is this pretty much the most valuable thing ever emitted from Unity! Phew!

Regarding the question as asked, I really have never found the solution.

score 1 · Answer 2 · answered Jan 15 '19 at 16:13

1

You need to call UpdateExternalTexture method.

Updates Unity texture to use different native texture object.

This function is mostly useful for native code plugins that create platform specific texture objects outside of Unity, and need to use these textures in Unity Scenes. For a texture created with CreateExternalTexture, this function switches to another underlying texture object if/when it changes.

answered Jan 15 '19 at 16:13

Hamid Yusifli

9,688
2
24
48

Ah - checking it out, but I think that's if you want to change to "another" texture which your plugin made. In the case at hand, the plugin is *changing* (the pixels of) an existing texture which Unity made! Do you know what I mean? – Fattie Jan 15 '19 at 17:10
Yes, while useful information unfortunately this is not correct. – Fattie Jan 22 '19 at 11:23

Can you write to a Unity texture, from an iOS C plugin, "on the spot"?

It does not appear on screen.

Can we simply make Unity take the new texture "now"?

2 Answers2

Solution in the "new" Unity

Linked