VLCJ in JMonkey3 ByteBuffer ArrayList issue

Question

I want to be able to play a video inside the JME3 environment, so that I have video texture-like sprites/object in my application. I found a example on google which demonstrates that, but it looks like one of the function calls changed in jme, and now accepts a different kind of data.

Example url: http://vlcj.googlecode.com/svn/wiki/JMonkeyEngineExample.wiki

In the example the display method looks like this:

    public void display(Memory memory) {
        // BEWARE!
        //
        //  Synchronisation might be required...

        // Copy the native memory to the video image
        videoImage.setData(memory.getByteBuffer(0, WIDTH * HEIGHT * DEPTH));
        // Set the new image on the texture
        videoTexture.setImage(videoImage);
    }

But when I copied it into jme, i got a warning about a missing method, and that was:

public void display(DirectMediaPlayer arg0, Memory[] arg1, BufferFormat arg2)

So clearly, it is the same method, but it is different now. I tried creating a ByteBuffer array list, and feeding in the data from arg1, but every time I get a critical error, and the entire application just crashes, but I do hear the audio until it crashes, so vlcj obviously works, but the ByteBuffer seems to be a mess.

How do I transform the data from the Memory[] type into a ByteBuffer ArrayList?

Edit: I managed to stop the crashing at exit by adding the following method:

public void destroy(){
    System.exit(0);
}

It crashed because there were threads left running, and System.exit(0); terminates them all before exiting.

I managed to get something by generating a ByteBuffer and directly allocating 614400 bytes, and then by using two for loops to iterate through the memory and copy the elements into the byte buffer. The image is full of vertical lines, and it is in black and white.When I copy the data in a single loop I don't get anything. — DLabinac, Dec 04 '13 at 23:27

score 0 · Answer 1 · answered Dec 05 '13 at 14:45

Both of these linked examples correctly allocate the correct buffer size, get access to the native video buffer and render it (albeit not via JME).

[1] https://github.com/caprica/vlcj/blob/vlcj-2.4.1/src/test/java/uk/co/caprica/vlcj/test/component/DirectMediaPlayerComponentTest.java

[2] https://github.com/caprica/vlcj-javafx/blob/master/src/test/java/uk/co/caprica/vlcj/javafx/test/JavaFXDirectRenderingTest.java

So these examples are not specifically for JME, but the principle should be the same.

Those example examples do not crash on exit either.

You also probably only need to consider Memory[0], this is what the default vlcj RenderCallbackAdapter does:

public void display(DirectMediaPlayer mediaPlayer, Memory[] nativeBuffer, BufferFormat bufferFormat) {
    nativeBuffer[0].getByteBuffer(0L, nativeBuffer[0].size()).asIntBuffer().get(rgbBuffer(), 0, bufferFormat.getHeight() * bufferFormat.getWidth());
    onDisplay(mediaPlayer, rgbBuffer());
}

In this fragment of code, the rgbBuffer() method simply returns a reference to the previously allocated Java int[] buffer.

Ideally you would be able to eliminate the intermediate copy from native buffer to Java buffer to texture, and instead go direct from native buffer to texture, but I don't know JME.

JME only accepts a ByteBuffer, so I can't give him an int type array of data. — DLabinac, Dec 05 '13 at 19:14
I managed to convert that IntBuffer to a ByteBuffer using this example: http://stackoverflow.com/a/1086092/1626222 But when I ran the program, the video was spitted it two smaller identical videos, and the colors were messed up, and it still crashed while stopping the application. — DLabinac, Dec 10 '13 at 00:36

score 0 · Accepted Answer · answered Dec 10 '13 at 02:12

0

I found a simpler way to do this, without copying stuff. I just wrote this, without any other extra ByteBuffer stuff:

videoImage.setData(arg1[0].getByteBuffer(0, arg1[0].size()));

At first it didn't work, but then I noticed that the texture format was wrong, so I needed to change the TEXTURE_FORMAT from RGBA16 to RGBA8.

But the video shows up flipped upside down, and rotating the object will make the video flipped horizontally. The only fix I found so far is to manually rearrange the bytes in the byte array.

answered Dec 10 '13 at 02:12

DLabinac

47
8

The video, i.e. texture, being flipped 'incorrectly' is because your 3d geometry for your video surface is not set up correctly, i.e. your normals are likely wrong. This part is nothing to do with your video, it would likely appear like this if you used a regular static texture. – caprica Dec 10 '13 at 07:07
I fixed it! Thanks for the note, I changed the example to show only a flat plane, by creating a geometry Quad. Now I just set one of the dimensions negative, and the video shows up correctly. Geometry geom = new Geometry("A shape", new Quad(2,-2)); – DLabinac Dec 10 '13 at 15:24

VLCJ in JMonkey3 ByteBuffer ArrayList issue

2 Answers2