0

I'm making a 2D game in Java with Swing and I got to a point where i needed to do Active Rendering to play the game's animations smoothly, so i set-up a test project where I just implemented the whole double buffering/page flipping/active rendering thing, and it's working, I get 60 fps and I can draw my swing components in my buffer through a SwingUtilities.InvokeAndWait() call so that I still perform draw calls within the EDT.

Now the problem is that I also have to play video cinematic in the game and I found JavaFX's MediaPlayer to be the simplest way to do so and it works great with a JFXPanel and passive rendering. But that's where I'm kinda stuck :

When I start the MediaPlayer I have to make the call through Platform.runLater() so that it works on the JavaFX's side (it's using another and unique thread to perform rendering). The problem is that by doing so I can't actually render this video through my game loop's rendering thread, where every swing components are actually being drawn.

So right now what happens is that whenever I play the video my framerate goes from 60 to 30fps, I tried to bake the same video in both 60 and 30 fps and see if it would change anything but it's independant of the video's framerate, always goes down to 30fps. And the other problem is that it's not synchronized at all with what is being drawn in the game loop, like for example the simple FPS string debug on the top left corner of the screen is flickering when the video is playing since the draw calls are not synchronized.

So my question is how could I possibly integrate this media player with Active Rendering ?

Thank you !

The Main class where the active rendering is done :

import javax.swing.*;
import java.awt.*;
import java.awt.image.BufferStrategy;
import java.awt.event.KeyAdapter;
import java.awt.event.KeyEvent;
import java.io.*;
import java.lang.reflect.InvocationTargetException;

public class Main extends JFrame
{
    private final static boolean DEBUG_FPS = true;

    private Container contentPane;
    private GraphicsDevice graphicsDevice;
    private GraphicsEnvironment graphicsEnvironment;
    private BufferStrategy bufferStrategy;
    private Graphics2D graphics;

    private boolean running;
    private long lastTime;
    private long timeSinceLastFPSCalculation;
    private long deltaTime;
    private int fps;
    private int frameCount;

    private int screenWidth;
    private int screenHeight;
    private int screenBitDepth;
    private int screenRefreshRate;
    private int TARGET_FPS;
    private long TARGET_FRAME_TIME;

    private Font font;
    private boolean isAnimationRunning;

    public static void main(String[] args){new Main();}

    private void debugFPS()
    {
        graphics.setColor(Color.WHITE);
        graphics.setFont(font); 
        graphics.drawString("FPS: "+fps,30,30);
    }

    public Main()
    {
        graphicsEnvironment = GraphicsEnvironment.getLocalGraphicsEnvironment();
        graphicsDevice = graphicsEnvironment.getDefaultScreenDevice();
        bufferStrategy = null;
        graphics = null;
        
        screenWidth = graphicsDevice.getDisplayMode().getWidth();
        screenHeight = graphicsDevice.getDisplayMode().getHeight();
        screenBitDepth = graphicsDevice.getDisplayMode().getBitDepth();
        screenRefreshRate = graphicsDevice.getDisplayMode().getRefreshRate();
        TARGET_FPS = screenRefreshRate;
        TARGET_FRAME_TIME = (long)1e9/TARGET_FPS;
        
        try
        {
            font = Font.createFont(Font.TRUETYPE_FONT,new File("Font.ttf")).deriveFont(30f);
            graphicsEnvironment.registerFont(font);
        }
        catch(IOException|FontFormatException e){}

        contentPane = getContentPane();
        contentPane.setBackground(new Color(0,0,0,0));
        contentPane.setLayout(null);

        BackgroundPanel backgroundPanel = new BackgroundPanel(screenWidth,screenHeight);
        contentPane.add(backgroundPanel.jfxPanel);
        backgroundPanel.startVideo();
        isAnimationRunning = true;

        setSize(screenWidth,screenHeight);
        setResizable(false);
        setLocationRelativeTo(null);
        setDefaultCloseOperation(EXIT_ON_CLOSE);
        setUndecorated(true);
        setFocusTraversalKeysEnabled(false);
        setIgnoreRepaint(true);
        setVisible(true);
        
        if(graphicsDevice.isFullScreenSupported()) graphicsDevice.setFullScreenWindow(this);
        createBufferStrategy(2);
        bufferStrategy = getBufferStrategy();

        addKeyListener(new KeyAdapter()
        {
            public void keyPressed(KeyEvent e)
            {
                if(e.getKeyCode() == KeyEvent.VK_ESCAPE)
                {
                    running = false;
                }
                else if(e.getKeyCode() == KeyEvent.VK_SPACE)
                {
                    isAnimationRunning = !isAnimationRunning;
                    if(isAnimationRunning) backgroundPanel.startVideo();
                    else backgroundPanel.stopVideo();
                }
            }
        });
        
        new Thread(new Runnable()
        {
            public void run()
            {
                gameLoop();
            }
        }).start();
    }

    public void gameLoop()
    {
        running = true;
        lastTime = System.nanoTime();
        timeSinceLastFPSCalculation = lastTime;
        deltaTime = 0;
        fps = 0;
        frameCount = 0;

        while(running)
        {
            long now = System.nanoTime();
            deltaTime = now - lastTime;
            lastTime += deltaTime;

            update();

            do
            {
                do
                {
                    graphics = (Graphics2D)bufferStrategy.getDrawGraphics();
                    render();
                    graphics.dispose();
                }
                while(bufferStrategy.contentsRestored());

                bufferStrategy.show();
            }
            while(bufferStrategy.contentsLost());

            frameCount++;
            
            if(now-timeSinceLastFPSCalculation >= 1e9)
            {
                fps = frameCount;
                timeSinceLastFPSCalculation = now;
                frameCount = 0;
            }
            
            long sleepTime = TARGET_FRAME_TIME-(System.nanoTime()-lastTime);
            
            if(sleepTime > 0)
            {
                long sleepPhaseBegin = System.nanoTime();
                while(System.nanoTime()-sleepPhaseBegin < sleepTime) Thread.yield();
            }
        }

        graphicsDevice.setFullScreenWindow(null);
        System.exit(0);
    }

    public void update()
    {

    }

    public void renderSwingComponents()
    {
        try
        {
            SwingUtilities.invokeAndWait(new Runnable()
            {
                public void run()
                {
                    contentPane.paintAll(graphics);
                }
            });
        }
        catch(InterruptedException e){Thread.currentThread().interrupt();}
        catch(InvocationTargetException e){}
    }

    public void render()
    {
        graphics.setColor(Color.BLACK);
        graphics.fillRect(0,0,screenWidth,screenHeight);
        graphics.setRenderingHint(RenderingHints.KEY_TEXT_ANTIALIASING,RenderingHints.VALUE_TEXT_ANTIALIAS_ON);
        graphics.setRenderingHint(RenderingHints.KEY_ANTIALIASING,RenderingHints.VALUE_ANTIALIAS_ON);
        graphics.setRenderingHint(RenderingHints.KEY_ALPHA_INTERPOLATION,RenderingHints.VALUE_ALPHA_INTERPOLATION_QUALITY);
        graphics.setRenderingHint(RenderingHints.KEY_COLOR_RENDERING,RenderingHints.VALUE_COLOR_RENDER_QUALITY);
        graphics.setRenderingHint(RenderingHints.KEY_RENDERING,RenderingHints.VALUE_RENDER_QUALITY);
        renderSwingComponents();
        if(DEBUG_FPS) debugFPS();
    }
}

The BackgroundPanel class where the jfxPanel is being held :

import javax.swing.JPanel;
import java.awt.Graphics;
import java.awt.image.BufferedImage;
import javax.imageio.ImageIO;
import java.io.IOException;
import java.io.File;

import javafx.embed.swing.JFXPanel;
import javafx.scene.Scene;
import javafx.scene.media.Media;
import javafx.scene.media.MediaPlayer;
import javafx.scene.media.MediaView;
import javafx.animation.Animation;
import javafx.application.Platform;
import javafx.scene.Group;
import javafx.beans.binding.Bindings;
import java.net.URISyntaxException;
import javafx.concurrent.Task;

public final class BackgroundPanel extends JPanel
{
    private BufferedImage background;

    private JFXPanel jfxPanel;
    private MediaPlayer mediaPlayer;
    private MediaView mediaView;
    private Scene scene;

    public BackgroundPanel(int screenWidth, int screenHeight)
    {
        setOpaque(false);
        setBounds(0,0,screenWidth,screenHeight);
        setLayout(null);
        
        try
        {
            background = ImageIO.read(new File("MenuPanel.png"));
        }
        catch(IOException e){}

        Platform.setImplicitExit(false);
        jfxPanel = new JFXPanel();
        jfxPanel.setBounds(0,0,screenWidth,screenHeight);
        jfxPanel.setFocusable(false);
        
        Platform.runLater(new Runnable()
        {
            public void run()
            {
                String videoPath = "";
                
                try
                {
                    videoPath = getClass().getResource("/Animation.mp4").toURI().toString();
                }
                catch(URISyntaxException e){e.printStackTrace();}

                mediaPlayer = new MediaPlayer(new Media(videoPath));
                mediaPlayer.setCycleCount(0);
                mediaView = new MediaView(mediaPlayer);
                jfxPanel.setScene(new Scene(new Group(mediaView),screenWidth,screenHeight));
                mediaView.fitWidthProperty().bind(Bindings.selectDouble(mediaView.sceneProperty(),"width"));
                mediaView.fitHeightProperty().bind(Bindings.selectDouble(mediaView.sceneProperty(),"height"));
                mediaView.setPreserveRatio(false);
            }
        });

        add(jfxPanel);
    }

    public void startVideo()
    {
        jfxPanel.setVisible(true);
        Platform.runLater(new Runnable(){public void run(){mediaPlayer.play();}});
    }

    public void stopVideo()
    {
        jfxPanel.setVisible(false);
        Platform.runLater(new Runnable(){public void run(){mediaPlayer.stop();}});
    }

    public void paintComponent(Graphics g)
    {
        g.drawImage(background,0,0,this.getWidth(),this.getHeight(),null);
    }
}
Eorix
  • 81
  • 8
  • 2
    As a "general" rule of thumb, you should not be mixing Swing's (passive rendering) with `BufferStrategy`. Also, Swing doesn't understand how to deal with alpha based colors (`contentPane.setBackground(new Color(0,0,0,0));` is going to cause no end of issues). You're creating a `BufferStrategy` which is attached to the native peer of the `Window` (which `JFrame` is extend from), but `JFrame` is a composite component, it's made up of a number of different components laid out to produce the window content you see – MadProgrammer Feb 24 '22 at 23:01
  • 1
    ... so, even without the issues of bridging JavaFX, this it's all competing against each other in ways it shouldn't be – MadProgrammer Feb 24 '22 at 23:01
  • Also note that `Window` itself (the canvas that you `BufferStrategy` is painting to) is not transparent (AWT components are always opaque), so you're running into another incompatibility – MadProgrammer Feb 24 '22 at 23:04
  • Hi @MadProgrammer, thank you for your quick reply. I modified the color alpha issue thanks for the tip. So for the bufferStrategy your saying i should just draw swing's render without double buffering right? I'm not sure if i got that right. – Eorix Feb 24 '22 at 23:06
  • 1
    No, I'm saying you shouldn't mix Swing components and `BufferStrategy`, there's a lot more going under the hood which you're not taking into account. I've been trying to find an example how this "might" be done, but I can't find any one whose accomplished this – MadProgrammer Feb 24 '22 at 23:08
  • 2
    I'd almost consider using a pure JavaFX workflow, it might be simpler to achieve, as JavaFX was originally intended as Flash competitor – MadProgrammer Feb 24 '22 at 23:09
  • Oh okay so that's why i've seen people drawing on AWT.Canvas instead of the JFrame ? – Eorix Feb 24 '22 at 23:10
  • Yeah pure FX would be easier I guess but it's a school project and I have to use swing and a game loop so i've been trying stuff I have to admit ^^ – Eorix Feb 24 '22 at 23:12
  • [How to Use Root Panes](https://docs.oracle.com/javase/tutorial/uiswing/components/rootpane.html) has a nice picture of what a `JFrame` looks like, but `BufferStrategy` will be using the `Frame`'s layer (below the `rootPane`) to render it's content, this could be causing a lot of rendering overlap issues – MadProgrammer Feb 24 '22 at 23:12
  • Alright thank you, so I should paint in another "thing" i.e. use the bufferStrategy from let say an AWT canvas (anything but a swing component) and put that canvas in the contentPane ? – Eorix Feb 24 '22 at 23:17
  • That would be preferable. You might also consider having a look at [Swing animation running extremely slow](https://stackoverflow.com/questions/14886232/swing-animation-running-extremely-slow/14902184#14902184), [Rotating multiple images causing flickering. Java Graphics2D](https://stackoverflow.com/questions/23417786/rotating-multiple-images-causing-flickering-java-graphics2d/23419824#23419824) and [Slow movement using paintComponent method](https://stackoverflow.com/questions/16865391/slow-movement-using-paintcomponent-method/16867943#16867943) - these are all pure Swing implementations – MadProgrammer Feb 24 '22 at 23:21
  • Alright thank you very much that was really helpful ! – Eorix Feb 24 '22 at 23:24
  • Perhaps convert the whole thing to use only JavaFX rather than dealing with interleaving issues between two disparate toolkits. – jewelsea Feb 24 '22 at 23:29
  • Thanks @jewelsea but like I said I wish I could but it's a school project and I have no choice but to use swing .. But maybe i've misunderstood something, I thought that a bufferstrategy was required for active rendering but i just saw that swing component's are already double buffered so i think i could just draw in a JPanel ? Doesn't fix the media player issue but it still something. – Eorix Feb 24 '22 at 23:40
  • Sorry, missed your earlier comment that you must use Swing. I don’t know how to achieve the desired functionality given that requirement. – jewelsea Feb 24 '22 at 23:47
  • *"it's a school project"* I doubt the school would specify that a project has music. My advice would be to stick to the spec. A robust, well written, well-documented implementation of the spec will get better marks (and just as importantly teach what the project was expected to teach) than can likely be achieved by going over-spec in a less rigorous manner. – Andrew Thompson Feb 25 '22 at 06:48
  • HI @AndrewThompson, thank you for your wise reply and indeed this should be an advice to follow most of the time at least it's what I did during the past years for every projects. Now this one I didn't really explained but it's different, it's the final project of my Bachelor's Degree in Computer Science in France and the source code itself is not given to the teachers at the end, there is just an oral with a jury and random people and you have to put on the show, make your demo, and proved that what you've been learning for the past 3 years wasn't for nothing. Hence music and cinematics ! – Eorix Feb 26 '22 at 08:53

0 Answers0