4

I'm implementing video capture in a Java program by creating BufferedImages at a user-defined interval (for my testing, 100ms), and then using those images to make a movie file. The JFrame that I am trying to record from includes a dashboard-like interface contained by a JLayeredPane. The JFrame also has two Canvas3Ds. I'm telling each of these 3 things to render or paint into their own LinkedBlockingDeque<BufferedImage>, and I combine them later. The dashboard is set to only render every dashboardFrameRepaintFrequency frames.

Thread captureThread = new Thread(new Runnable() {
    public void run() {
        final Object LOCK = new Object();
            final Thread captureDashboard = new Thread(new Runnable() {
                public void run() {
                    while (m_isCapturingMovie) {
                        synchronized (LOCK) {
                            try {
                                LOCK.wait();
                            } catch (InterruptedException e) {
                                e.printStackTrace();
                            } finally {
                                System.err.println("Calling getDashboardImage");
                                m_unsavedDash.add(getDashboardImage(m_dashboard.getWidth(), m_dashboard.getHeight(), BufferedImage.TYPE_INT_ARGB));
                                System.err.println("captureDashboard returned from calling m_unsavedDash.add...");
                            }
                        }
                    }
                }
            });
            captureDashboard.start();

            while (m_isCapturingMovie) {
                startTime = System.currentTimeMillis();
                captureCanvases();
                if (++frameCount > dashboardFrameRepaintFrequency) {
                    frameCount = 0;
                    synchronized (LOCK) {
                        LOCK.notify();
                    }
            }
            endTime = System.currentTimeMillis();
            millisToSleep = captureDelayInMillis - (endTime - startTime);
            if (millisToSleep > 0) {
                try {
                    Thread.sleep(millisToSleep);
                } catch (InterruptedException e) {
                    e.printStackTrace();
                }
            }
        }
        synchronized (captureDashboard) {
        captureDashboard.notify();
        }
    }
});

I've found that after 15-20 notify()s, the program locks up - it stops recording the Canvases, stops responding to keyboard input.. I can still change which window has focus, and the buttons (such as the X button to close a window) still change visual state from mouse rollover or clicking, but they don't execute their commands.

From the console output, it seems that the captureDashboard thread, after those 15-20 iterations, does not return from the getDashboardImage method:

private BufferedImage getDashboardImage(int width, int height, int type) {
    BufferedImage dashImg = new BufferedImage(m_dashboard.getWidth(), m_dashboard.getHeight(), type);
    Graphics2D g = dashImg.createGraphics();
    m_dashboard.paintAll(g);
    g.dispose();

    return getScaledImage(width, height, dashImg);
}

private BufferedImage getScaledImage(int width, int height, BufferedImage original) {
    BufferedImage scaled = new BufferedImage(width, height, original.getType());

    AffineTransform at = new AffineTransform();
    at.scale((double) scaled.getWidth() / original.getWidth(), (double) scaled.getHeight() / original.getHeight());
    AffineTransformOp scaleOp;
    if (at.getScaleX() + at.getScaleY() > 2.0) {
        scaleOp = new AffineTransformOp(at, AffineTransformOp.TYPE_BICUBIC); // Better quality for enlargement
    } else {
        scaleOp = new AffineTransformOp(at, AffineTransformOp.TYPE_BILINEAR); // Better quality for ensmallment
    }

    scaled = scaleOp.filter(original, scaled);
    original.flush();
    return scaled;
}

Any ideas? I've been working at this for a few days and I'm stumped.

Szeren
  • 61
  • 5
  • 1
    +1 A first post that's actually well formatted, not asinine and effort was obviously put into it. Welcome to SO. Here, take the [tour](http://stackoverflow.com/about). – Steve P. Jul 19 '13 at 16:17
  • Maybe OP did the formatting because for the first time I am having problem also posting the answer with formatting. For some weird reason, the formatting gets lost. Tried different ways but couldn't get it to work in my last answer. – Sajal Dutta Jul 19 '13 at 16:31
  • @SajalDutta I can almost guarantee it's because you were using tabs, which become 8 spaces, instead of just having 4 actual spaces. 99% of the problems with code formatting in questions are because of that. – Brian Jul 19 '13 at 16:40
  • Is there a reason that you aren't just using the `Robot` class? See [this question](http://stackoverflow.com/questions/58305/is-there-a-way-to-take-a-screenshot-using-java-and-save-it-to-some-sort-of-image). All you'd do is just limit the rectangle passed to the screenshot method to the rectangle of your window. Plus, this uses native code so will be much faster. – Brian Jul 19 '13 at 16:40
  • @Brian, no tab as tab doesn't work (or I am wrong) in SO editor. Let me try again. – Sajal Dutta Jul 19 '13 at 16:43
  • @Brian I tried copying my answer in notepad++ to see if there's any tab. None. All 4 spaces each. Doesn't work. See- http://stackoverflow.com/questions/17750575/flelwriter-in-java-not-writing-to-txt-file/17750611#17750611 – Sajal Dutta Jul 19 '13 at 16:47
  • @Brian `Robot` had too much of a performance impact on the program - everything ran much slower from the user's perspective AND the actual captured frame rate was much worse (about half the fps of rendering extra frames). – Szeren Jul 19 '13 at 17:02
  • @SajalDutta I might have messed up by using pasted tabs. I just replaced them with four spaces. Sorry! – Szeren Jul 19 '13 at 17:11

1 Answers1

2

The problem was that I needed to call paintAll from the AWT dispatch thread.

So instead of:

m_dashboard.paintAll(g);

I needed to have:

final Graphics2D g = dashImg.createGraphics();

EventQueue.invokeLater(new Runnable() {
    public void run () {
        m_dashboard.paintAll(g);
    }
});

This, however, caused my program to "get ahead of itself" and return the BufferedImage before it had been painted to, but ONLY if the program was under heavy load. To account for this, I just added the following:

final Graphics2D g = dashImg.createGraphics();
final SyncObj LOCK = new SyncObj();

EventQueue.invokeLater(new Runnable() {
    public void run () {
        m_dashboard.paintAll(g);
        LOCK.doNotify();
    }
});

LOCK.doWait();
g.dispose();

Where SyncObj is just a simple:

class SyncObj {
    private boolean condition = false;
    private Object obj = new Object();

    public void doWait() {
        synchronized (obj) {
            while (!condition) {
                try {
                    obj.wait();
                } catch (InterruptedException e) {
                    e.printStackTrace();
                }
            }
            condition = false;
        }
    }

    public void doNotify() {
        synchronized (obj) {
            condition = true;
            obj.notify();
        }
    }
}
Szeren
  • 61
  • 5