How to take screenshot (high fps) in Linux (programming)

Question

First of all I want to say that I've been reading a lot about this and I've learnt many ways to do it, but I haven't been able to do it in linux.

My project is an ambilight with arduino, so I need to take a screenshot of the desktop and analyze its colour.

At the beginning I used Processing 2.0 with the class 'Robot' from 'java.awt'. Initially I could take 5 frames per second and then I got 13fps. This works but I want more perfomance, so I start reading.

In Windows or Mac you have libraries that let you access directly to the 'frameBuffer', so you can take screenshot really 'easy' and really fast.

In Ubuntu I have tried python with Gtk, PIL, Qt... and the fastest way is GTK but I can only have about 15fps too.

My problem is: I want to do it cross platform but I prefer that my program work in Linux at the beginning and then in Windows (I don't like it too much :P).

So, the first question: is python able to offer that perfomance? Because I think that C++ can be a better option.

And the second question: what do I need to do it? I've read about Xlib (X11) but I can't find documentation that let me take a screenshot. Also I know, for example, FFmpeg which is a powerful tool but I don't know how to implement it.

I hope that you could help me (and excuse me if I've made any mistakes).

http://manpages.ubuntu.com/manpages/hardy/man1/fbgrab.1.html have you tried fbgrab? maybe this will help you, just an idea. — cerkiewny, Aug 06 '14 at 14:16
@cerkiewny It can be useful but it's launched from terminal and saves the img to a file that I will have to read again (it can be slower?), but we can read the code and look how does it. — pirobtumen, Aug 06 '14 at 14:19
possible duplicate of [Taking screenshot with libx11](http://stackoverflow.com/questions/10284858/taking-screenshot-with-libx11) — Raydel Miranda, Aug 06 '14 at 14:25
@pirobtumen you can directly access /dev/fb1 by simply reading this, but any activity in the user space will require copying a lot of kernel space data from the frame buffer. I agree you can do better than saving to a file. — cerkiewny, Aug 06 '14 at 14:25
@pirobtumen If you search in SO with the words `screenshot` and `Xlib` you'll find a lot of questions about this. — Raydel Miranda, Aug 06 '14 at 14:29
@RaydelMiranda Do you know if Xlib call is copying memory or remapping it? — cerkiewny, Aug 06 '14 at 14:31
@cerkiewny No, I don´t. I flag that question as duplicate because the question OP prefers C over Python (for performance) and there is in fact an answer here in StackOverflow that shows how to take the screenshot with C in linux, Using Xlib. But yours classify as a new (and very good) question. — Raydel Miranda, Aug 06 '14 at 14:41
@RaydelMiranda Oh, I am not referring to that, I was just curious myself whether there even exist the remapping method. — cerkiewny, Aug 06 '14 at 14:43
@cerkiewny As I said in my previous comment, thats a very good question, I recommend you to post it here. :D — Raydel Miranda, Aug 06 '14 at 14:44
@RaydelMiranda I'm not asking how to take a screenshot using X11 because I read that post. I'm asking which can be the best way to do it and id C is better than python for that or not. Anyway thanks for answering. — pirobtumen, Aug 06 '14 at 14:57

score 2 · Answer 1 · edited May 23 '17 at 10:33

Making this work cross platform is likely to be quite a bit of work. If your final target is windows, then why not use the amblone project, which seems to do exactly what you want?

http://amblone.com/guide

At any rate, here is a solution with ffmpeg & graphicsmagick that is pretty fast (on my i7 8GB laptop). ffmpeg captures exactly one screen, reduces it to the smallest square size that it can, pipes the output to graphicsmagick convert, where it is resized to 1x1 pixel and then reports the image rgb values.

#!/bin/bash

mkfifo /tmp/screencap.fifo

while true
    do
        # this version will send the info to a fifo
        # ffmpeg -y -loglevel error -f x11grab -s 1920x1080 -i :0.0 -s 32x32 \
        # -vframes 1 -f image2 -threads 2 - |  gm convert - -resize 1x1 \
        # txt:- > /tmp/screencap.fifo

        # this version will write out the info to the command line
        # and will show you what is going on.
        ffmpeg -y -loglevel error -f x11grab -s 1920x1080 -i :0.0 -s 32x32 \
         -vframes 1 -f image2 -threads 2 - |  gm convert - -resize 1x1 txt:-
    done
exit

This will give you something like the following:

0,0: ( 62, 63, 63) #3E3F3F
0,0: (204,205,203) #CCCDCB
0,0: ( 77, 78, 76) #4D4E4C

The 0,0 is the location of the pixel being read. The numbers in parenthesis are the respective R,G,B values, and the numbers at the end are your typical html-esque hex values. In the case above there is only 1 pixel, but you could (if you wanted to have the cardinal directions as generalized RGB values) simply change the -resize 1x1 part above to -resize 3x3 and you'll get something like:

0,0: ( 62, 63, 65) #3E3F41
1,0: ( 90, 90, 91) #5A5A5B
2,0: (104,105,106) #68696A
0,1: ( 52, 51, 52) #343334
1,1: ( 60, 60, 59) #3C3C3B
2,1: ( 64, 64, 64) #404040
0,2: ( 49, 49, 50) #313132
1,2: ( 60, 60, 60) #3C3C3C
2,2: ( 65, 65, 65) #414141

I'll leave it to you to pass that information to your arduino.

ffmpeg is great, but you'll have to remember to switch out the screen-capture bit (here in my example -f x11grab) with whatever your windows system uses. Here is a SO link that goes into a bit more detail.

If you really insist on making something cross-platform, then I would recommend diving into openCV with python bindings and using the framebuffer device as a video input, scaling the result down to 1x1 pixel and using the resulting color average to drive your pwm through some type of UDP broadcast.

How to take screenshot (high fps) in Linux (programming)

1 Answers1

Linked