0

I am trying to develop a basic screen sharing and collaboration app in C#. I am currently working on capturing the screen, finding areas of the screen that have changed and subsequently need to be transmitted to the end client.

I am having a problem in that the overall frame rate of the screen capture is too low. I have a fairly good algorithm for finding areas of the screen that have changed. Given a byte array of pixels on the screen it calculates areas that have changed in 2-4ms, however the overall frame rate I am getting is 15-18 fps (i.e. taking somewhere around 60ms per frame). The bottleneck is capturing the data on the screen as a byte array which is taking around 35-50ms. I have tried a couple of different techniques and can't push the fps past 20.

At first I tried something like this:

var _bmp = new Bitmap(screenSectionToMonitor.Width, screenSectionToMonitor.Height);
var _gfx = Graphics.FromImage(_bmp);
_gfx.CopyFromScreen(_screenSectionToMonitor.X, _screenSectionToMonitor.Y, 0, 0, new Size(_screenSectionToMonitor.Width, _screenSectionToMonitor.Height), CopyPixelOperation.SourceCopy);
var data = _bmp.LockBits(new Rectangle(0, 0, _screenSectionToMonitor.Width, _screenSectionToMonitor.Height), ImageLockMode.ReadOnly, _bmp.PixelFormat);
var ptr = data.Scan0;
Marshal.Copy(ptr, _screenshot, 0, _screenSectionToMonitor.Height * _screenSectionToMonitor.Width * _bytesPerPixel);
_bmp.UnlockBits(data);

This is too slow taking around 45ms just to run the code above for a single 1080p screen. This makes the overall frame rate too slow to be smooth, so I then tried using DirectX as per the example here:

http://www.codeproject.com/Articles/274461/Very-fast-screen-capture-using-DirectX-in-Csharp

However this didn't really net any results. It marginally increased the speed of the screen capture but it was still much too slow (taking around 25-40ms, and the small increase wasn't worth the overhead of the extra DLLs, code, etc.

After googling around a bit I couldn't really find any better solutions, so my question is what is the best way to capture the pixels currently displaying on the screen? An ideal solution would:

  • Capture the screen as an array of bytes as RGBA
  • Work on older windows platforms (e.g. Windows XP and above)
  • Work with multiple displays
  • Uses existing system libraries rather than 3rd party DLLs

All these points are negotiable for a solution that return a decent overall framerate, in the region of 5-10ms for the actual capturing so the framerate can be 40-60fps.

Alternatively, If there no solution that matches above, am I taking the wrong path to calculate screen changes. Is there a better way to calculate areas of the screen that have changed?

Anduril
  • 1,236
  • 1
  • 9
  • 33
  • Have you looked at DFMirage? It is what TightVNC uses to gain performance on windows. – John Koerner Nov 25 '14 at 15:24
  • I have feeling your approach is wrong. Grab screenshot of the whole desktop and then process areas, etc. Try to profile what takes most of time? I am quite sure it will be some function to lock or convert, which you call *multiple times for each region*. – Sinatr Nov 25 '14 at 15:27
  • I agree with @Sinatr - obtaining raw screen data frequently may be fast, but processing that data won't be. – aevitas Nov 25 '14 at 15:29
  • @Sinatr I have profiled the app. The processing takes between 2 - 4 ms. the call to `_gfx.CopyFromScreen(_screenSectionToMonitor.X, _screenSectionToMonitor.Y, 0, 0, new Size(_screenSectionToMonitor.Width, _screenSectionToMonitor.Height), CopyPixelOperation.SourceCopy);` is taking 35+ms on its own, and that is not my code, and there is no locking. That single call is the bottleneck – Anduril Nov 25 '14 at 15:51
  • `Graphics.CopyFromScreen` is [slow](http://stackoverflow.com/q/6812068/1997232). What I meant is to use `DirectX` to capture stream of whole screen at once (locking is not `lock` in `c#` but a technique to get direct access to buffer, similar to `Bitmap.LockBits`). That should be fastest you can expect to get without using drivers. – Sinatr Nov 25 '14 at 16:14

1 Answers1

0

Perhaps you can access the screen buffers at a lower level of code and hook directly into the layers and regions Windows uses as part of its screen updates. It sounds like you are after the raw display changes and Windows already has to keep track of this data. Just offering a direction for you to pursue while you find someone more knowledgeable.

Suncat2000
  • 966
  • 1
  • 12
  • 15
  • That is exactly the kind of approach I was hoping for. Since the data is obviously being displayed on the screen, it makes sense that it is sitting in memory/buffer somewhere, just need to know how to access it, no luck so far though – Anduril Nov 25 '14 at 15:25