1

I am trying to get the color of pixels/points (doesn't matter for my use case) of the current screen content in iOS. So, for example, I want to get the color of each pixel from screen coordinates 0, 0 to 10, 10. Additionally, the operation should be as fast as possible, since I will do it at regular intervals as a Timer. The timer should run multiple times a second, but it doesn't have to be 25fps.

Acceptable solutions: Anything that returns the current color of a pixel or point on screen at a given position, doesn't produce noticable UI lag and doesn't turn my app into a battery hog. The result might be a CGImage, UIImage, buffer array, I don't really care. I also don't care if the solution uses additional Apple frameworks, such as OpenGL or Metal.
It is also acceptable if the solution does not capture system-UI, like the statusbar. Capturing the content of my app is sufficient.

Things I tried so far:

  • Using UIWindows drawHierarchy(in:afterScreenUpdates:). This method turns out to be way too slow. On my iPad Pro, it took 0.25s which causes noticable UI lag.
  • Using CALayers render(in:), but this method does not render UIVisualEffectViews, which I require. Also, while faster than drawHierarchy, I measured it at about 0.04s, which still causes noticable lag in the UI.
  • Use OpenGL, as for example described here. I don't know anything about OpenGL, so I might be using this wrong, but I never got it to return anything other than a black image.
Hamid Yusifli
  • 9,688
  • 2
  • 24
  • 48
BlackWolf
  • 5,239
  • 5
  • 33
  • 60
  • You need to provide a bit more information. ***When*** are you trying to get the pixel colors? On a button tap? On some other event? So you can do something to them? Might help to review [ask]. – DonMag Mar 27 '20 at 18:34
  • Thanks, I added information about that. I can sort of control the when freely, but it should be multiple times a second (since I update other UI components based on that). Most likely, the code will end up in a timer that runs a couple times a second. – BlackWolf Mar 27 '20 at 18:39

0 Answers0