2

We want to share the screen (screenshots) to a browser from an iPad. At the moment we take screenshots and send them over a WebRTC DataChannel, however that requires to much bandwidth.

Sending 5 frames per second fully compressed and scaled, still requires about 1.5-2mb/s upload speed.

We need to utilize some form of video encoding, so we can lower the bandwidth requirements and let WebRTC handle the flow control, depending on connection speed.

AVAssetWriter takes images and converts them to a .MOV file, however doesn't let us get a stream from it.

Any ideas for us? Pretty stuck at the moment, all ideas appreciated.

Thank for suggesting that this is a duplicate, however that doesn't help me very much. I have already a working solution, however it's not good enough.

Edit:

UIGraphicsBeginImageContextWithOptions(view.frame.size, NO, 0.7); //Scaling is slow, but that's not the problem. Network is


[view drawViewHierarchyInRect:view.bounds afterScreenUpdates:NO];

UIImage *image = UIGraphicsGetImageFromCurrentImageContext();

UIGraphicsEndImageContext();

NSData *data = UIImageJPEGRepresentation(image, 0.0); //Compress alot, 0.0 is max, 1.0 is least


NSString *base64Content = [data base64EncodedStringWithOptions:NSDataBase64EncodingEndLineWithLineFeed];

And then I send that base64 data over a WebRTC DataChannel in 16Kb blocks, as suggested by the docs.

dc.send(...)
Jakkra
  • 641
  • 8
  • 25
  • Possible duplicate of [Screen sharing in iOS app?](http://stackoverflow.com/questions/13967252/screen-sharing-in-ios-app) – Kevin May 20 '16 at 14:27
  • @Kevin That question was closed, and was very generic. The answer only shows how to capture the screen to a file. This question has good definition and explains how they're currently implementing the solution (capturing the screen and sending it via data channel). – xdumaine May 20 '16 at 18:11
  • @xdumaine It's simply a suggestion. Who knows, maybe it helps Jakkra. Flagging as duplicate is also a convenient way to link to related information within SO. Notice how it adds a ? at the end, it's not a statement. – Kevin May 21 '16 at 05:36
  • @Kevin Thanks, but as xdumaine said, doesn't help me very much. Already got a working solution, but it's not good enough. Thanks anyway! – Jakkra May 21 '16 at 11:13
  • @Jakkra Can you add your relevant code? We can't help you optimise if we don't know how you handle your data. How do you take screenshots? How do you compress and scale them? How do you pack them? How do you unpack them? If you could post the code for those parts, we can perhaps tell you how to improve it. – Kevin May 21 '16 at 11:37
  • @kevin I didn't mean for you to take it personally. There's just a lot of quick question closures when people flag as duplicate, in my experience. – xdumaine May 21 '16 at 12:14
  • @Kevin added some code. – Jakkra May 21 '16 at 12:23
  • @Jakkra Alright, well one thing to do is stop using base64, because it's about 30% bigger than the original data. – Kevin May 21 '16 at 12:31
  • @Kevin thanks, that will surely improve our current solution :) original problem still stands though. – Jakkra May 21 '16 at 12:50

1 Answers1

0

I would compress the screenshots using a javascript encoder, i.e. MPEG, then transcode this stream on server-side to VP8 for WebRTC.

However it may do not work properly on old iOS devices, i.e. iPad 2010-2011 due low CPU resources, so even if you encode this stream, it may be choppy and not suitable for a smooth playback.

Alex
  • 96
  • 4