20

Some companies are offering manual testing of real iPhone/iPad devices. With your mouse and keyboard, you can control the device straight from your browser.

They probably use something like AirPlay to stream the device graphics to the browser. But how do they convert the mouse-clicks to touch events on iPhone/iPad? Since it's not possible to run a VNC server on the device, I'm wondering if there's another way to do this.

Jochen
  • 1,853
  • 3
  • 20
  • 28
  • Are they using real devices, or could it be a simulator running remotely? Have an example? – Beau Nouvelle Oct 17 '15 at 10:10
  • http://stackoverflow.com/questions/6742227/how-to-connect-wireless-mouse-with-iphone , http://apple.stackexchange.com/questions/203178/using-mouse-from-osx-host-on-ios-device same as this – Leena Oct 17 '15 at 10:40
  • @BeauYoung they're using real devices: perfectomobile.com - keynotedeviceanywhere.com – Jochen Oct 17 '15 at 11:05
  • Based on this comment (http://stackoverflow.com/questions/33184943/controlling-ios-device-via-mouse-keyboard#comment54618358_33362663), it seems like you're looking for a tool recommendation. Those kinds of questions are off-topic for StackOverflow. – JAL Oct 29 '15 at 14:04
  • @JAL I'm looking for a solution, which is on-topic for StackOverflow. – Jochen Oct 29 '15 at 14:17
  • 2
    @Jochen By that reasoning, "why isn't my laptop recognizing my USB drive" would be on-topic. I'm looking for a solution, after all. – Paul Roub Oct 29 '15 at 14:34
  • 3
    By that reasoning, asking what you get when you dissolve salt in water would also be on topic. – TylerH Nov 02 '15 at 20:10
  • @TylerH thanks for your on-topic input, much appreciated – Jochen Nov 03 '15 at 07:52

1 Answers1

0

I do not know exactly how this services work, but i guess you have to include a framework from them in your app. This framework may contain a little server component which receives the mouse events and applies them to the app.

func applicationDidBecomeActive(application: UIApplication) {
    MyRemoteControlFramework.startServer()
}

If one mouse-clicks on a location on the screen, the framework may first perform a hittest to find the view which should receive the click (see Hittest on UIView). When the target view is found, the framework can call touchesBegan, touchesMoved, touchesEnded, ... on this view. It is a little work but i think it is not to complex. Also the screen can be captured easily and sent through the server component (see How to take a screenshot).

Community
  • 1
  • 1
Florian L.
  • 849
  • 2
  • 8
  • 22
  • A caveat of this solution is that you have to add extra code to support the incoming data from the framework, and have to handle the parsing. It seems that OP wants VNC-like solution, which is independent of any specific app and interacts with the phone itself. (But is that even possible? Hmm) – Flying_Banana Oct 27 '15 at 08:19
  • 1
    It is for sure not possible to control the whole phone (without jailbreak it or having a robot arm touching the screen). But if you just want to remote control your app, then i think this is the way to go. – Florian L. Oct 27 '15 at 08:24
  • You can attach hardware to the phone (lightning connector) which emulates touches like a stylus or keyboard etc., that would also work. It depends on what the use case is. – Florian L. Oct 27 '15 at 08:26
  • I'm aware of this solution. What I'm looking for is an overall solution, which works on a vanilla iPhone without modifying apps. Something like https://github.com/iolate/SimulateTouch but without having to jailbreak the device – Jochen Oct 29 '15 at 12:33