I'm running puppeteer on AWS EC2 AMI (Linux) in headless mode to do some web scraping. Is it possible to track the GUI of it remotely, maybe from my local Windows setup?
I referred to this article and was wondering if adding --remote-debugging-port=9222 to my code and then visiting http://localhost:9222 on a local browser would let me see the GUI. Is it possible to do this? I'm not sure if I'm on the right track here. Any help would be appreciated.
Yes, this is possible. You have to set the --remote-debugging-address argument.
Quote from the List of Chromium Command Line Switches:
Use the given address instead of the default loopback for accepting remote debugging connections. Should be used together with --remote-debugging-port. Note that the remote debugging protocol does not perform any authentication, so exposing it too widely can be a security risk.
Explanation
By default, Chrome binds to 127.0.0.1 (the local-only interface), which only allows connections from the machine itself. If you set the argument to 0.0.0.0 Chrome listens to all network interfaces allowing connections from outside of the machine (check out this answer on stackoverflow for more information). However, you still need to make sure there are no firewalls in place that might block connections, but by default that should not be the case.
That means, starting Chrome like this will allow debugging from another computer:
chrome --remote-debugging-port=9222 --remote-debugging-address=0.0.0.0 [possible other flags]
You could now visit http://IP_OF_YOUR_MACHINE:9222 and would be confronted with the DevTools GUI of Chrome. Alternatively, you could connect to the machine using puppeteer.connect:
const browser = await puppeteer.connect({ browserURL: 'http://...:9222' });
// ...
Security Considerations
Keep in mind that this port is exposed to anyone with access to the Internet. You might want to consider using a Firewall to block connections or use a library like node-http-proxy to filter any connections before passing them on to your browser.
