Is it possible to capture websocket traffic using selenium and python?

Question

we are using selenium on python and as a part of our automation, we need to capture the messages that a sample website sends and receives after the web page loaded completely.

I have check here and it is stated that what we want to do is achievable using BrowserMobProxy but after testing that, the websocket connection did not work on website and certificate errors were also cumbersome.

In another post here it is stated that, this can be done using loggingPrefs of Chrome but it seemed that we only get the logs up to the time when website loads and not the data after that.

Is it possible to capture websocket traffic using only selenium?

I think [this article](https://medium.com/@lagenar/using-headless-chrome-via-the-websockets-interface-5f498fb67e0f) may just be what you're looking for. Hope it helps. — Nico V, Dec 09 '19 at 11:37

score 14 · Accepted Answer · answered Jun 13 '21 at 13:57

Turned out that it can be done using pyppeteer; In the following code, all the live websocket traffic of a sample website is being captured:

import asyncio
from pyppeteer import launch

async def main():
    browser = await launch(
        headless=True,
        args=['--no-sandbox'],
        autoClose=False
    )
    page = await browser.newPage()
    await page.goto('https://www.tradingview.com/symbols/BTCUSD/')
    cdp = await page.target.createCDPSession()
    await cdp.send('Network.enable')
    await cdp.send('Page.enable')

    def printResponse(response):
        print(response)

    cdp.on('Network.webSocketFrameReceived', printResponse)  # Calls printResponse when a websocket is received
    cdp.on('Network.webSocketFrameSent', printResponse)  # Calls printResponse when a websocket is sent
    await asyncio.sleep(100)


asyncio.get_event_loop().run_until_complete(main())

Logged in to thank you for this, didn't know this existed You are amazing — John13, Jan 25 '23 at 02:20
This is nice that you can capture traffic from websocket. But how can you make a click from callback function `printResponse`, as it is synchronous? Basically how you can evaluate something like `await page.click('a')` inside the sync function? For example, I would like to click or not based on info taken from websocket. How to do that? Will appreciate for answer. — goodgrief, Jun 11 '23 at 19:23

Is it possible to capture websocket traffic using selenium and python?

1 Answers1