What I need:
I need to get all the currently applicable cookies for a page state. For example, I navigate to a site in the US, the browser has a lot of cookies populated by that page. I then decline all cookies on that site. Afterwards the browser reports much fewer cookies for that page. I need to be able to get that list of applicable cookies for the page as displayed in Chrome's View site information pop-out dialog left of the URL.
What I've tried:
I'm currently using Puppeteer, though I've also tried using Playwright. Both can utilize the Chrome Devtools Protocol (CDP) to gather information about a browser session and other related HTTP data.
I have a use case where I need to see the current state of cookies as it pertains to a web page. CDP provides two methods to accomplish this. Network.getAllCookies and Network.getCookies.
Network.getCookies returns cookies for a given URL, or else it provides all the cookies for the URL of the current page when no URL is provided. The problem with this is it's not a full picture. Network.getCookies does not include 3rd party cookies unless that domain is specifically requested. But there's no way for me to know all the 3rd party cookies for a given page.
If I switch it up and use Network.getAllCookies, that has too much information. It returns with all the cookies currently known to the browser session, regardless of applicability to the current page. For example, if I visit site A, then nav to site B, then I call Network.getAllCookies, I get all of the cookies from both pages since they are all in the same browser context. But I only want the 1st and 3rd party cookies for the current page I'm on.
In a headful browser it is possible to see all the applicable 1st and 3rd party cookies are for a site. If you click on the lock icon (secure site) to the left of the url, there is an option to view cookies. This is the most complete and accurate view of cookies I've come to know. Somehow the browser knows what 3rd party cookies originated from the current page and requests made by the current page and therefore displays them there. But how? And how can I get that list through CDP?
Note that document.cookie is useless as it does not include httpOnly cookies, and to my knowledge also doesn't return detailed cookie information (such as sameSite, secure, expiry, etc.)
I've tried using the CDP session directly attached to both of the above mentioned libraries. I've tried monitoring HTTP requests using Puppeteer and gathering set-cookie data from response headers, but CDP is unreliable when it comes to providing all the response header set-cookie information.
TLDR:
I'm expecting to programmatically retrieve only the relevant cookies for a page visit up to that point. If cookies become irrelevant in that page session, I don't want to receive them any more.
Network.getAllCookies returns too many cookies as it gives the whole browser context and not just the page relevant cookies.
Network.getCookies only returns cookies by a url.
document.cookie only returns application cookies.
How can I get all the 1st and 3rd party cookies that pertain to my current page?
I want this exact list:
