Cloudflare bypass with Puppeteer

Question

I'm trying to enter hotbit.io, with my Puppeteer. But I'm met with "Checking your browser before accessing www.hotbit.io" the moment puppeteer tries entering the page. When I run my program in "headless: false" it redirects to the page after 5 seconds. But my problem is, that I want to run it in headless: true. When I run it in headless: true, it timesout on the cloudflare page Screenshot at timeout

I have tried:

"puppeteer-extra-plugin-stealth"
"Cloudflare-scraper (https://www.npmjs.com/package/cloudflare-scraper)". This has very limited documentation (non-existing), but I saw under "Issues" on their github, that it is not supported anymore.

It seems like, that cloudflare knows, that I'm having headless activated.

Does anyone know, how I can skip the cloudflare redirecting page?

You require a header to tell cloudflare who you are. – BGPHiJACK Apr 19 '22 at 13:17 — BGPHiJACK, Apr 19 '22 at 13:17
setUserAgent() should be a start to your solution, GL! – BGPHiJACK Apr 19 '22 at 13:17 — BGPHiJACK, Apr 19 '22 at 13:17

score 22 · Accepted Answer · answered Apr 19 '22 at 17:41

22

Thank you @BGPHiJACK !

It helped by setting user agent to: Mozilla/5.0 (Windows NT 5.1; rv:5.0) Gecko/20100101 Firefox/5.0

So right after I have init the page, I set user agent.

const page = await browser.newPage();
    
await page.setUserAgent('Mozilla/5.0 (Windows NT 5.1; rv:5.0) Gecko/20100101 Firefox/5.0')

answered Apr 19 '22 at 17:41

Ivan G.

299
1
2
6

1

Be aware this will not necessarily always work. CloudFlare's scraper protection systems update frequently. Long-term, you may want to coordinate a deal with the site you're accessing for uninterrupted access. – ceejayoz Apr 19 '22 at 17:45
3

Just to emphasize, i used chrome as userAgent and it did not work , so make sure you used latest firefox version as of today it should be `await page.setUserAgent('Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:109.0) Gecko/20100101 Firefox/110.0')` – Emmanuel David Apr 12 '23 at 13:36

Cloudflare bypass with Puppeteer

1 Answers1

Linked