1

This website is to able tell the difference between a real chrome browser and chromedriver. Does anybody know what is the difference between a real chrome browser and chromedriver? Thanks.

https://www.impactaging.com/full/11/908

$ cat chrdvrget.py
#!/usr/bin/env python3

import sys

from selenium import webdriver

options = webdriver.ChromeOptions()
options.add_argument('--headless')

browser = webdriver.Chrome('chromedriver', options=options)

browser.get(sys.argv[1])
sys.stdout.write(browser.page_source)
browser.close()
$ ./chrdvrget.py https://www.impactaging.com/full/11/908
<html><head>
  <script src="https://ajax.googleapis.com/ajax/libs/webfont/1/webfont.js" type="text/javascript" async=""></script><script id="meteor-headers" type="application/ejson">{"token":1590854299420.4485,"headers":{"x-forwarded-for":"128.194.2.41","x-forwarded-proto":"https","x-forwarded-port":"443","host":"www.aging-us.com","x-amzn-trace-id":"Root=1-5ed2829b-12b85ab4e6b408f839aca21c","upgrade-insecure-requests":"1","accept":"text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9","sec-fetch-site":"none","sec-fetch-mode":"navigate","sec-fetch-user":"?1","sec-fetch-dest":"document","accept-encoding":"gzip, deflate, br","accept-language":"en-US","x-ip-chain":"128.194.2.41,172.16.3.155"}}</script>

  <link rel="stylesheet" type="text/css" class="__meteor-css__" href="/23e8c653e8c598c40de2bfed84e64681cf9fe6b7.css?meteor_css_resource=true">
<script id="irga-analytics" async="" src="//www.google-analytics.com/analytics.js"></script>
<meta name="fragment" content="!">
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1, maximum-scale=1.0">
    <meta name="google-site-verification" content="d2_-UPxLNh2h2_LXNOVTluwzz1X0G8w1o7NcXwNDWjY">
    <meta name="p:domain_verify" content="e99d9967df904cd1dd4e4063bf796a0a">
    <meta name="p:domain_verify" content="6022de5b1b2e4515847cbcbc8f4fc3ad">
    <link rel="stylesheet" href="https://use.fontawesome.com/releases/v5.0.6/css/all.css">
    <meta name="fragment" content="!">
    <title>Aging</title>
    <script>
        window.prerenderReady = false;
    </script>

<script id="altmetric-embed-js" src="https://d1bxh8uas1mnw7.cloudfront.net/assets/altmetric_badges-75bc9437b4bcd96622a3f013e4e9519d1b65ea847ab601ad6158cf84b9291df9.js"></script></head>
<body>
  <script type="text/javascript">__meteor_runtime_config__ = JSON.parse(decodeURIComponent("%7B%22meteorRelease%22%3A%22METEOR%401.8.1%22%2C%22meteorEnv%22%3A%7B%22NODE_ENV%22%3A%22production%22%2C%22TEST_METADATA%22%3A%22%7B%7D%22%7D%2C%22PUBLIC_SETTINGS%22%3A%7B%22journal%22%3A%7B%22issn%22%3A%221945-4589%22%2C%22archive_description%22%3A%22Aging%20US%20has%20been%20publishing%20since%202009%20and%20has%20amassed%20vol_count%20volumes%20as%20of%20current_yr.%22%2C%22home_title%22%3A%22Revolutionizing%20gerontology%20by%20abolishing%20dogmas%22%2C%22logo%22%3A%7B%22banner%22%3A%22%2Fimages%2Faging_logo.png%22%2C%22sharing%22%3A%22%2Fimages%2Faging-logo-blue.png%22%2C%22shareMeta%22%3A%22%2Fimages%2Faging-meta-logo.png%22%7D%2C%22name%22%3A%22Aging%22%2C%22nameExtra%22%3Anull%2C%22site%22%3A%7B%22spec%22%3A%7B%22color%22%3A%7B%22main_hex%22%3A%220a588f%22%2C%22main_rgb%22%3A%2255%2C%2071%2C%2079%22%7D%7D%7D%2C%22submissionsLink%22%3A%22http%3A%2F%2Faging.msubmit.net%2F%22%2C%22altmetric%22%3A%7B%22reportLink%22%3A%22https%3A%2F%2Faging.altmetric.com%2Fdetails%2F%22%2C%22threshold%22%3A10%2C%22template%22%3A%22aging%22%7D%2C%22reprintEmail%22%3A%22printing%40oncotarget.com%22%2C%22siteUrl%22%3A%22https%3A%2F%2Fwww.aging-us.com%22%2C%22sitemapHostUrl%22%3A%22http%3A%2F%2Flocalhost%3A3031%22%7D%2C%22ga%22%3A%7B%22id%22%3A%22UA-74807910-2%22%2C%22trackUserId%22%3Atrue%7D%2C%22s3%22%3A%7B%22bucket%22%3A%22paperchase-aging%22%7D%7D%2C%22ROOT_URL%22%3A%22http%3A%2F%2Faging-cyan.papercha.se%22%2C%22ROOT_URL_PATH_PREFIX%22%3A%22%22%2C%22autoupdate%22%3A%7B%22versions%22%3A%7B%22web.browser%22%3A%7B%22version%22%3A%22b6de6109e579c8788504642644e2aaa8e4fbe19e%22%2C%22versionRefreshable%22%3A%2254bc5b3a9be8ab8b81d73cf07c7a383577471433%22%2C%22versionNonRefreshable%22%3A%22a297acb1cb2103faf27cadad76f96cec9e061066%22%7D%2C%22web.browser.legacy%22%3A%7B%22version%22%3A%22c6651435e54c4057af380b2d941b454c975bd199%22%2C%22versionRefreshable%22%3A%2254bc5b3a9be8ab8b81d73cf07c7a383577471433%22%2C%22versionNonRefreshable%22%3A%22a4fb4e266d67d4f9368deaa186db7760aff155bb%22%7D%7D%2C%22autoupdateVersion%22%3Anull%2C%22autoupdateVersionRefreshable%22%3Anull%2C%22autoupdateVersionCordova%22%3Anull%2C%22appId%22%3A%221w0aki1inxkymkvvdn6%22%7D%2C%22appId%22%3A%221w0aki1inxkymkvvdn6%22%2C%22isModern%22%3Atrue%7D"))</script>

  <script type="text/javascript" src="/4926fb393a332fa3481bd3a225f0ee7d42684908.js?meteor_js_resource=true"></script>











  <div class="hiddendiv common"></div></body></html> 

P.S. These links don't provide an answer to distinguish the difference. And the answers there are also contradictory among them. Therefore they should not be considered as answering my question.

Please provide a working example using the URL at the beginning so that the python code can download the content of the webpage same as a real browser.

Webpage Is Detecting Selenium Webdriver with Chromedriver as a bot Selenium webdriver: Modifying navigator.webdriver flag to prevent selenium detection

user1424739
  • 11,937
  • 17
  • 63
  • 152
  • Probably the User-Agent header is different. Using ChromeDriver you can change this value like this: ```var ChromeOptions options = new ChromeOptions(); options.addArguments("--user-agent = Another UserAgent Value") ``` – zforgo May 30 '20 at 16:10
  • Here is the Python version: ```chrome_options = webdriver.ChromeOptions() chrome_options.add_argument('--user-agent="Another UserAgent Value") driver = webdriver.Chrome(chrome_options=chrome_options)``` – zforgo May 30 '20 at 16:12
  • No. That is not the reason. The chromdriver and my real chrome browser have the same user agent. The website should have some advanced bot detection method. – user1424739 May 30 '20 at 16:13
  • @DebanjanB The link pointed is quite disorganized. I don't it answerd my question. What is the answer to my question? Is the answer "There is no way to make them the same"? – user1424739 May 31 '20 at 15:01

0 Answers0