Scraping etoro with python

Question

I'm trying to use Selenium to automatically connect to my etoro account and get some figures from my portfolio. I work on Google Colab and from now, here is what I have:

from selenium import webdriver

options = webdriver.ChromeOptions()
options.add_argument('--headless')
options.add_argument('--no-sandbox')
options.add_argument('--disable-dev-shm-usage')

wd = webdriver.Chrome(options=options)
wd.get("https://www.etoro.com/fr/login")

username = "username@mail.com"
password = "password"

elementID = wd.find_element_by_css_selector("input[automation-id='login-sts-username-input']")
elementID.send_keys(username)
elementID = wd.find_element_by_id("password")
elementID.send_keys(password)

However, I have this error message

Message: no such element: Unable to locate element: {"method":"css selector","selector":"input[automation-id='login-sts-username-input']"}
  (Session info: headless chrome=91.0.4472.77)

I have tried to change and use find_element_by_class, by_xpath, etc but I couldn't find how to do it.

Could you give me a hand here?

score 1 · Answer 1 · answered Jun 11 '21 at 16:43

1

If you do not need to run with headless mode, removing that argument fixes this. In any case, that is where your problem lies.

I suspect that in headless browser mode we may be encountering some kind of bot detection. If you NEED to run in headless perhaps we can find a way around that.

answered Jun 11 '21 at 16:43

C. Peck

3,641
3
19
36

Couldn't make it work on colab without the "options.add_argument('--headless')". I switched to VScode and used @itronic1990 code and it works. Thank you for your help! – Louis Viallard Jun 12 '21 at 16:53

score 0 · Accepted Answer · answered Jun 11 '21 at 16:37

0

See if this works:-

driver.find_element_by_xpath(".//input[@id='username']")

answered Jun 11 '21 at 16:37

itronic1990

1,231
2
4
18

1

Switched to VScode and works perfectly with your code. Thank you very much! – Louis Viallard Jun 12 '21 at 16:53

score 0 · Answer 3 · edited Jul 10 '22 at 19:29

0

For the sake of completeness, Selenium removed some method in version 4.3.0, so your code shall be modified as:

...
elementID = wd.find_element("xpath",".//*[@id='username']")
...
elementID = wd.find_element("id","password")
...

See AttributeError: 'WebDriver' object has no attribute 'find_element_by_xpath'

edited Jul 10 '22 at 19:29

ChrisGPT was on strike

127,765
105
273
257

answered Jul 09 '22 at 20:19

JulianF

1

Scraping etoro with python

3 Answers3