Python Web Scraping: Unable to Log into https://www.vet-ebooks.com/ for Downloading Books

Question

I have been trying to iterate through a website book listing using python. The website is https://www.vet-ebooks.com/. Issue is i need to perform a login before download the book. but login using the python is not working. Initial code that i have written is:

import requests

# Start a session
session = requests.Session()

# Define the login data
login_payload = {
    'log': 'test',
    'pwd': 'test',
    'rememberme': 'forever',
    'ihcaction':'login',
    'ihc_login_nonce': '52af5a580a'
}

# Send a POST request to the login page
login_req = session.post('https://www.vet-ebooks.com/user-login/', data=login_payload)

# Check if login was successful
if login_req.status_code == 200:
    print("Login successful")
else:
    print("Login failed")

However, it shows that the Login was successful but it does not worked. I had switched my approach to Selenium.

# Selenium 4
from selenium import webdriver

chrome_options = webdriver.ChromeOptions()
chrome_options.set_capability('browserless:token', 'TOKENHERE')
chrome_options.add_argument("--no-sandbox")
chrome_options.add_argument("--headless")

driver = webdriver.Remote(
    command_executor='https://chrome.browserless.io/webdriver',
    options=chrome_options
)

driver.get("https://www.vet-ebooks.com/user-login/")

# replace with your username and password
username = "testusername"
password = "testpasshere"

wait = WebDriverWait(driver, 10)

# again, get the login page and set the cookies
driver.get('https://www.vet-ebooks.com/user-login/')

# find username field and enter username
username_field = wait.until(EC.presence_of_element_located((By.NAME, "log")))
username_field.clear()
username_field.send_keys(username)

# find password field and enter password
password_field = wait.until(EC.presence_of_element_located((By.NAME, "pwd")))
password_field.clear()
password_field.send_keys(password)

# find login button and click it
login_button = wait.until(EC.presence_of_element_located((By.NAME, "Submit")))
login_button.click()

# Print the redirected URL
redirected_url = driver.current_url
print("Redirected URL:", redirected_url)

driver.quit()

But this code does not work either, redirect_url remains the same at /user-login/ please can anyone suggest me how this issue can be resolved?

score 0 · Answer 1 · answered Jun 04 '23 at 11:51

I see you’re running Selenium in headless mode. Have you tried non-headless mode to ensure the elements you believe you’re catching are actually catching?

I believe that some dynamically loading web pages can have elements that shift (in name) ever so slightly over time.

score 0 · Answer 2 · answered Jun 04 '23 at 12:39

To login within the website website using a valid set of credentials you need to induce WebDriverWait for the element_to_be_clickable() and you can use the following locator strategies:

Using CSS_SELECTOR:

WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.CSS_SELECTOR, 'input#iump_login_username'))).send_keys(username)
driver.find_element(By.CSS_SELECTOR, "input#iump_login_password").send_keys(password)
driver.find_element(By.CSS_SELECTOR, "input[name=Submit]").click()

Note: You have to add the following imports :

from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

Python Web Scraping: Unable to Log into https://www.vet-ebooks.com/ for Downloading Books

2 Answers2