Can i do anything to make my webscraper look better?

Question

I got my desired output, but it is all over the place, how do I make it cleaner?

import requests
from bs4 import BeautifulSoup

page = requests.get('https://www.titan.fitness/strength/dumbbells/rubber-coated-hex/rubber-hex-dumbbells-with-cast-iron-handle/HEXDBB-GROUP.html')
soup = BeautifulSoup(page.text, 'html.parser')

dumbbells_availability = soup.find(class_="set-items bundle-items container")
item = dumbbells_availability.find_all(class_="row product-detail set-item")
dumbbell1 = item[0]

availability = dumbbell1.find(class_="availability").get_text()
product_name = dumbbell1.find(class_="product-name").get_text()
print(availability)
print(product_name)

Here's the result:

Does this answer your question? [How do I trim whitespace from a string?](https://stackoverflow.com/questions/761804/how-do-i-trim-whitespace-from-a-string) — Pranav Hosangadi, Feb 19 '21 at 22:20

score 1 · Answer 1 · answered Feb 19 '21 at 22:49

Yes, you can.

Just use the getText() method with the strip argument set to True.

For example:

import requests
from bs4 import BeautifulSoup

page = requests.get('https://www.titan.fitness/strength/dumbbells/rubber-coated-hex/rubber-hex-dumbbells-with-cast-iron-handle/HEXDBB-GROUP.html')
soup = BeautifulSoup(page.text, 'html.parser')

dumbbells_availability = soup.find(class_="set-items bundle-items container")
item = dumbbells_availability.find_all(class_="row product-detail set-item")
dumbbell1 = item[0]

availability = dumbbell1.find(class_="availability").getText(strip=True)
product_name = dumbbell1.find(class_="product-name").getText(strip=True)
print(availability)
print(product_name)

Output:

availability:Out of Stock.Notify me when this is in stockSign up
5 LB Rubber Hex Dumbbells

it is possible to write an if then statement? if "out of stock" write something? — , Feb 19 '21 at 23:00

Can i do anything to make my webscraper look better?

1 Answers1