0

i'm trying to scrape some data from a site called laced.co.uk, and i'm a tad confused on whats going wrong. i'm new to this, so try and explain it simply (if possible please!). Here is my code ;

from bs4 import BeautifulSoup
import requests

url = "https://www.laced.co.uk/products/nike-dunk-low-retro-black-white?size=7"

result = requests.get(url)

doc = BeautifulSoup(result.text, "html.parser")

prices = doc.find_all(text=" £195 ")
print(prices)

thank you! (the price at time of posting was 195 (it showed as the size 7 buy now price on the page).

TobyWR
  • 3
  • 1

1 Answers1

0

The price is loaded within a <script> tag on the page:

<script>
    typeof(dataLayer) != "undefined" && dataLayer.push({
      'event': 'eec.productDetailImpression',
      'page': {
        'ecomm_prodid': 'DD1391-100'
      },
      'ecommerce': {
        'detail': {
          'actionField': {'list': 'Product Page'},
          'products': [{
            'name': 'Nike Dunk Low Retro Black White',
            'id': 'DD1391-100',
            'price': '195.0',
            'brand': 'Nike',
            'category': 'Dunk, Dunk Low, Mens Nike Dunks',
            'variant': 'White',
            'list': 'Product Page',
            'dimension1': '195.0',
            'dimension2': '7',
            'dimension3': '190',
            'dimension4': '332'
          }]
        }
      }
    });
  </script>

You can use a regular expression pattern to search for the price. Note, there's no need for BeautifulSoup:

import re
import requests

url = "https://www.laced.co.uk/products/nike-dunk-low-retro-black-white?size=7"
result = requests.get(url)

price = re.search(r"'price': '(.*?)',", result.text).group(1)
print(f"£ {price}")
MendelG
  • 14,885
  • 4
  • 25
  • 52
  • wow. Thats crazy. Could you explain this bit to me please if you have enough time? re.search(r"'price': '(.*?)',", result.text).group(1) and (f"£ {price}") Really helps a lot that i dont need beautiful soup! – TobyWR Feb 07 '22 at 22:08
  • @TobyWR This is called regular expressions. It searches for the text "price" and then the digits after that – MendelG Feb 07 '22 at 22:11
  • 1
    Ahhhhhh right that makes so much sense. so it searches through the source HMTL of the page for a certain text, then the digits next to it. I need to start making notes of all of this stuff. insanely useful. im gonna try it on more products in a minute. Cheers for the help! – TobyWR Feb 07 '22 at 22:13
  • @TobyWR See [What does this regex mean?](https://stackoverflow.com/questions/22937618/reference-what-does-this-regex-mean/22944075#22944075) – MendelG Feb 07 '22 at 22:14