0

I wrote some code which should check whether a product is back in stock and when it is, send me an email to notify me. This works when the things I'm looking for are in the html.

However, sometimes certain objects are loaded through JavaScript. How could I edit my code so that the web scraping also works with JavaScript?

This is my code thus far:

import time
import requests

while True:
    # Get the url of the IKEA page
    url = 'https://www.ikea.com/nl/nl/p/flintan-bureaustoel-vissle-zwart-20336841/'

    # Get the text from that page and put everything in lower cases
    productpage = requests.get(url).text.lower()

    # Set the strings that should be on the page if the product is not available
    outofstockstrings = ['niet beschikbaar voor levering', 'alleen beschikbaar in de winkel']

    # Check whether the strings are in the text of the webpage
    if any(x in productpage for x in outofstockstrings):
        time.sleep(1800)
        continue
    else:
        # send me an email and break the loop
Jem
  • 557
  • 9
  • 28
  • Why don't you test it? SO is not a code-running service. – underscore_d Jun 11 '20 at 14:51
  • it'll be fine, you have it on [sleep](https://stackoverflow.com/questions/17075788/python-is-time-sleepn-cpu-intensive) – user120242 Jun 11 '20 at 14:54
  • I did run it and it gave me no errors. But the problem is that it won't do anything until the product is back in stock. So I would still not know whether it really does what I want it to. – Jem Jun 11 '20 at 14:55
  • 1
    feed it a mock string of the HTML content when it's in stock and see if it breaks out of the loop. Or test it on a few other pages with in-stock items to see if it works. If they change the URL on you, or they redesign the site, there's nothing you can do about it anyways. if you want just add some code to detect missing out of stock messages and have it email you to fix the code – user120242 Jun 11 '20 at 14:56
  • Ahh yes, good recommendation, thanks! – Jem Jun 11 '20 at 14:59

1 Answers1

1

Instead of scraping and analyzing the HTML you could use the inofficial stock API that the IKEA website is using too. That API return JSON data which is way easier to analyze and you’ll also get estimates when the product gets back to stock.

There even is a project written in javascript / node which provides you this kind of information straight from the command line: https://github.com/Ephigenia/ikea-availability-checker

You can easily check the stock amount of the chair in all stores in the Netherlands:

npx ikea-availability-checker stock --country nl 20336841
ephigenia
  • 9
  • 1
  • 2