0
def main(url):
   
    # content of URL
    r = requests.get(url)
 
    # Parse HTML Code
    soup = BeautifulSoup(r.text, 'html.parser')
 
    print(soup.find_all('h1')
)

This code gets the html from a website using the Beautiful Soup Library & narrows it down. It returns:

[<h1 class="card-title pricing-card-title" id="ticker">Loading...</h1>]

When the website is opened in a browser, it has a random Ticker Symbol located in that line of html. However, when code is used to get the html, it has the word "Loading..." instead.

How do I use code to extract the randomly generated ticker symbol when opening the site?

I was expecting to get a random ticker symbol as one does when the website is opened normally. The website I am attempting to scrap is url = "https://www.rayberger.org/random-stock-picker/"

Sprinkled Difficulty
  • 11
  • 2
  • BS4 is not your tool. It is going to get a version of the site that one would get doing a view source on the page. i.e. one where the javascript has not been run. You need somehting more like `selenium` – JonSG Aug 29 '23 at 18:58

1 Answers1

1

The page loads the symbol via JavaScript from external URL (in fact, loads the whole array of symbols and then uses javascript to print random symbol), so beautifulsoup doesn't see the ticker symbol.

You can simulate this with this example:

import random

import requests

url = "https://www.rayberger.org/random-stock-picker/stocks.json"
data = requests.get(url).json()
print(random.choice(data))

Prints (for example):

FVE
Andrej Kesely
  • 168,389
  • 15
  • 48
  • 91