0

I'm trying to build a web scraper for an NBA stats site for a school project, but for whatever reason it looks like when I convert the html link to a JSoup document the elements aren't loading into the array I've set up to hold the data. Any idea what I'm doing wrong? It's my first run with JSoup.

Code is:

import java.io.File;
import java.io.FileInputStream;
import java.io.IOException;
import java.util.ArrayList;
import org.jsoup.Jsoup;
import org.jsoup.nodes.Document;
import org.jsoup.nodes.Element;
import org.jsoup.select.Elements;
public class Comparison {
public void loadTeams() {
try {
                ArrayList<Team> NBA = new ArrayList<Team>(30);
                String reference = "https://www.nba.com/stats/teams/four-factors";
                Document nba = Jsoup.connect(reference).get();          
                Elements team = nba.getElementsByAttributeValue("field", "TEAM_NAME");
                Elements shooting = nba.getElementsByAttributeValue("field" , "EFG_PCT");
                Elements turnovers = nba.getElementsByAttributeValue("field" , "TM_TOV_PCT");
                Elements rebounds = nba.getElementsByAttributeValue("field" , "OREB_PCT");
                Elements freethrows = nba.getElementsByAttributeValue("field" , "FTA_RATE");
for (int i = 1; i < team.size(); i++) {
                    String name = team.get(i).text();
double shootingScore = Double.parseDouble(shooting.get(i).text());
double turnoversScore = Double.parseDouble(turnovers.get(i).text());    
double reboundsScore = Double.parseDouble(rebounds.get(i).text());  
double freethrowsScore = Double.parseDouble(freethrows.get(i).text());  
NBA.add(new Team(name, shootingScore, turnoversScore, reboundsScore, freethrowsScore));
                }
//for testing//
for(int i=0; i<NBA.size(); i++) {
                System.out.print(NBA.getClass());
            }
        }
catch (IOException e)
        {
            System.out.println(e.getMessage());
        }
    }
}

Any advice would be great. Thanks!

ReginaldRey
  • 49
  • 1
  • I looked at the page's source HTML. The data you are looking for is not contained in it. JSoup can work on the HTML but it cannot work on the JavaScript. It's not going to be possible with JSoup. Looking at the page load requests, I narrowed the creation of the table to https://www.nba.com/_next/static/chunks/6406.2ccdccd6e27a5966.js but I cannot make out where the data actually comes from. – ProgrammersBlock Feb 17 '23 at 16:13
  • 1
    Your question is likely a duplicate of [Page content is loaded with JavaScript and Jsoup doesn't see it](https://stackoverflow.com/questions/7488872/page-content-is-loaded-with-javascript-and-jsoup-doesnt-see-it) – ProgrammersBlock Feb 17 '23 at 16:13

0 Answers0