0

trying to get the conetnt of site when calling URL.
url=https://seekingalpha.com/symbol/AAPL/dividends/scorecard
in browser - it is working !
from java code i get:
Your current browser configuration
is not compatible with this site.

my code:

package com;

import java.util.*;
import java.util.regex.Matcher;
import java.util.regex.Pattern;
import java.io.*;
import java.net.MalformedURLException;
import java.net.URL;
import java.net.URLConnection;
import java.nio.charset.Charset;

public class Div4u {

public static String getURLToString(String strUrl) throws IOException {



   String content = "";

   URLConnection connection = new URL(strUrl).openConnection();
   //connection.setRequestProperty("User-Agent", "Mozilla/5.0");
   connection.setRequestProperty("User-Agent", "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/73.0.3683.103 Safari/537.36 OPR/60.0.3255.170");
                                                 
   //connection.setRequestProperty("User-Agent", "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.11 (KHTML, like Gecko) Chrome/23.0.1271.95 Safari/537.11");
   connection.connect();
   BufferedReader br = new BufferedReader(new InputStreamReader(connection.getInputStream(), Charset.forName("UTF-8")));

   String inputLine;
   while ((inputLine = br.readLine()) != null) {
       content += inputLine;
       System.out.println(inputLine);
       
   }

   br.close();
   
   System.out.println("content:" + content);
   return content;
}
  

// main method
    public static void main(String[] args)
        throws NumberFormatException,
               IOException
    {

        try {
            String urlContenet = getURLToString("https://seekingalpha.com/symbol/AAPL/dividends/scorecard");
        } catch (Exception e) {
            e.printStackTrace();
        }   
    }

}
Div4u
  • 1
  • 3
    Content of that site is generated dynamically via JavaScript. If you disable JS in your browser and reload this page you should see `Javascript is Disabled Your current browser configuration is not compatible with this site.`. You will need to use other tool like Selenium web-driver if you want to see page generated dynamically. – Pshemo Jan 04 '22 at 19:30
  • Possibly related: [Page content is loaded with JavaScript and Jsoup doesn't see it](https://stackoverflow.com/q/7488872) – Pshemo Jan 04 '22 at 19:35
  • so ... is there a free package to embed a browser in Java? – Div4u Jan 05 '22 at 18:39
  • Sorry, I am not expert on this field and never had need to face such problem, but form what I heard people usually use tools like Selenium web driver (like I mentioned in my first comment). It should be free (from what I remember). Have you tried it? – Pshemo Jan 05 '22 at 19:04
  • 1
    thanks. manage to solve it with Selenium. though it is not perfect ... sometimes i get a message that i'm a robot :-) – Div4u Jan 07 '22 at 15:27
  • @Heitor do you have simple solution in java ? without using Selenium web driver ? – Div4u Mar 09 '22 at 21:35

0 Answers0