using jsoup getting content from url

Question

i want to get the entire content of url (i.e) till the end of that url.. but after getting few contents because of partial loading ..i cant able to get remaining contents...is there is any way to get whole content from url even after the partial loading ...

         String url = "URL/"; // getting URL
          try {
         Document doc = Jsoup.connect(url).get();
         FileWriter fw=null;
         BufferedWriter bw=null;
        fw=new FileWriter("D:\\url.txt");
        bw=new BufferedWriter(fw);
        String line = doc.text();
         System.out.println(line);
            bw.write(line);
        } catch (IOException ex) {
       Logger.getLogger(NewJFrame.class.getName())
       .log(Level.SEVERE, null,  ex);
    }

@MineRockers do share your views even if OP doesn't accepts that as answer — Gaur93, Oct 25 '16 at 09:38
I'm sorry, I'm not very good at English, perhaps you can clarify what you said @Gaur93 Did you mean that even if the OP didn't accept no jsoup answer I will still post an answer? — Mine Rockers, Oct 25 '16 at 09:39
url like...https://news.google.co.in/...here after a few post ..there is a button called More Health stories...for further news..by coding i want to get the news even after that button using coding — surya, Oct 25 '16 at 09:54
After a first glance at the request, I think a headless browser with JavaScript support will be the better choice, maybe parse with jsoup afterwards. Try HtmlUnit, though I prefer PhantomJS. — Frederic Klein, Oct 25 '16 at 10:23

score 0 · Answer 1 · edited May 23 '17 at 12:31

0

Sorry for the late answer, dinner...

An answer can be found here: Read url to string in few lines of java code

Idk if it's your question is a duplicate question or not...

Anyways, the traditional way to do it would be like this:

URL website = new URL("example.com");
        URLConnection connection = website.openConnection();
        BufferedReader in = new BufferedReader(
                                new InputStreamReader(
                                    connection.getInputStream()));

        StringBuilder response = new StringBuilder();
        String inputLine;

        while ((inputLine = in.readLine()) != null) 
            response.append(inputLine);

        in.close();

Hope I helped!

edited May 23 '17 at 12:31

Community

1
1

answered Oct 25 '16 at 10:29

Mine Rockers

213
3
12

do i want to import anything for that – surya Oct 25 '16 at 10:45
Exception in thread "main" java.lang.UnsupportedOperationException: Not supported yet. – surya Oct 25 '16 at 10:51
i used this coding – surya Oct 25 '16 at 11:04

score 0 · Answer 2 · answered Oct 25 '16 at 11:05

    URL website;
    try {
        website = new URL("https://news.google.co.in/");

    URLConnection connection = website.openConnection();
    BufferedReader in = new BufferedReader(
                            new InputStreamReader(
                                connection.getInputStream()));

    StringBuilder response = new StringBuilder();
    String inputLine;

    while ((inputLine = in.readLine()) != null) 
        response.append(inputLine);

    in.close();
        System.out.print(response.toString());
  } catch (MalformedURLException ex) {
                                                                                    Logger.getLogger(NewJFrame.class.getName()).log(Level.SEVERE, null, ex);
    } catch (IOException ex) {
    Logger.getLogger(NewJFrame.class.getName()).log(Level.SEVERE, null, ex);
    }

and getting an html output

using jsoup getting content from url

2 Answers2