Getting desired information from website using jsoup

Question

I recently read a nice example of using jsoup, posted by BalusC.

His code:

import org.jsoup.Jsoup;
import org.jsoup.nodes.Document;
import org.jsoup.nodes.Element;
import org.jsoup.select.Elements;

public class test{

    public static void main(String[] args) throws Exception {
        String url = "https://stackoverflow.com/questions/2835505";
        Document document = Jsoup.connect(url).get();

        String question = document.select("#question .post-text").text();
        System.out.println("Question: " + question);

        Elements answerers = document.select("#answers .user-details a");
        for (Element answerer : answerers) {
            System.out.println("Answerer: " + answerer.text());
        }
    }
}

The code works like a charm, and appears to be very straightforward, however I dont know how to modify it, so that it would get the cheapest tyre prices from here. How would i do that?

Seems like the class of the price tag is "lst-product-item-price".. So get all of them with the select and parse the number? :) — Icewind, May 22 '14 at 14:22
Can you tell me, where you got the `.lst-product-item-price` ? — Kumara, May 22 '14 at 14:32
Just inspect the html source of the site. In Firefox you can right click the price and click on "Inspect element" to get the source of the selected element. — Icewind, May 22 '14 at 14:34

score 2 · Answer 1 · answered May 22 '14 at 14:26

2

Try:

import org.jsoup.Jsoup;
import org.jsoup.nodes.Document;
import org.jsoup.nodes.Element;
import org.jsoup.select.Elements;

public class test{

    public static void main(String[] args) throws Exception {
        String url = "http://www.mimovrste.com/letne-avtomobilske-pnevmatike?p-min=31.65&p-max=348.35&o=_price";
        Document document = Jsoup.connect(url).get();

        Elements prices = document.select(".lst-product-item-price");
        for (Element price : prices) {
            System.out.println("Price: " + price.text());
        }
    }
}

answered May 22 '14 at 14:26

Mr. P

1,387
1
11
25

You sir, are a life saver. Can you tell me, where you got the `.lst-product-item-price` ? – Kumara May 22 '14 at 14:28
1

You must inspect the webpage you want and find the ID or CLASS attribute of desired html tag. In Chrome just enter a webpage and press Ctrl+Shift+C and then select desired value on the webpage. Then on the most bottom bar you will see the whole path to this particular element. Just copy the last part of it. For more you can read at http://www.w3schools.com/css/css_selectors.asp – Mr. P May 22 '14 at 14:34

score 1 · Accepted Answer · answered May 22 '14 at 18:01

I suppose there is more to this question. Pisek's logic is correct the class lst-product-item-price gives the list of all prices, but it gives only the list in one page. In this case we have 27 pages. Also it has different sorting order. So you might have to navigate through all the pages and then find the cheapest.

However, I was wondering why do we need a parsing to the html here? The URL in question is http://www.mimovrste.com/letne-avtomobilske-pnevmatike?p-min=31.65&p-max=348.35&o=_price. This itself says the minimum and maximum price. So the result could be parsed right away from the URL string.

Suppose you need a the result from an unfiltered list, that is you need to find the cheapest of all without any min or max limit. Say you want to get the limit by just typing http://www.mimovrste.com/letne-avtomobilske-pnevmatike, then there are two tricks to achieve this.

First (Simplest)

Notice the menu on the right side which says "Cena" meaning "Price". That field get defaulted to the minimum and maximum. And this is applicable even if you use previous url. If you inspect the html it shows the value is stored in a hidden field and a text field with the name "p-min". So select an input that has a name attribute as name="p-min" and that is your cheapest value. The advantage of this approach is it give a direct decimal value and conversiont to double is straight forward.

cheapest = doc.select("input[name=p-min]").first().attr("value");

Second

Notice the url. The sorting order is named with 'o'. So if you say o=price in url it sorts the list in ascending order based on price and if you give it as o=_price it give descending order of prices. So pass in a parameter "o" with value "_price" and as Pisek said select the value of first lst-product-item-price will give you the cheapest. This will give a String value with comma and € symbol.

doc = Jsoup.connect(URL).data("0", "_price").get();
cheapest = doc.select("b.lst-product-item-price").first().text();

Put it all together

import java.io.IOException;

import org.jsoup.Jsoup;
import org.jsoup.nodes.Document;


public class TyrePrice {

    public static void main(String[] args) {
        try {
            String URL = "http://www.mimovrste.com/letne-avtomobilske-pnevmatike";
            Document doc = Jsoup.connect(URL).get();

            // Approach 1
            String cheapest = doc.select("input[name=p-min]").first().attr("value");
            System.out.println(cheapest); // Prints 31.65

            // Approach 2
            doc = Jsoup.connect(URL).data("0", "_price").get();
            cheapest = doc.select("b.lst-product-item-price").first().text();
            System.out.println(cheapest); // Prints 31,65 €

        } catch (IOException e) {
            e.printStackTrace();
        }
    }
}

Hope this helps. :)

Getting desired information from website using jsoup

2 Answers2