0

I want to crawl the apartment list from this website: String website = http://en.midland.com.hk/find-property/

Here is the code I tried to use:

String urlToPropertyList= website +"data/search_list_results";
        Connection.Response loginForm = Jsoup
            .connect(website)
            .method(Connection.Method.GET).execute();
    Document doc = Jsoup
            .connect(urlToPropertyList)
            .data("areaFrom", "0")
            .data("areaTo", "null")
            .data("area_type", "net_area")
            .data("autocompleteString", "")
            .data("bedroom", "")
            .data("bldgIds", "")
            .data("districtIds", "")
            .data("estIds", "")
            .data("estate_name", "")
            .data("feature", "")
            .data("is_hos", "false")
            .data("is_random", "false")
            .data("latLngBounds",
                    "22.409949,113.848296,22.531141,114.374267")
            .data("page", "2").data("priceFrom", "0")
            .data("priceTo", "null").data("sort", "").data("tx_type", "L")
            .data("zoomLevel", "11").cookies(loginForm.cookies())
            .timeout(0).post();

This is the error that I am getting:

org.jsoup.HttpStatusException: HTTP error fetching URL. Status=415, URL=http://en.midland.com.hk/find-property/data/search_list_results at org.jsoup.helper.HttpConnection$Response.execute(HttpConnection.java:537) at org.jsoup.helper.HttpConnection$Response.execute(HttpConnection.java:493) at org.jsoup.helper.HttpConnection.execute(HttpConnection.java:205) at org.jsoup.helper.HttpConnection.post(HttpConnection.java:200) at crawlers.MidlandListCrawler.crawl(MidlandListCrawler.java:37) at main.MidlandRealtyCrawlController.start(MidlandRealtyCrawlController.java:13) at main.CrawlingController$2.run(CrawlingController.java:32)

Request payload, as copied from the browser (inspect element ->network -> headers) :

{  
   "estate_name":"",
   "priceFrom":0,
   "priceTo":null,
   "areaFrom":null,
   "areaTo":null,
   "bedroom":"",
   "tx_type":"L",
   "area_type":"net_area",
   "is_hos":false,
   "autocompleteString":"",
   "districtIds":"",
   "estIds":"",
   "latLngBounds":"22.40614,113.848296,22.531141,114.374267",
   "page":1,
   "sort":"",
   "bldgIds":"",
   "zoomLevel":11,
   "feature":"",
   "is_random":false
}

Can someone help me please, how can I get the list of properties please? Thanks in advance

Alkis Kalogeris
  • 17,044
  • 15
  • 59
  • 113
Saurabh J
  • 123
  • 1
  • 8

0 Answers0