I want to crawl the apartment list from this website: String website = http://en.midland.com.hk/find-property/
Here is the code I tried to use:
String urlToPropertyList= website +"data/search_list_results";
Connection.Response loginForm = Jsoup
.connect(website)
.method(Connection.Method.GET).execute();
Document doc = Jsoup
.connect(urlToPropertyList)
.data("areaFrom", "0")
.data("areaTo", "null")
.data("area_type", "net_area")
.data("autocompleteString", "")
.data("bedroom", "")
.data("bldgIds", "")
.data("districtIds", "")
.data("estIds", "")
.data("estate_name", "")
.data("feature", "")
.data("is_hos", "false")
.data("is_random", "false")
.data("latLngBounds",
"22.409949,113.848296,22.531141,114.374267")
.data("page", "2").data("priceFrom", "0")
.data("priceTo", "null").data("sort", "").data("tx_type", "L")
.data("zoomLevel", "11").cookies(loginForm.cookies())
.timeout(0).post();
This is the error that I am getting:
org.jsoup.HttpStatusException: HTTP error fetching URL. Status=415, URL=http://en.midland.com.hk/find-property/data/search_list_results at org.jsoup.helper.HttpConnection$Response.execute(HttpConnection.java:537) at org.jsoup.helper.HttpConnection$Response.execute(HttpConnection.java:493) at org.jsoup.helper.HttpConnection.execute(HttpConnection.java:205) at org.jsoup.helper.HttpConnection.post(HttpConnection.java:200) at crawlers.MidlandListCrawler.crawl(MidlandListCrawler.java:37) at main.MidlandRealtyCrawlController.start(MidlandRealtyCrawlController.java:13) at main.CrawlingController$2.run(CrawlingController.java:32)
Request payload, as copied from the browser (inspect element ->network -> headers) :
{
"estate_name":"",
"priceFrom":0,
"priceTo":null,
"areaFrom":null,
"areaTo":null,
"bedroom":"",
"tx_type":"L",
"area_type":"net_area",
"is_hos":false,
"autocompleteString":"",
"districtIds":"",
"estIds":"",
"latLngBounds":"22.40614,113.848296,22.531141,114.374267",
"page":1,
"sort":"",
"bldgIds":"",
"zoomLevel":11,
"feature":"",
"is_random":false
}
Can someone help me please, how can I get the list of properties please? Thanks in advance