Jsoup, parse html loaded ajax table

Question

I have a problem with the table parsing, which is loaded with ajax:

Document doc = Jsoup.connect("http://lfl.ru/club553").get();

That's what i get:

<div class="tournament_stats_table_tournament_3442 tournament-stats-table" style="display: block;" url="/?ajax=1&amp;method=tournament_stats_table&amp;tournament_id=3442&amp;club_id=553">
                        подождите...                    </div>

Tell me please what can be done in this situation. Thx.

Possible duplicate of [Page content is loaded with javascript and Jsoup doesn't see it](https://stackoverflow.com/questions/7488872/page-content-is-loaded-with-javascript-and-jsoup-doesnt-see-it) — luksch, Aug 28 '17 at 10:08

score 0 · Answer 1 · answered Aug 28 '17 at 07:59

0

You won't be able to get data using server->server request. When you do your request on server JS won't be executed and therefore table is simply not available.

As alternative think about these:

If you are the owner of website you parse - try to avoid ajax if possible;
Check what is endpoint of ajax request and simply parse it instead of webpage;

answered Aug 28 '17 at 07:59

Dmytro Pastovenskyi

5,240
5
37
56

thanks for answer. I'm not the site owner, I need to automate the process of updating this table on my site. – Ivan Vishnyak Aug 28 '17 at 08:12
1

@ivanVishnyak check ajax request and parse it's result. – Dmytro Pastovenskyi Aug 28 '17 at 08:15

score 0 · Answer 2 · edited Dec 09 '17 at 03:49

First: you need get Cookies ,just using

 Connection.Response = Jsoup.connect.method(get).excute

Then: make request with cookies and headers

document=Jsoup.connect.header.data.post

e.g.

Connection.Response loginForm=Jsoup.connect("http://www.a5.cn").
        method(Connection.Method.GET).execute();

Document document=Jsoup.connect("http://www.a5.cn/login.html").
        data("formhash","97bfbf").data("hdn_refer","http://www.a5.cn/")
        data("account","userID").data("autoLogin","1").data("password","your password").
        cookies(loginForm.cookies()).header("Accept","application/json, text/javascript, */*; q=0.01").header("X-Requested-With","XMLHttpRequest").post();

System.out.println(document.body().text());

Jsoup, parse html loaded ajax table

2 Answers2