As a step in auto-filling some fields in a flow I need to scrape a URL that has its interesting content filled dynamically by JavaScript.
Based on a cars license plates I can look up information on a government website, set an option and a value and submit.
The resulting page will hold the desired content in different tabs so I'll also have to navigate those. (Sorry cant give direct link to this but selecting "Registreringsnummer", using "YN28579" as the value and pressing "Søg" will take you to the page.)
I've done this with a WebViewClient in another Activity so it is possible to navigate the website directly on the android phone/tablet. But in this other Activity I don't want to display the resulting page just scrape some of the data items.
Here is how I've done it with the WebViewClient:
public void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.webview);
((Button)findViewById(R.id.btnBack)).setVisibility(View.VISIBLE);
wv = (WebView) findViewById(R.id.wv);
reg = getIntent().getStringExtra("reg");
wv.getSettings().setJavaScriptEnabled(true);
wv.setWebViewClient(new WebViewClient() {
private ProgressDialog pd;
private int count = 0;
@Override
public void onPageFinished(WebView view, String url) {
if (count==1) {
view.loadUrl("javascript:document.getElementById('regnr').checked=true;"
+"document.getElementById('soegeord').value='"+reg+"';"
+"document.getElementById('searchForm').submit();"
+"DMR.WaitForLoad.on();");
} else if (count>=2) {
view.loadUrl("javascript:document.body.innerHTML " +
"= '<div class=\"tabNav\">'+document.getElementsByClassName('tabNav')[0].innerHTML+'</div>';" +
"document.getElementsByClassName('h-tab-content')[0].style.width='320px';" +
"document.getElementsByClassName('h-tab-btns')[0].style.width='320px';" +
"document.getElementsByClassName('h-tab-btns')[0].style.height='45px';" +
"document.getElementsByTagName('ul')[0].style.display='inline';" +
"document.head.appendChild='<meta name=\"viewport\" content=\"width=device-width\">';" +
"document.body.style.minWidth ='300px';");
if (pd!=null) {
pd.dismiss();
}
view.setVisibility(View.VISIBLE);
}
}
@Override
public void onPageStarted(WebView view, String url, Bitmap favicon) {
if (pd==null || !pd.isShowing()) {
pd = ProgressDialog.show(SkatActivity.this, "cars.dk", "Vent venligst...", true, false);
}
count++;
}
@Override
public boolean shouldOverrideUrlLoading(WebView view, String url) {
view.loadUrl(url);
return true;
}
});
wv.loadUrl("https://motorregister.skat.dk/dmr-front/appmanager/skat/dmr?_nfpb=true&_nfpb=true&_pageLabel=vis_koeretoej_side&_nfls=false");
}
I can't seem to figure out how to incorporate this in Do you set the WebView as invisible? Found this question that seems to be working along those same lines, but I can't figure out how to get the working WebViewClient hidden but usable inside this other activity?
