2

I'm not sure if the title says what I needed. But that's the best way I can put it.

My requirement: I want to check if a series of engine numbers (of vehicles) are registered with the local transportation authority. They have provide a web-interface for this task, but I can only check for a single engine number at once. Usually I need to check over 200 numbers. So, its a humongous task to check each one individually.

A couple of years ago, I have created a small standalone python script to do this automatically using web scraping, but now I want to do this on a server.

A user gives all the numbers in the text file which they upload/paste the contents in to a text field. Then I'll have to submit the form on the transportation website (using web scraping) for each number and display a final status for all of them.

What I want to know is how to do this on a server? What technologies could be helpful. I'm comfortable with Java & JavaScript. I don't know PHP (But i can learn if needed). I don't have slightest Idea how to do this on server side. Any Ideas and Help is greatly appreciated.

Thanks.

DisappointedByUnaccountableMod
  • 6,656
  • 4
  • 18
  • 22
inblueswithu
  • 953
  • 2
  • 18
  • 29

1 Answers1

4

Java has a library called JSoup, which provides a mostly-familiar api that uses css selectors.

And obviously there are built-in functions that can get you the html from a given URL.

Put those together and you've got a server-side scraper

[edit] Your question, on a re-read, isn't just about scraping -- it's about how to automatically submit an html form from within Java to an external server. This is an interesting question, one I've wondered myself.

this may be an answer: How to send post form with java?

Community
  • 1
  • 1
TKoL
  • 13,158
  • 3
  • 39
  • 73