0

I am writing a program in Java which requires that I extract information from my college's website (menu items to be specific). I can do this perfectly given today's menu. On the webpage, if I want to look at tomorrow's menu it only appends a #menu to the current URL. If I put that specific URL into my program, it still gives me the menu for today. Basically, I need to figure out how to properly alter the URL to get the menu for any meal of the day and any day. I don't have much experience at all with HTML, but here is part of the source code from the website. I appreciate any advice!

</table></div> 
<div id="cs_control_38750" class="cs_control CS_Element_CustomCF">
<div id="CS_CCF_9441_38750">
<link rel="stylesheet" type="text/css" media="screen" href="/living/style/lac_menuitems.css" />

<a name="menu"></a>
<FORM action="#menu" method="POST" name="menuform" id="menuform">
<select name="menudates" id="menudates" onchange="submit();">
    <option value="2015-05-21">05/21/2015</option>
    <option value="2015-05-22">05/22/2015</option>
    <option value="2015-05-23">05/23/2015 ... 
</select>
<select name="menuperiod" id="menuperiod" onchange="submit();">
    <option value="Breakfast">Breakfast</option>
    <option value="Brunch">Brunch</option>
    <option value="Lunch" selected>Lunch</option>
    <option value="Dinner">Dinner</option>
</select>
<select name="menulocations" id="menulocations" onchange="submit();">
....
</select>
miselking
  • 3,043
  • 4
  • 28
  • 39

1 Answers1

0

When you change one of the selection fields, the form submits. Then:

  1. The data is posted to the same url as the form.

  2. The page scrolls to the link named "menu" i.e. the original form.

Unless the server is doing fancy stuff behind the scenes, you can easily replicate this behavior by "submitting" your own form values in your HTTP request to the url that has the form.

You could also open the browser console and watch all the HTTP requests (google how if you don't know). The headers will be shown somewhere obvious along with any other data you may need. On my browser, the url gets logged to the console, and clicking it opens up a dialogue with all of the headers.

I don't have much experience with Java, but it looks like you'll need to do this: Sending HTTP POST Request In Java. The headers should look like this:

menudates: 2015-05-21 menuperiod: Dinner other-options: not-shown

I edited the code on the other question:

HttpClient httpclient = HttpClients.createDefault();
HttpPost httppost = new HttpPost("http://www.url-of-school.com/menu/");

// Request parameters and other properties.
List<NameValuePair> params = new ArrayList<NameValuePair>(2);
params.add(new BasicNameValuePair("menudates", "2015-05-21"));
params.add(new BasicNameValuePair("menuperiod", "Dinner"));
params.add(new BasicNameValuePair("other-options", "not-shown"));
httppost.setEntity(new UrlEncodedFormEntity(params, "UTF-8"));

//Execute and get the response.
HttpResponse response = httpclient.execute(httppost);
HttpEntity entity = response.getEntity();

if (entity != null) {
    InputStream instream = entity.getContent();
    try {
        // do something useful
    } finally {
        instream.close();
    }
}
Community
  • 1
  • 1
sudo rm -rf slash
  • 1,156
  • 2
  • 16
  • 27
  • I appreciate the quick response. My first inclination was to submit my own form values. Sadly, this does not change anything. I'm working on looking through the browser console to see if they have a large database of the menu items from which they pull all of this information. One thing I have noticed is that if I submit my own values to the URL, and then manually change the date on the website, the site takes me to the menu for the values I put into the URL (it overrides the POST form). This is the only way I can get that date. It doesn't work if I only input the values and load. Any ideas? – Andrew Peterson May 27 '15 at 16:31
  • I missed that it's a post request. I'll edit the answer. – sudo rm -rf slash May 27 '15 at 22:41