3

My problem is i'm trying to get into scopus using a crawler but it requires my crawler to enter the site through my school proxy server. I tried authenticating but it keep responding with 401 status.

public void testConnection() throws ClientProtocolException, IOException {
    DefaultHttpClient httpclient = new DefaultHttpClient();
    List<String> authpref = new ArrayList<String>();
    authpref.add(AuthPolicy.NTLM);
    httpclient.getParams().setParameter(AuthPNames.TARGET_AUTH_PREF, authpref);
    NTCredentials creds = new NTCredentials("username","password","ezlibproxy1.ntu.edu.sg","ntu.edu.sg");//this is correct
    httpclient.getCredentialsProvider().setCredentials(AuthScope.ANY, creds);

    HttpHost target = new HttpHost("ezlibproxy1.ntu.edu.sg", 443, "https");//this is correct

    // Make sure the same context is used to execute logically related requests
    HttpContext localContext = new BasicHttpContext();

    // Execute a cheap method first. This will trigger NTLM authentication
    HttpGet httpget = new HttpGet("http://www-scopus-com.ezlibproxy1.ntu.edu.sg/authid/detail.url?authorId=14831850700");
    HttpResponse response = httpclient.execute(target, httpget, localContext);
    HttpEntity entity = response.getEntity();
    System.out.println(EntityUtils.toString(entity));

    int statusCode = response.getStatusLine().getStatusCode();
    System.out.println("Status Code:" + statusCode);
}

The status code respond is 401 (unauthorised). Any suggestion on this?

flx
  • 14,146
  • 11
  • 55
  • 70
user3205474
  • 73
  • 1
  • 1
  • 5
  • I am guessing the HttpHost instance you have created should be used as a proxy. See this post - http://stackoverflow.com/questions/6962047/apache-httpclient-4-1-proxy-authentication – Vino Jan 17 '14 at 08:01
  • if you are using httpclient 3 you can use https://github.com/DovAmir/httpclientAuthHelper – dov.amir May 22 '14 at 17:11

0 Answers0