Parsing JSON and sorting by outputted dates in Java

Question

This REST query gives among other stuff, the created_at dates. I want to parse the output to get all the dates and then sort them. The lines in question i need to parse look like

  "created_at": "2013-01-08T20:11:48Z",

Query here

https://api.github.com/search/repositories?q=language:Java

Is there an efficient way of doing this?

My code sofar

 public static String processRestResponse(String language){
    HttpClient client = new DefaultHttpClient();
    HttpGet request = new HttpGet("https://api.github.com/search/repositories?q=language:" + language);
    HttpResponse response = null;
    BufferedReader rd = null;
    try {
        response = client.execute(request);
        rd = new BufferedReader
            (new InputStreamReader(
                    response.getEntity().getContent()));
    } catch (IOException e) {
        e.printStackTrace();
    }
    String line = "";

    String output  = language;
    try {
        while ((line = rd.readLine()) != null) {
            output += "\n" + line.substring(line.indexOf("\""), line.indexOf(",")) + "\n";
        }
    }catch (IOException e){}

    return output;
}

what library are you using to make the HTTP calls? Can you post your code which gets the repositories(but not yet sorted) ? — Anoop, Jun 01 '17 at 20:28
Use a proper JSON parsing library for this. Either javax.json, or any of the others linked to from http://json.org/. — ziesemer, Jun 01 '17 at 20:33
Please also remove the javax.swing / jframe components here. They are not related to your current efforts... — ziesemer, Jun 01 '17 at 20:36

score 0 · Answer 1 · answered Jun 06 '17 at 00:16

All right, I'll bite:

import java.io.InputStream;
import java.net.HttpURLConnection;
import java.net.URL;
import java.net.URLEncoder;
import java.util.Arrays;
import java.util.Comparator;
import java.util.List;

import javax.json.Json;
import javax.json.JsonArray;
import javax.json.JsonObject;
import javax.json.JsonStructure;
import javax.json.JsonValue;

public class StackOverflow44316515{

    public static void main(final String[] args) throws Exception{
        final StackOverflow44316515 example = new StackOverflow44316515();

        // Get data:
        final JsonObject jsonLangObjs = example.getLanguages("java");

        // Copy items into a mutable array:
        final JsonArray items = jsonLangObjs.getJsonArray("items");
        final JsonObject[] itemsArray = items.toArray(new JsonObject[items.size()]);

        // Sort the array:
        Arrays.sort(itemsArray, new Comparator<JsonValue>(){
            @Override
            public int compare(final JsonValue o1, final JsonValue o2){
                return ((JsonObject)o1).getString("created_at")
                    .compareTo(((JsonObject)o2).getString("created_at"));
            }

        });

        // Re-wrap in a list API for convenience (optional):
        final List<JsonObject> itemsArrayList = Arrays.asList(itemsArray);

        // Print sample:
        for(final JsonObject jo : itemsArrayList){
            System.out.println(jo.getString("created_at") + " - " + jo.getString("full_name"));
        }
    }

    public JsonObject getLanguages(final String language) throws Exception{
        final URL url = new URL("https://api.github.com/search/repositories?q=language:"
            + URLEncoder.encode(language, "UTF-8")
            + "&per_page=10");

        final HttpURLConnection huc = (HttpURLConnection)url.openConnection();
        if(huc.getResponseCode() != 200){
            throw new Exception("Error calling web service: "
                +  huc.getResponseCode() + " - " + huc.getResponseMessage());
        }

        try(final InputStream is = huc.getInputStream()){
            final JsonStructure json = Json.createReader(is).read();
            return (JsonObject)json;
        }
    }

}

Output:

2010-02-08T13:20:56Z - elastic/elasticsearch
2010-09-06T21:39:43Z - square/retrofit
2012-07-23T13:42:55Z - square/okhttp
2013-01-08T20:11:48Z - ReactiveX/RxJava
2013-03-05T08:18:59Z - JakeWharton/butterknife
2013-07-08T22:52:33Z - bumptech/glide
2014-04-25T14:29:47Z - PhilJay/MPAndroidChart
2014-05-29T16:23:17Z - google/guava
2014-08-09T16:45:18Z - iluwatar/java-design-patterns
2015-04-29T23:54:16Z - square/leakcanary

ISO-8601-formatted dates are fortunately properly sorted by their lexicographical ordering - so for a simple exercise as this, we don't even need to first parse them to Java data objects or such.

See also: https://xkcd.com/1179/

Make note of GitHub's server-side pagination here - necessary to not flood you with all 3.2+ million results at once. I limited the default of 30 results to 10 for this example. If you truly need to analyze ALL of the results, I'll leave this as an exercise to the reader. https://developer.github.com/v3/guides/traversing-with-pagination/ has some additional details around this.

This sample code assumes that the created_at value always exists for all records. Unless this is an API guarantee by GitHub (and even if), additional checks should be added here for production code.

Note the URL-encoding of the language argument, to help prevent against injection attacks (assuming the "language" parameter is not trusted).

Note the use of Java's built-in java.net classes - no need for a 3rd-party HTTP library here.

Note that this code doesn't catch any exceptions it can't handle. An empty catch block, or even a catch block that only outputs the exception is almost always of no benefit to anyone. ("OK, now what?") I.E., do you really want the code to proceed on as if nothing bad happened?

This code also builds each set of JSON results into an in-memory object model (JsonArray of JsonObjects). While the simplest for the example here, a more efficient implementation would make use of the streaming interfaces, e.g. JsonParser. See also:

Parsing JSON and sorting by outputted dates in Java

1 Answers1