Fetching Starred repositories from Github API in Spring

Question

I'm trying to fetch the repositories which a particular user has starred. For this, I can use GET /users/:username/starred endpoint of Github's REST API. It runs perfectly fine when I run for any username using some REST Client or either by curl.

But the problem occurs when I try to use Spring and do this. I will attach the code here.

@CrossOrigin("http://localhost:4200")
@RequestMapping(method = RequestMethod.GET, value = "/getstarsperlang/{username}")
public @ResponseBody ArrayList<Object> getStarsPerLang(@PathVariable String username) {

    ArrayList<Object> output = new ArrayList<>();
    ArrayList<String> languages = new ArrayList<>();
    ArrayList<Integer> star_counts = new ArrayList<>();

    String URL = "https://api.github.com/users/" + username + "/starred?client_id=" + Configuration.client_id
            + "&client_secret=" + Configuration.client_secret;

    restTemplate = new RestTemplate();
    headers = new HttpHeaders();
    headers.set("User-Agent", "profile-analyzer");
    HttpEntity<String> entity = new HttpEntity<>("parameters", headers);

    ResponseEntity<Repo[]> repos = restTemplate.exchange(URL, HttpMethod.GET, entity, Repo[].class);

    Repo[] arr = repos.getBody();

    for (Repo repo : arr) {
        languages.add(repo.getLanguage());
    }

    // Remove Duplicates
    languages = new ArrayList<String>(new LinkedHashSet<String>(languages));

    // Synthesize output.
    for(String language : languages) {
        int count = 0;
        for(Repo repo: arr){
            if(repo.getLanguage().equals(language)){
                count++;
            }
        }
        star_counts.add(count);
    }

    output.add(languages);
    output.add(star_counts);

    return output;

}

When I spin up the server and try to make a request, it works for the users who have starred a small number of repos. Here is a sample response when requested for a username who is having a small number of starred repos.

[["Java","Kotlin","JavaScript","TypeScript"],[1,1,3,1]]

The array at index 1 represents the stars per each language.

But when I try to do this for users who have starred more repos (like 80 - 90) it responds with a 500 code.

{"timestamp":1517758896463,"status":500,"error":"Internal Server Error","exception":"java.lang.NullPointerException","message":"No message available","path":"/getstarsperlang/ims94"}

How can I fix this?

Thank you.

Here's the console output.

21:25:50.531 [main] DEBUG org.springframework.boot.devtools.settings.DevToolsSettings - Included patterns for restart : []
21:25:50.603 [main] DEBUG org.springframework.boot.devtools.settings.DevToolsSettings - Excluded patterns for restart : [/spring-boot-starter/target/classes/, /spring-boot-autoconfigure/target/classes/, /spring-boot-starter-[\w-]+/, /spring-boot/target/classes/, /spring-boot-actuator/target/classes/, /spring-boot-devtools/target/classes/]
21:25:50.603 [main] DEBUG org.springframework.boot.devtools.restart.ChangeableUrls - Matching URLs for reloading : [file:/home/dasun/IdeaProjects/spring-github-repo-dev/target/classes/]

  .   ____          _            __ _ _
 /\\ / ___'_ __ _ _(_)_ __  __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
 \\/  ___)| |_)| | | | | || (_| |  ) ) ) )
  '  |____| .__|_| |_|_| |_\__, | / / / /
 =========|_|==============|___/=/_/_/_/
 :: Spring Boot ::        (v1.5.9.RELEASE)

2018-02-04 21:25:54.103  INFO 8480 --- [  restartedMain] c.s.s.SpringGithubApplication            : Starting SpringGithubApplication on dasun-X555LJ with PID 8480 (/home/dasun/IdeaProjects/spring-github-repo-dev/target/classes started by dasun in /home/dasun/IdeaProjects/spring-github-repo-dev)
2018-02-04 21:25:54.104  INFO 8480 --- [  restartedMain] c.s.s.SpringGithubApplication            : No active profile set, falling back to default profiles: default
2018-02-04 21:25:54.859  INFO 8480 --- [  restartedMain] ationConfigEmbeddedWebApplicationContext : Refreshing org.springframework.boot.context.embedded.AnnotationConfigEmbeddedWebApplicationContext@a185d1a: startup date [Sun Feb 04 21:25:54 IST 2018]; root of context hierarchy
2018-02-04 21:26:05.097  INFO 8480 --- [  restartedMain] s.b.c.e.t.TomcatEmbeddedServletContainer : Tomcat initialized with port(s): 8080 (http)
2018-02-04 21:26:05.109  INFO 8480 --- [  restartedMain] o.apache.catalina.core.StandardService   : Starting service [Tomcat]
2018-02-04 21:26:05.110  INFO 8480 --- [  restartedMain] org.apache.catalina.core.StandardEngine  : Starting Servlet Engine: Apache Tomcat/8.5.23
2018-02-04 21:26:05.397  INFO 8480 --- [ost-startStop-1] o.a.c.c.C.[Tomcat].[localhost].[/]       : Initializing Spring embedded WebApplicationContext
2018-02-04 21:26:05.397  INFO 8480 --- [ost-startStop-1] o.s.web.context.ContextLoader            : Root WebApplicationContext: initialization completed in 10543 ms
2018-02-04 21:26:05.488  INFO 8480 --- [ost-startStop-1] o.s.b.w.servlet.ServletRegistrationBean  : Mapping servlet: 'dispatcherServlet' to [/]
2018-02-04 21:26:05.490  INFO 8480 --- [ost-startStop-1] o.s.b.w.servlet.FilterRegistrationBean   : Mapping filter: 'characterEncodingFilter' to: [/*]
2018-02-04 21:26:05.491  INFO 8480 --- [ost-startStop-1] o.s.b.w.servlet.FilterRegistrationBean   : Mapping filter: 'hiddenHttpMethodFilter' to: [/*]
2018-02-04 21:26:05.491  INFO 8480 --- [ost-startStop-1] o.s.b.w.servlet.FilterRegistrationBean   : Mapping filter: 'httpPutFormContentFilter' to: [/*]
2018-02-04 21:26:05.491  INFO 8480 --- [ost-startStop-1] o.s.b.w.servlet.FilterRegistrationBean   : Mapping filter: 'requestContextFilter' to: [/*]
2018-02-04 21:26:06.061  INFO 8480 --- [  restartedMain] s.w.s.m.m.a.RequestMappingHandlerAdapter : Looking for @ControllerAdvice: org.springframework.boot.context.embedded.AnnotationConfigEmbeddedWebApplicationContext@a185d1a: startup date [Sun Feb 04 21:25:54 IST 2018]; root of context hierarchy
2018-02-04 21:26:06.109  INFO 8480 --- [  restartedMain] s.w.s.m.m.a.RequestMappingHandlerMapping : Mapped "{[/getuser/{username}],methods=[GET]}" onto public com.springgithub.springgithub.model.User com.springgithub.springgithub.controller.GithubController.getUser(java.lang.String)
2018-02-04 21:26:06.111  INFO 8480 --- [  restartedMain] s.w.s.m.m.a.RequestMappingHandlerMapping : Mapped "{[/getrepo/{username}],methods=[GET]}" onto public java.lang.Object com.springgithub.springgithub.controller.GithubController.getRepository(java.lang.String)
2018-02-04 21:26:06.111  INFO 8480 --- [  restartedMain] s.w.s.m.m.a.RequestMappingHandlerMapping : Mapped "{[/getcommits/{username}/{repo}],methods=[GET]}" onto public java.lang.Object com.springgithub.springgithub.controller.GithubController.getCommits(java.lang.String,java.lang.String)
2018-02-04 21:26:06.111  INFO 8480 --- [  restartedMain] s.w.s.m.m.a.RequestMappingHandlerMapping : Mapped "{[/getcommitsadapterRe/{username}],methods=[GET]}" onto public java.util.Map com.springgithub.springgithub.controller.GithubController.getCommitsAdaptorRe(java.lang.String)
2018-02-04 21:26:06.111  INFO 8480 --- [  restartedMain] s.w.s.m.m.a.RequestMappingHandlerMapping : Mapped "{[/getstarsperlang/{username}],methods=[GET]}" onto public java.util.ArrayList<java.lang.Object> com.springgithub.springgithub.controller.GithubController.getStarsPerLang(java.lang.String)
2018-02-04 21:26:06.114  INFO 8480 --- [  restartedMain] s.w.s.m.m.a.RequestMappingHandlerMapping : Mapped "{[/error],produces=[text/html]}" onto public org.springframework.web.servlet.ModelAndView org.springframework.boot.autoconfigure.web.BasicErrorController.errorHtml(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse)
2018-02-04 21:26:06.114  INFO 8480 --- [  restartedMain] s.w.s.m.m.a.RequestMappingHandlerMapping : Mapped "{[/error]}" onto public org.springframework.http.ResponseEntity<java.util.Map<java.lang.String, java.lang.Object>> org.springframework.boot.autoconfigure.web.BasicErrorController.error(javax.servlet.http.HttpServletRequest)
2018-02-04 21:26:06.136  INFO 8480 --- [  restartedMain] o.s.w.s.handler.SimpleUrlHandlerMapping  : Mapped URL path [/webjars/**] onto handler of type [class org.springframework.web.servlet.resource.ResourceHttpRequestHandler]
2018-02-04 21:26:06.136  INFO 8480 --- [  restartedMain] o.s.w.s.handler.SimpleUrlHandlerMapping  : Mapped URL path [/**] onto handler of type [class org.springframework.web.servlet.resource.ResourceHttpRequestHandler]
2018-02-04 21:26:06.187  INFO 8480 --- [  restartedMain] o.s.w.s.handler.SimpleUrlHandlerMapping  : Mapped URL path [/**/favicon.ico] onto handler of type [class org.springframework.web.servlet.resource.ResourceHttpRequestHandler]
2018-02-04 21:26:06.292  INFO 8480 --- [  restartedMain] o.s.b.d.a.OptionalLiveReloadServer       : LiveReload server is running on port 35729
2018-02-04 21:26:06.378  INFO 8480 --- [  restartedMain] o.s.j.e.a.AnnotationMBeanExporter        : Registering beans for JMX exposure on startup
2018-02-04 21:26:06.470  INFO 8480 --- [  restartedMain] s.b.c.e.t.TomcatEmbeddedServletContainer : Tomcat started on port(s): 8080 (http)
2018-02-04 21:26:06.474  INFO 8480 --- [  restartedMain] c.s.s.SpringGithubApplication            : Started SpringGithubApplication in 15.807 seconds (JVM running for 28.552)
2018-02-04 21:26:13.768  INFO 8480 --- [nio-8080-exec-1] o.a.c.c.C.[Tomcat].[localhost].[/]       : Initializing Spring FrameworkServlet 'dispatcherServlet'
2018-02-04 21:26:13.769  INFO 8480 --- [nio-8080-exec-1] o.s.web.servlet.DispatcherServlet        : FrameworkServlet 'dispatcherServlet': initialization started
2018-02-04 21:26:13.783  INFO 8480 --- [nio-8080-exec-1] o.s.web.servlet.DispatcherServlet        : FrameworkServlet 'dispatcherServlet': initialization completed in 14 ms
2018-02-04 21:26:17.133 ERROR 8480 --- [nio-8080-exec-1] o.a.c.c.C.[.[.[/].[dispatcherServlet]    : Servlet.service() for servlet [dispatcherServlet] in context with path [] threw exception [Request processing failed; nested exception is java.lang.NullPointerException] with root cause

java.lang.NullPointerException: null
    at com.springgithub.springgithub.controller.GithubController.getStarsPerLang(GithubController.java:135) ~[classes/:na]
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_161]
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_161]
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_161]
    at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_161]
    at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205) ~[spring-web-4.3.13.RELEASE.jar:4.3.13.RELEASE]
    at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:133) ~[spring-web-4.3.13.RELEASE.jar:4.3.13.RELEASE]
    at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:97) ~[spring-webmvc-4.3.13.RELEASE.jar:4.3.13.RELEASE]
    at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:827) ~[spring-webmvc-4.3.13.RELEASE.jar:4.3.13.RELEASE]
    at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:738) ~[spring-webmvc-4.3.13.RELEASE.jar:4.3.13.RELEASE]
    at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:85) ~[spring-webmvc-4.3.13.RELEASE.jar:4.3.13.RELEASE]
    at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:967) ~[spring-webmvc-4.3.13.RELEASE.jar:4.3.13.RELEASE]
    at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:901) ~[spring-webmvc-4.3.13.RELEASE.jar:4.3.13.RELEASE]
    at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:970) ~[spring-webmvc-4.3.13.RELEASE.jar:4.3.13.RELEASE]
    at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:861) ~[spring-webmvc-4.3.13.RELEASE.jar:4.3.13.RELEASE]
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:635) ~[tomcat-embed-core-8.5.23.jar:8.5.23]
    at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:846) ~[spring-webmvc-4.3.13.RELEASE.jar:4.3.13.RELEASE]
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:742) ~[tomcat-embed-core-8.5.23.jar:8.5.23]
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231) ~[tomcat-embed-core-8.5.23.jar:8.5.23]
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat-embed-core-8.5.23.jar:8.5.23]
    at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52) ~[tomcat-embed-websocket-8.5.23.jar:8.5.23]
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[tomcat-embed-core-8.5.23.jar:8.5.23]
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat-embed-core-8.5.23.jar:8.5.23]
    at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:99) ~[spring-web-4.3.13.RELEASE.jar:4.3.13.RELEASE]
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) ~[spring-web-4.3.13.RELEASE.jar:4.3.13.RELEASE]
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[tomcat-embed-core-8.5.23.jar:8.5.23]
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat-embed-core-8.5.23.jar:8.5.23]
    at org.springframework.web.filter.HttpPutFormContentFilter.doFilterInternal(HttpPutFormContentFilter.java:108) ~[spring-web-4.3.13.RELEASE.jar:4.3.13.RELEASE]
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) ~[spring-web-4.3.13.RELEASE.jar:4.3.13.RELEASE]
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[tomcat-embed-core-8.5.23.jar:8.5.23]
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat-embed-core-8.5.23.jar:8.5.23]
    at org.springframework.web.filter.HiddenHttpMethodFilter.doFilterInternal(HiddenHttpMethodFilter.java:81) ~[spring-web-4.3.13.RELEASE.jar:4.3.13.RELEASE]
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) ~[spring-web-4.3.13.RELEASE.jar:4.3.13.RELEASE]
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[tomcat-embed-core-8.5.23.jar:8.5.23]
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat-embed-core-8.5.23.jar:8.5.23]
    at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:197) ~[spring-web-4.3.13.RELEASE.jar:4.3.13.RELEASE]
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) ~[spring-web-4.3.13.RELEASE.jar:4.3.13.RELEASE]
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[tomcat-embed-core-8.5.23.jar:8.5.23]
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat-embed-core-8.5.23.jar:8.5.23]
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:199) ~[tomcat-embed-core-8.5.23.jar:8.5.23]
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96) [tomcat-embed-core-8.5.23.jar:8.5.23]
    at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:478) [tomcat-embed-core-8.5.23.jar:8.5.23]
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:140) [tomcat-embed-core-8.5.23.jar:8.5.23]
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:81) [tomcat-embed-core-8.5.23.jar:8.5.23]
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87) [tomcat-embed-core-8.5.23.jar:8.5.23]
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:342) [tomcat-embed-core-8.5.23.jar:8.5.23]
    at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:803) [tomcat-embed-core-8.5.23.jar:8.5.23]
    at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66) [tomcat-embed-core-8.5.23.jar:8.5.23]
    at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:868) [tomcat-embed-core-8.5.23.jar:8.5.23]
    at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1459) [tomcat-embed-core-8.5.23.jar:8.5.23]
    at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) [tomcat-embed-core-8.5.23.jar:8.5.23]
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [na:1.8.0_161]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_161]
    at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) [tomcat-embed-core-8.5.23.jar:8.5.23]
    at java.lang.Thread.run(Thread.java:748) [na:1.8.0_161]

I can almost guarantee that your request is timing out at `restTemplate.exchange`, and you're continuing on, assuming that data is going to be returned — OneCricketeer, Feb 04 '18 at 16:02
It's that if statement in the nested for loop. `if(repo.getLanguage().equals(language)){ count++; }` — dasunpubudumal, Feb 04 '18 at 16:02
Oh.. I see.. Is there anything I could do for that? I'm sorry but I'm new at this.. — dasunpubudumal, Feb 04 '18 at 16:03
1) You need to check if `repo!=null` before you attempt to get its language 2) https://stackoverflow.com/questions/13837012/spring-resttemplate-timeout — OneCricketeer, Feb 04 '18 at 16:08
I used https://stackoverflow.com/questions/32983328/setting-timeouts-in-spring-rest-template this and increased the timeout but still not working :/ — dasunpubudumal, Feb 04 '18 at 16:40
You should check `repos.getStatusCode()` before you get it's body, then. In my opinion, error handling is more important than perfect functionality — OneCricketeer, Feb 04 '18 at 16:44
I just added a `System.out.println("Response code is: " + repos.getStatusCode());` before getting the body and it returns a 200 :/ — dasunpubudumal, Feb 04 '18 at 16:55
Alright, then something is up with the Repo class that it cannot be parsed correctly, so it's null — OneCricketeer, Feb 04 '18 at 17:03
I tried debugging a little. The issue is with this `if(repo.getLanguage().equals(language)){ count++; }` thing.. When I comment it out, the error vanishes. — dasunpubudumal, Feb 04 '18 at 17:31
And it is not about the timeouts I'm afraid. The request is fine. This if statement is what messes things up. — dasunpubudumal, Feb 04 '18 at 17:35
I tried commenting out the conditional statement and logging `repo.getLanguage()` in that nested loop. I get the languages. So `repo` is not null.. :/ — dasunpubudumal, Feb 04 '18 at 17:55
` for(String language: languages) { int count = 0; for(Repository repo: repositories) { // Issue is with this conditional statement. // if(repo.getLanguage().equals(language)){ // count++; // } System.out.println(repo.getLanguage()); } star_counts.add(count); } ` — dasunpubudumal, Feb 04 '18 at 17:56
Hmm. Alright then. In any case, start with the languages being a hashset to begin with, then you don't need to shuffle them into one to remove duplicates — OneCricketeer, Feb 04 '18 at 17:58
Hey I figured it out. Used `Objects.equals()` method to compare the strings. I have added the code below. Thanks for your help :) — dasunpubudumal, Feb 04 '18 at 18:01

score 0 · Answer 1 · answered Feb 04 '18 at 18:00

0

The issue was with the conditional statement. I fixed it by doing this.

    for(String language: languages) {
        int count = 0;
        for(Repository repo: repositories) {
            // Issue is with this conditional statement.
            if(Objects.equals(repo.getLanguage(), language)){
                count++;
            }
        }
        star_counts.add(count);
    }

answered Feb 04 '18 at 18:00

dasunpubudumal

29
9

Okay, then one of the language instances was null, not the repo. Glad you figured it out – OneCricketeer Feb 04 '18 at 18:08
yeah.. Some repository languages are not identified by the Github API.. Took some time to figure that out.. Thanks for your help. – dasunpubudumal Feb 04 '18 at 18:10

Fetching Starred repositories from Github API in Spring

1 Answers1