2

I got a project handed over to me that is in production with Kubernetes. The issue is that my localhost works fine but on the acceptance server there is an internal server error when I send a particular POST request to that API endpoint. Another issue is that I have very little experience with Kubernetes and need to deploy preferably tomorrow.

On my dev machine, I'd simply find the Java stacktrace and figure out what would go wrong. On my dev machine it is simply runned with: ./mvnw my_app. It is a Jhipster microservices application.

However, on Kubernetes I get logs like:

2018-09-16 13:25:48.687  INFO 16 --- [rter-1-thread-1] metrics                                  : type=TIMER, name=com.hw.auth.web.rest.UserResource.sendEmails, count=0, min=0.0, max=0.0, mean=0.0, stddev=0.0, median=0.0, p75=0.0, p95=0.0, p98=0.0, p99=0.0, p999=0.0, mean_rate=0.0, m1=0.0, m5=0.0, m15=0.0, rate_unit=events/second, duration_unit=milliseconds
2018-09-16 13:25:48.687  INFO 16 --- [rter-1-thread-1] metrics                                  : type=TIMER, name=com.hw.auth.web.rest.UserResource.updateUser, count=0, min=0.0, max=0.0, mean=0.0, stddev=0.0, median=0.0, p75=0.0, p95=0.0, p98=0.0, p99=0.0, p999=0.0, mean_rate=0.0, m1=0.0, m5=0.0, m15=0.0, rate_unit=events/second, duration_unit=milliseconds

This doesn't tell me anything where something is failing.

How would I find the error/Java stacktrace that I'd normally also get on my dev machine? Or rather what is the strategy for finding this on Google? The search terms that I am typing in aren't working.

Edit: a related question that gives 50% of the answer: how to ssh docker container

For me the command was: kubectl exec -it my-pod-on-kubernetes-seen-via-the-kubectl-get-pods-command /bin/sh

Rico
  • 58,485
  • 12
  • 111
  • 141
Melvin Roest
  • 1,392
  • 1
  • 15
  • 31
  • This really depends on where your application sends its logs – Phil Sep 16 '18 at 13:47
  • Since I can SSH into it via the command that I just wrote all I need to ask myself now is how to redirect a running application to my stderr and stdout – Melvin Roest Sep 16 '18 at 13:50
  • If it's logging to `sdtout`, etc, you should be able to use `kubectl logs `. There's also the [Web UI (Dashboard)](https://kubernetes.io/docs/tasks/access-application-cluster/web-ui-dashboard/) which should show you just about everything – Phil Sep 16 '18 at 13:52
  • 1
    Hmm... those are the logs that I just described. Maybe it's something in the war file itself since it runs on a production profile. – Melvin Roest Sep 16 '18 at 14:00
  • Does your pod have only one container or multiple? If the latter, add `--all-containers=true` to the `kubectl logs` command. See https://kubernetes.io/docs/reference/generated/kubectl/kubectl-commands#logs – Phil Sep 16 '18 at 14:09
  • One container, per pod. But I'll try it :) -- edit: it didn't work and I think it's because it only logs metrics. I think I must reconfigure my Java application for it to also show debugging info. – Melvin Roest Sep 16 '18 at 14:11
  • I found it. It was a Java issue. The prod config had an `INFO` on logging and it needed to be on `DEBUG`. – Melvin Roest Sep 16 '18 at 21:27

0 Answers0