2

I have created a web application using jsp/tiles/struts/mysql/tomcat. I created new project on Openshift 3 console (Openshift online) https://console.preview.openshift.com/console/ then added tomcat/mySql. I was getting 503 error sometimes and other times, same page was working as expected. 503 error came randomly for any page from my project. When I get 503 error, I refresh some no of times and it goes away, and my page is correctly displayed.

Error that I see is: "503 Service Unavailable No server is available to handle this request. "

I did some research: What I understand from this openshift 2 link: https://blog.openshift.com/how-to-host-your-java-ee-application-with-auto-scaling/

is that to correct 503 error:

SSH into your application gear using rhc ssh --app <app_name>
Change directory to haproxy/conf
change the following in haproxy.cfg option httpchk GET / to option httpchk GET /api/v1/ping
Restart the HAProxy cartridge from your local machine using RHC rhc cartridge-restart --cartridge haproxy

I dont know if it is also applicable to openshift 3. In openshift 3 where is haproxy.log, haproxy.cfg, haproxy/conf or its slightly different in openshift 3. (Nut thanks to Warrens comments, yes he saw 503 error in openshift related to HAProxy)

Now after 1 week after posting this question: I am getting Quota Reached Error. I am able to build my project but all deployments are failing. I wonder if 503 error that I was getting earlier(either completely or partially) was related to Quota reached. How should I proceed now.

curl -i localhost:8080/GEA 

HTTP/1.1 302 Found Server: 
Apache-Coyote/1.1 
Location: http://localhost:8080/GEA/ 
Transfer-Encoding: chunked Date: Tue, 11 Apr 2017 18:03:25 GMT

Tomcat logs do not show any application error.

Will Readiness Probe and Liveness Probe help me? I have not set them yet. Nor do I know how to set them.

Will scaling help me (I dont know how to set it either)

Do I have to set memory/... all at maximum allowed to ensure project runs smooth?

Begineer
  • 21
  • 1
  • 6
  • You should completely ignore any documentation which talks about ``rhc`` as that is OpenShift 2. In OpenShift 3 things are done very differently. The first thing you should do is verify your application is up and running properly. To do that look at the logs for the pod in the web console, or use the command ``oc logs `` from the command line, where you work out the name of the pod using ``oc get pods``. Also, what OpenShift environment is this? Something you created, OpenShift Online, minishift, oc cluster up? – Graham Dumpleton Apr 09 '17 at 20:55
  • Thank You So much. – Begineer Apr 10 '17 at 10:18

1 Answers1

0

For me I had a similar situation of getting 503's sometimes and sometimes getting my actual page. the reason was because you have haproxy on the frontend handling the requests. Depending on your setup you may even have a few haproxy pods and your request could be funneled between one of the pods. So as in my case one pod was working and the other not.

So basically

oc get pods -n default
NAME                            READY     STATUS    RESTARTS   AGE
docker-registry-7-i02rh         1/1       Running   0          75d
registry-console-12-wciib       1/1       Running   0          67d
router-1-533cg                  1/1       Running   3          76d
router-1-9utld                  1/1       Running   1          76d
router-1-uwf64                  1/1       Running   1          76d

As you can see in my output default namespace is where my router(haproxy) pods live. If I change to that namespace

oc project default

Then run

oc logs -f router-1-533cg

on each of the pods you will most likely find a sepcific pod that is behaving bad. You can simply delete, and the replication controller will create a new one

Warren Paul
  • 56
  • 1
  • 5
  • Yea I'm using openshift 3.4 – Warren Paul Apr 11 '17 at 17:18
  • As far as debugging if your getting a 404 on localhost when you rsh into the box then there is some other issue. Is that the correct endpoint tho? Looks like your hitting root context in your app if its tomcat. Is the app actually deployed at root? Depending on what tomcat image your using, take a look in /opt/webserver/webapps/ You should see what folder "Context" your app is deployed as. Whatever your folder is if not ROOT then your curl should be something like ."curl -i http://localhost:8080/. – Warren Paul Apr 11 '17 at 17:24
  • Warren , so is curl -i localhost:8080/GEA HTTP/1.1 302 Found ok? – Begineer Apr 12 '17 at 07:14
  • Sorry got busy yesterday :) I would normally expect to see a 200, but that can also depend o if you actually have a valid resource. My suggestion is for now maybe try get the tomcat sample app working. Once you feel comfortable with that working then move onto your app. At least then you have a solid base to work from. As far as the syncing error I'm not sure. Are you using iptables or firewalld? – Warren Paul Apr 12 '17 at 14:24
  • I am not changing any default setting on openshift console. – Begineer Apr 12 '17 at 16:50