4

I'm using a paid Heroku ($35/mo 2x web dyno, $50/mo silver postgresdb) plan for a small internal app with typically 1-2 concurrent users. Database snapshot size is less than 1MB.

App is Rails 4.1. In the last two weeks, there's been a significant performance drop in production env, where Chrome dev tools reports response times like 8s

Typical:

Total ​8.13s
Stalled ​3.643 ms
DNS Lookup ​2.637 ms
Initial connection 235.532 ms
SSL ​133.738 ms
Request sent ​0.546 ms
Waiting (TTFB) ​3.43 s
Content Download ​4.47 s

I'm using Nitrous dev environment and get sub-1s response on dev server with non-precompiled assets (with mirrored db).

I'm a novice programmer and am not clear on how to debug this. Why would I see 800% slower performance than the dev environment on an $85/mo+ Heroku plan? Given my current programming skill level, my app is probably poorly optimized (a few N+1 queries in there...) but how bad can it be when production has 1-2 concurrent users??

Sample from logs if it helps: sample#current_transaction=16478 sample#db_size=18974904bytes sample#tables=28 sample#active-connections=6 sam ple#waiting-connections=0 sample#index-cache-hit-rate=0.99929 sample#table-cache-hit-rate=0.99917 sample#load-avg-1m=0.365 sample#load-avg-5m=0.45 sample#load-avg-15m=0.445 sample#read-iops= 37.587 sample#write-iops=36.7 sample#memory-total=15405616kB sample#memory-free=1409236kB sample#memory-cached=12980840kB sample#memory-postgres=497784kB

Sample from server logs: Completed 200 OK in 78ms (Views: 40.9ms | ActiveRecord: 26.6ms

I'm seeing similar numbers on dev server but the actual visual performance is night and day. The dev responds as you would expect - sub-1s response and render. The production server is 4-5s delay.

I don't think it's related to ISP as suggested because I've actually been traveling and seeing identical performance problems from USA and Europe.

Lone Dev
  • 53
  • 4
  • Start with the application logs, not the db ones. They'll tell you the breakdown between db queries, rendering etc – Frederick Cheung Feb 22 '15 at 22:04
  • Added further info -- I'm really stumped. Any more advice appreciated! I might be reading incorrectly but server logs don't seem to identify any response slowness -- I'm only seeing the issue on client side. But it's happening to me across all devices and on different physical continents! – Lone Dev Feb 23 '15 at 15:27

1 Answers1

4

Add NewRelic add-on to your app's ecosystem on Heroku and explore what's going on. You can do it from Heroku's dashboard. Choose free plan and you will be provided with 2 weeks trial period of full functionality. After trial period there will be some constraints but anyway it's enough for measuring performance of the small app. Also you can add the Logentries add-on and you will get access to the app's log history via web interface.

atomdev
  • 321
  • 1
  • 8
  • Thanks, but what am I looking for? For example, the db is posting this every 15 seconds to the logs -- included as edit in question. – Lone Dev Feb 22 '15 at 21:32
  • Just add NewRelic and you will see the hole picture on the graphs how your application react to requests on server side. Maybe in your case problem is in your internet provider or your wifi router or something else not correlated to your app. – atomdev Feb 23 '15 at 06:38
  • Well this is insane. I just finished installing New Relic and the app is screaming fast... but I didn't do anything! It's almost like the last build I pushed messed up somehow and this new commit with the NR gem added sorted it back out. I guess I'll monitor for a few days and close this question if the issue is resolved... – Lone Dev Feb 23 '15 at 15:41
  • 1
    So, what did you end up doing? Did you fix your problem? – Cos Mar 23 '17 at 12:38