12

Issue

I have an Express (Node.JS) + MongoDB app with a server response load time of 4 - 7 seconds on average (slow).

I understand that the average server response time is under 200ms as per google pagespeed tools.

This app is fetching data from a mongoDB asynchronously but the roundtrip times to the database is extremely slow with each call averaging about 500ms - 1s. These calls are simple findAll calls to retrieve data of less than < 100 records.

Context

  • Mongoose version: 4.13.14
  • DB server's MongoDB version is 3.4.16
  • DB server is hosted on MongoDB Atlas M10 in AWS / Oregon (us-west-1)
  • Web server is hosted with now.sh in SFO1 (us-west-1)
  • Have performed recommended indexes as advised by MongoDB Atlas's performance advisor
  • Data fetching perfectly fine in local environment (local server + local db) as data is queried in a matter of few ms
  • Mongoose logs for the affected page can be found in this gist

Mongo Server configuration

  • Mongo Atlas M10
  • 2GB Ram
  • 10 GB Storage
  • 100 IOPS
  • Encrypted
  • Auto-expand storage

Attempted solutions:

I have checked my DB metrics, they looked fine. There are also no slow queries. These are simple findAll queries. Performance advisor on mongo atlas reports nothing unusual.

The production application and database are both hosted in the same region.

I have already tried optimising the application layer of the query (mongoose) by running .lean()

Question:

Where else should i look to improve the database latency? How can a simple query take so long? Otherwise, why is my server response time taking up to 4s when the expected is about 200ms?

clodal
  • 1,615
  • 20
  • 33
  • Are your database calls going through an ORM/Driver? e.g Mongoose? That's the only thing I can think of – Prodigle Aug 14 '18 at 10:14
  • Sounds like there are a few areas of concern, the DB taking ~500-1s for < 100 records is pretty poor so you might want to think about indexing. However, outwith that, it's very difficult to diagnose a problem with no hard details like where it's hosted, who it's hosted with, what level of hosting (tier) it's running on, what physical hardware it's on, how distributed the app is.....and so on. – James Aug 14 '18 at 11:20
  • @James Thanks! I've updated the question – clodal Aug 14 '18 at 14:51
  • The roundtrip time is 0.5 -> 1s only? And 2-3s to wait for DB processing? – Mạnh Quyết Nguyễn Aug 15 '18 at 01:55
  • 1
    Can you enable debug mode and post the logs here? Also have you tried the mongo shell to connect to your remote database directly? What is the response time when you try Mongo shell? – manishg Aug 15 '18 at 03:30
  • What is your server configuration? [ram,cores,cpus etc.] – mehta-rohan Aug 17 '18 at 11:08
  • @mehta-rohan Have updated question with config – clodal Aug 19 '18 at 06:03
  • @manishg Surprisingly when making queries directly from mongo shell (bypassing my application), queries are fast (1s for a 3000 record findAll query) with connection latency from ap-southeast-1 to us-west-1. Given that production server and database are in the same region, we can expect that query to be even faster. It seems like bypassing the application (NodeJS/Mongoose) helped. What do you think about this? – clodal Aug 19 '18 at 06:54
  • Given that mongo shell is fast, my guess is that mongoose will be on similar lines as per as latency is concerned. You can assert this by looking at mongoose log. Since there is not enough info, it’s tough to guess the root cause of your problem. Either share the log or sample code so that someone can look at it. – manishg Aug 19 '18 at 07:05
  • @manishg Thanks for your input! Have updated the question with the said logs. – clodal Aug 19 '18 at 07:33
  • Just to make sure that there is actually some problem with your local mongo server, try to move your data from local to some third party MongoDB hosting service like **mLab**: https://mlab.com/ . See if you still face the same error. – UtkarshPramodGupta Aug 21 '18 at 05:56
  • Try what @UtkarshPramodGupta suggested and can I have look at schema please and exact query that you are trying. – Gandalf the White Aug 21 '18 at 07:18
  • @JoelKoh the app perform slow with queries that involve fetching big amount of data from the DB? Or fetching small amout of data performs slow as well? – Dmitry Shvetsov Aug 21 '18 at 09:36
  • @clodal I am also facing this issue, with very less number of records the response time for a simple find query is 1 sec. Number of records are going to increase going forward to thousands and even more than that. Did you get any solution for this? – Avani Khabiya Feb 02 '21 at 13:40

3 Answers3

2

Hey you can try hosting your server and database in the same region. I think the network is creating a overhead in this case. If the server and the database are in the same region, They are on the same network which will reduce the latency significantly. there is a diagram on aws for this enter image description here

Aditya_Anand
  • 525
  • 7
  • 17
  • I've move both app server and db server to the same region, but did not notice much difference, so this is probably not the answer. Thanks for your input! – clodal Aug 19 '18 at 05:40
0

I add some problem like yours with an app that i developed in my master degree. I add to put a node.js api running online to present it in class room.And i realized that every time i wanted to make a call in the api the response was taking allot of time. I realized that one of the problems was the school network because of the firewalls. Also the place where i put the server heroku.com was giving some delay as well. What i did was use Redis ( https://redis.io/ ) to improve the performance, also heroku was giving me some problems because of the requests being http and not https.

Make a test running the app and data on your localhost and see the performance. if you don´t have any issue try to check if nothing is messing with your request like the place where you host your node server.

Let me know if this helps or if you still have issues so i can try to help you out better.

Tiago Neiva
  • 337
  • 1
  • 6
  • Thanks for your input! I've checked on firewalls with Mongo support, there is none on their end. From my end, the issue persists across varying connections. Have no problems with the ssl protocols on my end. May have to explore redis as a last resort. But would still like to solve this core issue. Will try out that barebones test you suggested. Thanks for your input! – clodal Aug 19 '18 at 06:02
0

I had the same issue once with my nodejs code using the same development stack(mongodb,nodejs), I got into trouble of late response from api, and after spending a lot of time I found my server the real culprit I then changed from heroku to amazon aws EC2 instance and things started working fast and amazingly fast, so probably your web server is culprit

to make sure mongodb is not culprit, write an api endpoint where you can just return some json response without making any query to database.

Muhammad Usman Ali Baloch
  • 64
  • 3
  • 13
  • Setting up that API endpoint where no database queries were made, TTFB dropped from 3.35s to 868ms. It's probably mongodb yeah? – clodal Aug 19 '18 at 05:55
  • ok my recommendations for now is - try **mlab** mongodb [link](https://mlab.com/login/) , check if it affects because this works great, it has never disappointed me, create new account and use free mongodb instance.And one question, are you running code locally. – Muhammad Usman Ali Baloch Aug 20 '18 at 08:00