-1

here is the issue:

  • I used the CloudSQL price calculator to estimate the price of running a website, my website has 1000-2000 URLs and each URL will use the DB in some way, I don't have more than 1GB of data, and I mostly deal with reads, for a small 50k record table, nothing super-complicated, I don't currently have very complex queries either, and I write into the db only once a week maybe a couple records here and there, I've even considered SQLITE tbh.

  • I don't currently have a lot of traffic, maybe people come visit once a day, however, GoogleBot will continuously try to index the website via the sitemap, which causes some times lots of requests on the server.

  • Currently, I have a normal php+mysql website which does the job on a DigitalOcean instance, which doesn't take a lot of resources, however, I want to move to Cloud Run in order to try the Cloud Run technology, but running MySQL directly on the VM is discouraged (as per this question Should I run mysql on google cloud run? (or any database))

  • So I'm kind of afraid of using CloudSQL and then having GoogleBot destroying my credit card by doing lots of concurrent requests into the CloudSQL Database during daily indexation.

Traffic doesn't scare me (I don't have any), but crawlers do.

  • Should I use CloudSQL for this usecase?
  • Will my credit card be destroyed?
  • Are these valid concerns?

Any opinion from experienced CloudSQL Users would be greatly appreciated.

ahmet alp balkan
  • 42,679
  • 38
  • 138
  • 214
Felipe Valdes
  • 1,998
  • 15
  • 26

1 Answers1

1
  1. If you consider fully managed database instance Google Cloud is definitely good choice for you.
  2. If you want to optimize GoogleBot crawling, you can do it from here
  3. However, if you experience high server load from specific sites/services you may consider blocking them or using Google Cloud CDN caching

Please read this article will explain how to deal with heavy bot load on the website

Your concerns do not sound valid to me, since you can limit GoogleBot crawling rate.

Since Cloud Run is compute platform STATELESS container service, it is not suited to install MySQL. If you are searching to install your own MySQL server and manage it, you can do it on Cloud Compute Engine using one click solution from Marketplace

Pawel Czuczwara
  • 1,442
  • 9
  • 20
  • I appreciate the time you took to answer the question, do you have real world experience with CloudSQL on GCP? Limiting the crawlrate could be useful, but it feels like it would be like shooting myself in the foot in a sense... – Felipe Valdes Jul 18 '19 at 10:03
  • Yes I do. So if yo do not want to limit bot, why do not you give it a try? It all depends on how frequently your content changes. You have also to consider that Ingress traffic is for free. – Pawel Czuczwara Jul 18 '19 at 11:33