2

I am running some performance tests against PostgreSQL to test the performance when the DB server runs on a VSI and when it runs on a Kubernetes on a worker node. I use PgBench to run these tests.

My DB (that in production is configured in the cluster) has a big workload so I tested on VSI (on IBM Cloud) with 64 vCPU Dual Processor 2.3 GHz 32 cores and 128 Gb RAM, disk 2Tb 5IOPS per Gb.

Then I tested it on Kubernetes (IBM Cloud) with a worker node of 48 vCPU, 192 Gb RAM, disk 2 Tb 5 IOPS per Gb.

The problem is that performance on Kubernetes is 50% worst than VSI and I didn't expect all this difference. So I am trying to understand what could be the bottleneck.

  1. The disks in the tests are similar and throughput is similar, so the disk cannot be the bottleneck

  2. I used a service to access the Pod configured as Network Load Balancer (before it was Application Load Balancer and the performance is even worst). However, I did also a test with a Pod to Pod communication deploying PgBench on another worker node bypassing the Service. But no improvements have been noticed. So I think this should exclude the problem is the Load Balancer.

  3. The I though to reserve more CPU and RAM to PostgreSQL Pod adding something like this.

     resources:
   requests:
     memory: 128Gi
     cpu: 32
   limits:
     memory: 128Gi
     cpu: 32

At this point, I don't know what I can do to improve performance. Any suggestion? Is it normal that Kubernetes add all this overhead?

Salvatore D'angelo
  • 1,019
  • 3
  • 14
  • 39
  • I don't know PgBench, only database benchmarking in general. What type of workload is it, e.g., OLTP, disk vs. memory vs. network, concurrent users and transactions, ...? Is PgBench run on the same machine for all tests? Same software and software configuration? – data_henrik Jun 30 '21 at 11:53
  • How are the disks mounted by the Pods? Directly with a local storage manually created PVs? – AndD Jun 30 '21 at 12:11
  • PgBench is a tool shipped with PostgreSQL for test benchmarks. It creates 4 tables and creates a number of records proportional to a scale factor that determines the database size. You can use it in read-only mode (only SELECT) and read/write mode (Select+INSERT/UPDATE/DELETE). Data dir is mounted via PVClaim and the disk is the same used for VSI. The RAM is so high that the DB is all in memory. PgBench runs on another node to avoid it affects performance. The configuration is the same. – Salvatore D'angelo Jun 30 '21 at 12:43

0 Answers0