Docker slow non-local database access

Question

We are using Docker for our local development environments, and deploying to Google Cloud Container Engine for production. Our production SQL server is also on Google Cloud.

When our container is running on Container Engine, DB access is lightning fast, but is horribly slow when running in our local machines. Some page loads that normally took 1.9s on production is brought up to 25.0s and up on our locals.

I assumed proximity may have been an issue, so I tried booting up a MySQL Docker container on my local machine, and connecting the local Dockerized web server to that instead, and was fast. That 1.9s page load on production was hovering around 1.8s now on my local.

I decided I would get a MySQL Docker instance running on one of our servers in our office, but when I did those page loads spiked back up to 25.0s and up again.

It would seem that something is slowing down the connection the moment Docker needs to connect to any resource not on my local machine.

My docker info output if it helps at all:

Containers: 4
 Running: 2
 Paused: 0
 Stopped: 2
Images: 203
Server Version: 1.12.2
Storage Driver: aufs
 Root Dir: /mnt/sda1/var/lib/docker/aufs
 Backing Filesystem: extfs
 Dirs: 209
 Dirperm1 Supported: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
 Volume: local
 Network: null bridge host overlay
Kernel Version: 4.4.24-boot2docker
Operating System: Boot2Docker 1.12.2 (TCL 7.2); HEAD : 9d8e41b - Tue Oct 11 23:40:08 UTC 2016
OSType: linux
Architecture: x86_64
CPUs: 4
Total Memory: 3.858 GiB
Name: dev
ID: AYO6:TL7P:XSST:VI5T:HKCC:QTOZ:HP3X:NKYQ:YCWT:THV2:AQM2:5XER
Docker Root Dir: /mnt/sda1/var/lib/docker
Debug mode (client): false
Debug mode (server): true
 File Descriptors: 32
 Goroutines: 58
 System Time: 2016-10-26T22:46:19.218543394Z
 EventsListeners: 1
Registry: https://index.docker.io/v1/
Labels:
 provider=virtualbox

EDIT: While trying to narrow down the problem, I set up the MySQL Docker container on my Macbook, and then set up our Dockerized web server on another machine (running Ubuntu 14.04). Database access was fast, but when I reversed this (MySQL on the Ubuntu machine, web server on the Macbook) it was dreadfully slow again. Strange. I also tried updating Docker on my Macbook to 1.12, which removes VirtualBox from the equation, and this hasn't solved the issue either. It would SEEM that there's something going on with the network adapter for Docker perhaps? I'm stumped.

Answer 1

Not really the solution I was hoping for, but just in case anyone else comes across this and is banging their head off a wall.

Our solution was to run local copies of MySQL as a Docker service on our local machines and clone our database into that. Not an ideal solution, but it did away with the 10-30+ second response time on every single web request.

I also discovered that running your Docker container inside of a Linux VM, and using the host-only adapter made a big difference as well (unfortunately Docker for macOS doesn't support host-only adapter).

Answer 2

Which is the version of Docker for Mac you were using? Because back in the time I experienced a similar issue and now I found a known issue for Docker for MAC that could be related to you.

• "Experiencing sluggish I/O, particularly on MySQL official container, from container initialisation to CREATE TABLE and INSERT queries."

This issue has been mapped into the following one.

• Severe Docker 1.12.1 performance regression with DB2 images (~10x slower)

They provided also a workaround, but right now it should be enough to update the Docker version.

Answer 3

Try running OPTIMIZE on individual tables or if you want to fix all of them in a db see this answer