I'm running HAProxy 1.5.18 to front a MySQL Percona XtraDB Cluster with everything setup as per guidelines on Percona website.
I'm seeing that the "Current Sessions" statistic is not updating as I would expect for a backend that has gone down then come back up again.
Its therefore pretty confusing to get an accurate picture of which backend mysql node is taking all the traffic.
Here's the frontend/backend config I'm using:
frontend pxc_frontend
mode tcp
bind *:6033
timeout client 28800s
default_backend pxc_backend
backend pxc_backend
mode tcp
timeout connect 10s
timeout server 28800s
balance leastconn
option httpchk
stick-table type ip size 1
stick on dst
server percona-node1 10.99.1.21:3306 check port 9200 inter 1000 rise 3 fall 3
server percona-node2 10.99.1.22:3306 backup check port 9200 inter 1000 rise 3 fall 3
server percona-node3 10.99.1.23:3306 backup check port 9200 inter 1000 rise 3 fall 3
Here's what I've tried:
1) Start my application up - this makes 50 connections to the DB (via HAProxy) hence the "current sessions" stat in the HAProxy UI shows as 50 for the backend that is the active one (percona-node1 in my case). I verify this using netstat to check the number of connections between HAProxy and the backend MySQL Node.
2) I then shutdown the backend mysql node with all the connections (percona-node1) and let HAProxy failover connections to the next backend in the list (percona-node2). I verify using netstat that HAProxy has 0 connections to the old backend (obviously) and now has 50 connections to the new backend. The "current sessions" stat in the HAProxy UI shows as 50 for the new backend but typically has a number <50 for the old backend.
3) I then bring the old backend mysql node back up again (percona-node1). I verify again using netstat that HAProxy has 0 connections to the newly restarted backend and maintains its 50 connections to the backend percona-node2. The "current sessions" stat in the HAProxy UI shows the same non-zero number for percona-node1 as before and 50 for percona-node2. I would expect it to show 0 for percona-node1 and 50 for percona-node2.
So does the current sessions stats not get cleared down for a node that has gone down then come back again?
Thanks in advance for your wisdom.