MariaDB gcomm backend connection failed 110

Question

I'm trying to get work MariaDB Galera 10.1 under debian 8 jessie.

I've installed all neccesary components and did configurations but i cannot get it work.

The nodes are builded as VPS.

Configuration of node 1:

[mysqld]

# Cluster node configurations
wsrep_cluster_address="gcomm://172.16.0.102,172.16.0.112"
wsrep_node_address="172.16.0.102"
wsrep_node_name='n1' 
wsrep_cluster_name='cluster'
innodb_buffer_pool_size=400M
# Mandatory settings to enable Galera
wsrep_provider=/usr/lib/galera/libgalera_smm.so
binlog_format=ROW
default-storage-engine=InnoDB
innodb_autoinc_lock_mode=2
innodb_doublewrite=1
query_cache_size=0
bind-address=0.0.0.0
# Galera synchronisation configuration
wsrep_sst_method=rsync

Configuration of node 2:

[mysqld]

# Cluster node configurations
wsrep_cluster_address="gcomm://172.16.0.102,172.16.0.112"
wsrep_node_address="172.16.0.112"
wsrep_node_name='n2' 
wsrep_cluster_name='cluster'
innodb_buffer_pool_size=400M
# Mandatory settings to enable Galera
wsrep_provider=/usr/lib/galera/libgalera_smm.so
binlog_format=ROW
default-storage-engine=InnoDB
innodb_autoinc_lock_mode=2
innodb_doublewrite=1
query_cache_size=0
bind-address=0.0.0.0
# Galera synchronisation configuration
wsrep_sst_method=rsync

When I'm trying to run on node 1 bootstrap command

service mysql bootstrap

It fails with error's

May 13 15:59:28 test mysqld[2397]: 2016-05-13 15:59:28  139843152635840 [ERROR] WSREP: failed to open gcomm backend connection: 110: failed to reach primary view: 110 (Connection timed out)
May 13 15:59:28 test mysqld[2397]: at gcomm/src/pc.cpp:connect():162
May 13 15:59:28 test mysqld[2397]: 2016-05-13 15:59:28 139843152635840 [ERROR] WSREP: gcs/src/gcs_core.cpp:gcs_core_open():208: Failed to open backend connection: -110 (Connection timed out) 
May 13 15:59:28 test mysqld[2397]: 2016-05-13 15:59:28 139843152635840 [ERROR] WSREP: gcs/src/gcs.cpp:gcs_open():1379: Failed to open channel 'cluster' at 'gcomm://172.16.0.102,172.16.0.112': -110 (Connection timed out)
May 13 15:59:28 test mysqld[2397]: 2016-05-13 15:59:28 139843152635840 [ERROR] WSREP: gcs connect failed: Connection timed out
May 13 15:59:28 test mysqld[2397]: 2016-05-13 15:59:28 139843152635840 [ERROR] WSREP: wsrep::connect(gcomm://172.16.0.102,172.16.0.112) failed: 7
May 13 15:59:28 test mysqld[2397]: 2016-05-13 15:59:28 139843152635840 [ERROR] Aborting

The network configuration is private I'm using:

2x DEDICATED server installed with ProxmoxVE 4.0 the servers are in vRack network is configured on VPS as:

node1: 172.16.0.102 //node 1 is on server 1

node2: 172.16.0.112 //node 2 is on server 2

They're able to ping each other on private network.

did you check the network adapters settings for each proxmox server instance, there is a setting for allowing vms to talk to each other. — iGNEOS, May 15 '16 at 03:38
VMS does not have issues to talk to each other. I can simply do ping or ssh connection from virtual machine to other one using private network. So in this case is not an issue. — Kamil, May 16 '16 at 23:45

score 25 · Answer 1 · answered Aug 04 '16 at 02:51

Since MariaDB 10.1.8, systemd is the new init and it affects the way Galera is bootstrapped on RPM and Debian based Linux distributions (in my case Ubuntu 16.04). On previous versions, you would use something like service mysql start --wsrep-new-cluster or service mysqld bootstrap but that doesn't work any more as it fails with:

[ERROR] WSREP: gcs/src/gcs_core.cpp:gcs_core_open():208: Failed to open backend connection: -110 (Connection timed out)

To fix this issue run:

galera_new_cluster

Note that you only need to run this script on the 'first' server.

To test if it is running, enter mysql with mysql -u [your mysql user] -p and then run

SHOW GLOBAL STATUS LIKE 'wsrep_cluster_size';

You should see something like:

+--------------+
| cluster size |
+--------------+
| 1            |
+--------------+

Just in case it is useful to anyone, this is my my.conf (MariaDB 10.1.16)

[galera]
# Mandatory settings
wsrep_on=ON
wsrep_provider=/usr/lib/galera/libgalera_smm.so
wsrep_cluster_address="gcomm://[first ip],[second ip]"
binlog_format=row
default_storage_engine=InnoDB
innodb_autoinc_lock_mode=2

More details here:

MariaDB systemd and galera_new_cluster

Galera Cluster System Variables

Any ideas what could be causing this error on the docker image? — psiyumm, Sep 28 '16 at 11:12

Kamil · Accepted Answer · 2016-08-09T12:53:36.740

16

I found solution and exact issue.

The issue was in way of starting cluster node. It's needed to start by script included in installation with MariaDB galera_new_cluster what will bootstrap new cluster and after that everything is working fine.

Hope this will help to others pass configuration under debian.

edited Aug 09 '16 at 12:53

answered May 29 '16 at 19:34

Kamil

990
1
7
24

There is a typo in your command it is galera_new_cluster with a single l. – Nicolas Henneaux Aug 09 '16 at 11:49

score 7 · Answer 3 · edited Sep 17 '20 at 07:03

7

I have this error, but in my case galera_new_cluster not works.

problem solved by editing /var/lib/mysql/grastate.dat file and change safe_to_bootstrap to 1.

after doing it, galera_new_cluster works.

edited Sep 17 '20 at 07:03

Milad Yarmohammadi

1,253
2
22
37

answered Jul 06 '18 at 17:51

vahid kargar

800
1
9
23

7

That is very dangerous if you set it the wrong way without knowing what you're doing. safe_to_bootstrap = 1 is intended only if you know that the node you want too bootstrap contains the most recent transaction compared to the other nodes. – toytoy Oct 03 '18 at 05:41

score 0 · Answer 4 · answered Apr 02 '21 at 11:34

0

In my case everything was configured properly and this error was caused by firewall which was blocking MariaDB ports. I fixed it by allowing them on all servers:

ufw allow 3306,4567,4568,4444/tcp;
ufw allow 4567/udp;

answered Apr 02 '21 at 11:34

Ilyich

4,966
3
39
27

score 0 · Answer 5 · edited Dec 23 '21 at 22:33

I know the topic is already quite old. I was trying to install a kubernetes mariadb cluster and was running into that issue. I had to copy the information from the grastate.dat from one node to the next. I had to show the one form node 1 with cat grastate.dat and copy everything over to the next node without the uuid. The file looked like the following once done:

# GALERA saved state
version: 2.1
uuid:    
seqno:   -1
safe_to_bootstrap: 1

Then save and delete the container. Once it restarts it uses the details from that file and will start up properly.

score 0 · Answer 6 · answered May 23 '23 at 10:56

galera has changed : now turning the line safe_to_bootstrap = 1 is not enough.

to initiate the cluster, or even to restart it the first node has to start mariadb with the command galera_new_cluster. then you can start the other node, when all nodes are up restart the first one in normal mode.

MariaDB gcomm backend connection failed 110

6 Answers6