Dear all,
We have a problem with a MySQL 5.7 Multi Master Cluster. The last 2022/03/03 we had the 3 cluster nodes in online status but a day 2 of these nodes changed their status to missing.
We have try to change the missing nodes status to online status but we can't do and for this we need your help. In the cluster we have the online node called srvrh3 and a missing node called srvrh2.
Both nodes are in a production environment and have been fine until the 2022/03/03. When we try the to add the srvrh2 node to the cluster from the mysql shell installed in srvrh3 with the command cluster.addInstance("ic:<$PASSWORD>@<$HOST_NODE>:3306") we obtain the next errors:
2022-03-15T15:41:48.059796Z 1124340 [Note] Slave I/O thread for channel 'group_replication_recovery': connected to master 'mysql_innodb_cluster_r0481579755@srvrh03:3306',replication started in log 'FIRST' at position 4
2022-03-15T15:41:48.095095Z 1124341 [ERROR] Slave SQL for channel 'group_replication_recovery': Error 'Can't find any matching row in the user table' on query. Default database: 'mysql_innodb_cluster_metadata'. Query: 'GRANT REPLICATION SLAVE ON *.* TO 'mysql_innodb_cluster_r0440653904'@'%'', Error_code: 1133
2022-03-15T15:41:48.095184Z 1124341 [ERROR] Error running query, slave SQL thread aborted. Fix the problem, and restart the slave SQL thread with "SLAVE START". We stopped at log '1.000001' position 9205.