mirror of
https://github.com/openshift/openshift-docs.git
synced 2026-02-05 12:46:18 +01:00
67 lines
3.1 KiB
Plaintext
67 lines
3.1 KiB
Plaintext
// Module included in the following assemblies:
|
|
//
|
|
// * backup_and_restore/replacing-failed-master.adoc
|
|
|
|
[id="restore-remove-failed-master_{context}"]
|
|
= Removing a failed master host from the etcd cluster
|
|
|
|
Follow these steps to remove a failed master host from the etcd cluster.
|
|
|
|
.Prerequisites
|
|
|
|
* Access to the cluster as a user with the `cluster-admin` role.
|
|
* SSH access to an active master host.
|
|
|
|
.Procedure
|
|
|
|
. Access an active master host.
|
|
|
|
. View the list of Pods associated with etcd:
|
|
+
|
|
----
|
|
[core@ip-10-0-128-73 ~]$ oc get pods -n openshift-etcd
|
|
NAME READY STATUS RESTARTS AGE
|
|
etcd-member-ip-10-0-128-73.us-east-2.compute.internal 2/2 Running 0 15h
|
|
etcd-member-ip-10-0-147-172.us-east-2.compute.internal 2/2 Running 7 122m
|
|
etcd-member-ip-10-0-171-108.us-east-2.compute.internal 2/2 Running 0 15h
|
|
----
|
|
|
|
. Run the `etcd-member-remove.sh` script and pass in the name of the etcd member to remove:
|
|
+
|
|
----
|
|
[core@ip-10-0-128-73 ~]$ sudo -E etcd-member-remove.sh etcd-member-ip-10-0-147-172.us-east-2.compute.internal
|
|
Downloading etcdctl binary..
|
|
etcdctl version: 3.3.10
|
|
API version: 3.3
|
|
etcd client certs already backed up and available ./assets/backup/
|
|
Member 23e4736df4451b32 removed from cluster 6e25bab1bb556673
|
|
etcd member etcd-member-ip-10-0-147-172.us-east-2.compute.internal with 23e4736df4451b32 successfully removed..
|
|
----
|
|
|
|
. Verify that the etcd member has been successfully removed from the cluster:
|
|
|
|
.. Connect to the running etcd container:
|
|
+
|
|
----
|
|
[core@ip-10-0-128-73 ~] id=$(sudo crictl ps --name etcd-member | awk 'FNR==2{ print $1}') && sudo crictl exec -it $id /bin/sh
|
|
----
|
|
|
|
.. In the etcd container, export the variables needed for connecting to etcd:
|
|
+
|
|
----
|
|
sh-4.3# export ETCDCTL_API=3 ETCDCTL_CACERT=/etc/ssl/etcd/ca.crt ETCDCTL_CERT=$(find /etc/ssl/ -name *peer*crt) ETCDCTL_KEY=$(find /etc/ssl/ -name *peer*key)
|
|
----
|
|
|
|
.. In the etcd container, execute `etcdctl member list` and verify that the removed member is no longer listed:
|
|
+
|
|
----
|
|
sh-4.3# etcdctl member list -w table
|
|
|
|
+------------------+---------+------------------------------------------+------------------------------------------------------------------+---------------------------+
|
|
| ID | STATUS | NAME | PEER ADDRS | CLIENT ADDRS |
|
|
+------------------+---------+------------------------------------------+------------------------------------------------------------------+---------------------------+
|
|
| 29e461db6be4eaaa | started | etcd-member-ip-10-0-128-73.us-east-2.compute.internal | https://etcd-2.clustername.devcluster.openshift.com:2380 | https://10.0.128.73:2379 |
|
|
| cbe982c74cbb42f | started | etcd-member-ip-10-0-171-108.us-east-2.compute.internal | https://etcd-1.clustername.devcluster.openshift.com:2380 | https://10.0.171.108:2379 |
|
|
+------------------+---------+------------------------------------------+------------------------------------------------------------------+---------------------------+
|
|
----
|