// Module included in the following assemblies:
//
// * etcd/etcd-performance.adoc

:_mod-docs-content-type: PROCEDURE
[id="etcd-increase-db_{context}"]
= Increasing the database size for etcd

You can set the disk quota in gibibytes (GiB) for each etcd instance. If you set a disk quota for your etcd instance, you can specify integer values from 8 to 32. The default value is 8. You can specify only increasing values.

You might want to increase the disk quota if you encounter a `low space` alert. This alert indicates that the cluster is too large to fit in etcd despite automatic compaction and defragmentation. If you see this alert, you need to increase the disk quota immediately because after etcd runs out of space, writes fail.

Another scenario where you might want to increase the disk quota is if you encounter an `excessive database growth` alert. This alert is a warning that the database might grow too large in the next four hours. In this scenario, consider increasing the disk quota so that you do not eventually encounter a `low space` alert and possible write fails. 

If you increase the disk quota, the disk space that you specify is not immediately reserved. Instead, etcd can grow to that size if needed. Ensure that etcd is running on a dedicated disk that is larger than the value that you specify for the disk quota. 

For large etcd databases, the control plane nodes must have additional memory and storage. Because you must account for the API server cache, the minimum memory required is at least three times the configured size of the etcd database.

:FeatureName: Increasing the database size for etcd
include::snippets/technology-preview.adoc[]

[id="etcd-change-db-size_{context}"]
== Changing the etcd database size

To change the database size for etcd, complete the following steps.

.Procedure

. Check the current value of the disk quota for each etcd instance by entering the following command:
+
[source,terminal]
----
$ oc describe etcd/cluster | grep "Backend Quota"
----
+
.Example output
[source,terminal]
----
Backend Quota Gi B: <value>
----

. Change the value of the disk quota by entering the following command:
+
[source,terminal]
----
$ oc patch etcd/cluster --type=merge -p '{"spec": {"backendQuotaGiB": <value>}}'
----
+
.Example output
[source,terminal]
----
etcd.operator.openshift.io/cluster patched
----

.Verification

. Verify that the new value for the disk quota is set by entering the following command:
+
[source,terminal]
----
$ oc describe etcd/cluster | grep "Backend Quota"
----
+
The etcd Operator automatically rolls out the etcd instances with the new values. 

. Verify that the etcd pods are up and running by entering the following command:
+
[source,terminal]
----
$ oc get pods -n openshift-etcd
----
+
The following output shows the expected entries.
+
.Example output
[source,terminal]
----
NAME                                                   READY   STATUS      RESTARTS   AGE
etcd-ci-ln-b6kfsw2-72292-mzwbq-master-0                4/4     Running     0          39m
etcd-ci-ln-b6kfsw2-72292-mzwbq-master-1                4/4     Running     0          37m
etcd-ci-ln-b6kfsw2-72292-mzwbq-master-2                4/4     Running     0          41m
etcd-guard-ci-ln-b6kfsw2-72292-mzwbq-master-0          1/1     Running     0          51m
etcd-guard-ci-ln-b6kfsw2-72292-mzwbq-master-1          1/1     Running     0          49m
etcd-guard-ci-ln-b6kfsw2-72292-mzwbq-master-2          1/1     Running     0          54m
installer-5-ci-ln-b6kfsw2-72292-mzwbq-master-1         0/1     Completed   0          51m
installer-7-ci-ln-b6kfsw2-72292-mzwbq-master-0         0/1     Completed   0          46m
installer-7-ci-ln-b6kfsw2-72292-mzwbq-master-1         0/1     Completed   0          44m
installer-7-ci-ln-b6kfsw2-72292-mzwbq-master-2         0/1     Completed   0          49m
installer-8-ci-ln-b6kfsw2-72292-mzwbq-master-0         0/1     Completed   0          40m
installer-8-ci-ln-b6kfsw2-72292-mzwbq-master-1         0/1     Completed   0          38m
installer-8-ci-ln-b6kfsw2-72292-mzwbq-master-2         0/1     Completed   0          42m
revision-pruner-7-ci-ln-b6kfsw2-72292-mzwbq-master-0   0/1     Completed   0          43m
revision-pruner-7-ci-ln-b6kfsw2-72292-mzwbq-master-1   0/1     Completed   0          43m
revision-pruner-7-ci-ln-b6kfsw2-72292-mzwbq-master-2   0/1     Completed   0          43m
revision-pruner-8-ci-ln-b6kfsw2-72292-mzwbq-master-0   0/1     Completed   0          42m
revision-pruner-8-ci-ln-b6kfsw2-72292-mzwbq-master-1   0/1     Completed   0          42m
revision-pruner-8-ci-ln-b6kfsw2-72292-mzwbq-master-2   0/1     Completed   0          42m
----

. Verify that the disk quota value is updated for the etcd pod by entering the following command:
+
[source,terminal]
----
$ oc describe -n openshift-etcd pod/<etcd_podname> | grep "ETCD_QUOTA_BACKEND_BYTES"
----
+
The value might not have changed from the default value of `8`. 
+
.Example output
[source,terminal]
----
ETCD_QUOTA_BACKEND_BYTES:                               8589934592
----
+
[NOTE]
=====
While the value that you set is an integer in GiB, the value shown in the output is converted to bytes.
=====


[id="etcd-change-db-size-troubleshooting_{context}"]
== Troubleshooting

If you encounter issues when you try to increase the database size for etcd, the following troubleshooting steps might help.

[id="etcd-troubleshoot-small-value_{context}"]
=== Value is too small

If the value that you specify is less than `8`, you see the following error message:

[source,terminal]
----
$ oc patch etcd/cluster --type=merge -p '{"spec": {"backendQuotaGiB": 5}}'
----

.Example error message
[source,terminal]
----
The Etcd "cluster" is invalid: 
* spec.backendQuotaGiB: Invalid value: 5: spec.backendQuotaGiB in body should be greater than or equal to 8
* spec.backendQuotaGiB: Invalid value: "integer": etcd backendQuotaGiB may not be decreased
----

To resolve this issue, specify an integer between `8` and `32`.

[id="etcd-troubleshoot-large-value_{context}"]
=== Value is too large

If the value that you specify is greater than `32`, you see the following error message:

[source,terminal]
----
$ oc patch etcd/cluster --type=merge -p '{"spec": {"backendQuotaGiB": 64}}'
----

.Example error message
[source,terminal]
----
The Etcd "cluster" is invalid: spec.backendQuotaGiB: Invalid value: 64: spec.backendQuotaGiB in body should be less than or equal to 32
----

To resolve this issue, specify an integer between `8` and `32`.

[id="etcd-troubleshoot-decreasing-value_{context}"]
=== Value is decreasing

If the value is set to a valid value between `8` and `32`, you cannot decrease the value. Otherwise, you see an error message.

. Check to see the current value by entering the following command:
+
[source,terminal]
----
$ oc describe etcd/cluster | grep "Backend Quota"
----
+
.Example output
[source,terminal]
----
Backend Quota Gi B: 10
----

. Decrease the disk quota value by entering the following command:
+
[source,terminal]
----
$ oc patch etcd/cluster --type=merge -p '{"spec": {"backendQuotaGiB": 8}}'
----
+
.Example error message
[source,terminal]
----
The Etcd "cluster" is invalid: spec.backendQuotaGiB: Invalid value: "integer": etcd backendQuotaGiB may not be decreased
----

. To resolve this issue, specify an integer greater than `10`.
