mirror of
https://github.com/openshift/openshift-docs.git
synced 2026-02-05 12:46:18 +01:00
335 lines
9.5 KiB
Plaintext
335 lines
9.5 KiB
Plaintext
// Module included in the following assemblies:
|
|
//
|
|
// ../scalability_and_performance/compute-resource-quotas.adoc
|
|
|
|
:_mod-docs-content-type: CONCEPT
|
|
[id="admin-quota-overview_{context}"]
|
|
= Resources managed by quota
|
|
|
|
A resource quota, defined by a `ResourceQuota` object, provides constraints that limit aggregate resource consumption per project. It can limit the quantity
|
|
of objects that can be created in a project by type, as well as the total amount of compute resources and storage that may be consumed by resources in that project.
|
|
|
|
The following describes the set of compute resources and object types that may be managed by a quota.
|
|
|
|
[NOTE]
|
|
====
|
|
A pod is in a terminal state if `status.phase` is `Failed` or `Succeeded`.
|
|
====
|
|
|
|
.Compute resources managed by quota
|
|
[cols="3a,8a",options="header"]
|
|
|===
|
|
|
|
|Resource Name |Description
|
|
|
|
|`cpu`
|
|
|The sum of CPU requests across all pods in a non-terminal state cannot exceed this value. `cpu` and `requests.cpu` are the same value and can be used
|
|
interchangeably.
|
|
|
|
|`memory`
|
|
|The sum of memory requests across all pods in a non-terminal state cannot exceed this value. `memory` and `requests.memory` are the same value and can
|
|
be used interchangeably.
|
|
|
|
|`ephemeral-storage`
|
|
|The sum of local ephemeral storage requests across all pods in a non-terminal state cannot exceed this value. `ephemeral-storage` and
|
|
`requests.ephemeral-storage` are the same value and can be used interchangeably. This resource is available only if you enabled the ephemeral storage technology preview. This feature is disabled by default.
|
|
|
|
|`requests.cpu`
|
|
|The sum of CPU requests across all pods in a non-terminal state cannot exceed this value. `cpu` and `requests.cpu` are the same value and can be used
|
|
interchangeably.
|
|
|
|
|`requests.memory`
|
|
|The sum of memory requests across all pods in a non-terminal state cannot exceed this value. `memory` and `requests.memory` are the same value and can
|
|
be used interchangeably.
|
|
|
|
|`requests.ephemeral-storage`
|
|
|The sum of ephemeral storage requests across all pods in a non-terminal state cannot exceed this value. `ephemeral-storage` and
|
|
`requests.ephemeral-storage` are the same value and can be used interchangeably. This resource is available only if you enabled the ephemeral
|
|
storage technology preview. This feature is disabled by default.
|
|
|
|
|`limits.cpu`
|
|
|The sum of CPU limits across all pods in a non-terminal state cannot exceed this value.
|
|
|
|
|`limits.memory`
|
|
|The sum of memory limits across all pods in a non-terminal state cannot exceed this value.
|
|
|
|
|`limits.ephemeral-storage`
|
|
|The sum of ephemeral storage limits across all pods in a non-terminal state cannot exceed this value. This resource is available only if you enabled the
|
|
ephemeral storage technology preview. This feature is disabled by default.
|
|
|===
|
|
|
|
|
|
.Storage resources managed by quota
|
|
[cols="3a,8a",options="header"]
|
|
|===
|
|
|
|
|Resource Name |Description
|
|
|
|
|`requests.storage`
|
|
|The sum of storage requests across all persistent volume claims in any state cannot exceed this value.
|
|
|
|
|`persistentvolumeclaims`
|
|
|The total number of persistent volume claims that can exist in the project.
|
|
|
|
|`<storage-class-name>.storageclass.storage.k8s.io/requests.storage`
|
|
|The sum of storage requests across all persistent volume claims in any state that have a matching storage class, cannot exceed this value.
|
|
|
|
|`<storage-class-name>.storageclass.storage.k8s.io/persistentvolumeclaims`
|
|
|The total number of persistent volume claims with a matching storage class that can exist in the project.
|
|
|===
|
|
|
|
|
|
.Object counts managed by quota
|
|
[cols="3a,8a",options="header"]
|
|
|===
|
|
|
|
|Resource Name |Description
|
|
|
|
|`pods`
|
|
|The total number of pods in a non-terminal state that can exist in the project.
|
|
|
|
|`replicationcontrollers`
|
|
|The total number of replication controllers that can exist in the project.
|
|
|
|
|`resourcequotas`
|
|
|The total number of resource quotas that can exist in the project.
|
|
|
|
|`services`
|
|
|The total number of services that can exist in the project.
|
|
|
|
|`secrets`
|
|
|The total number of secrets that can exist in the project.
|
|
|
|
|`configmaps`
|
|
|The total number of `ConfigMap` objects that can exist in the project.
|
|
|
|
|`persistentvolumeclaims`
|
|
|The total number of persistent volume claims that can exist in the project.
|
|
|
|
|`openshift.io/imagestreams`
|
|
|The total number of image streams that can exist in the project.
|
|
|
|
|===
|
|
|
|
You can configure an object count quota for these standard namespaced resource types using the `count/<resource>.<group>` syntax.
|
|
|
|
[source,terminal]
|
|
----
|
|
$ oc create quota <name> --hard=count/<resource>.<group>=<quota> <1>
|
|
----
|
|
|
|
<1> `<resource>` is the name of the resource, and `<group>` is the API group, if applicable.
|
|
Use the `kubectl api-resources` command for a list of resources and their associated API groups.
|
|
|
|
== Setting resource quota for extended resources
|
|
|
|
Overcommitment of resources is not allowed for extended resources, so you must specify `requests` and `limits` for the same extended resource in a quota. Currently, only quota items with the prefix `requests.` are allowed for extended resources. The following is an example scenario of how to set resource quota for the GPU resource `nvidia.com/gpu`.
|
|
|
|
.Procedure
|
|
|
|
. To determine how many GPUs are available on a node in your cluster, use the following command:
|
|
+
|
|
[source,terminal]
|
|
----
|
|
$ oc describe node ip-172-31-27-209.us-west-2.compute.internal | egrep 'Capacity|Allocatable|gpu'
|
|
----
|
|
+
|
|
.Example output
|
|
|
|
[source,terminal]
|
|
----
|
|
|
|
openshift.com/gpu-accelerator=true
|
|
Capacity:
|
|
nvidia.com/gpu: 2
|
|
Allocatable:
|
|
nvidia.com/gpu: 2
|
|
nvidia.com/gpu: 0 0
|
|
----
|
|
+
|
|
In this example, 2 GPUs are available.
|
|
|
|
. Use this command to set a quota in the namespace `nvidia`. In this example, the quota is `1`:
|
|
+
|
|
[source,terminal]
|
|
----
|
|
$ cat gpu-quota.yaml
|
|
----
|
|
+
|
|
.Example output
|
|
|
|
[source,terminal]
|
|
----
|
|
apiVersion: v1
|
|
kind: ResourceQuota
|
|
metadata:
|
|
name: gpu-quota
|
|
namespace: nvidia
|
|
spec:
|
|
hard:
|
|
requests.nvidia.com/gpu: 1
|
|
|
|
----
|
|
. Create the quota with the following command:
|
|
+
|
|
[source,terminal]
|
|
----
|
|
$ oc create -f gpu-quota.yaml
|
|
----
|
|
+
|
|
.Example output
|
|
|
|
[source,terminal]
|
|
----
|
|
resourcequota/gpu-quota created
|
|
----
|
|
. Verify that the namespace has the correct quota set using the following command:
|
|
+
|
|
[source,terminal]
|
|
----
|
|
$ oc describe quota gpu-quota -n nvidia
|
|
----
|
|
+
|
|
.Example output
|
|
|
|
[source,terminal]
|
|
----
|
|
Name: gpu-quota
|
|
Namespace: nvidia
|
|
Resource Used Hard
|
|
-------- ---- ----
|
|
requests.nvidia.com/gpu 0 1
|
|
----
|
|
|
|
. Run a pod that asks for a single GPU with the following command:
|
|
+
|
|
[source,terminal]
|
|
----
|
|
$ oc create pod gpu-pod.yaml
|
|
----
|
|
+
|
|
.Example output
|
|
|
|
[source, terminal]
|
|
----
|
|
apiVersion: v1
|
|
kind: Pod
|
|
metadata:
|
|
generateName: gpu-pod-s46h7
|
|
namespace: nvidia
|
|
spec:
|
|
restartPolicy: OnFailure
|
|
containers:
|
|
- name: rhel7-gpu-pod
|
|
image: rhel7
|
|
env:
|
|
- name: NVIDIA_VISIBLE_DEVICES
|
|
value: all
|
|
- name: NVIDIA_DRIVER_CAPABILITIES
|
|
value: "compute,utility"
|
|
- name: NVIDIA_REQUIRE_CUDA
|
|
value: "cuda>=5.0"
|
|
|
|
command: ["sleep"]
|
|
args: ["infinity"]
|
|
|
|
resources:
|
|
limits:
|
|
nvidia.com/gpu: 1
|
|
----
|
|
|
|
. Verify that the pod is running bwith the following command:
|
|
+
|
|
[source,terminal]
|
|
----
|
|
$ oc get pods
|
|
----
|
|
+
|
|
.Example output
|
|
|
|
[source, terminal]
|
|
----
|
|
NAME READY STATUS RESTARTS AGE
|
|
gpu-pod-s46h7 1/1 Running 0 1m
|
|
----
|
|
|
|
. Verify that the quota `Used` counter is correct by running the following command:
|
|
+
|
|
[source,terminal]
|
|
----
|
|
$ oc describe quota gpu-quota -n nvidia
|
|
----
|
|
+
|
|
.Example output
|
|
|
|
[source, terminal]
|
|
----
|
|
Name: gpu-quota
|
|
Namespace: nvidia
|
|
Resource Used Hard
|
|
-------- ---- ----
|
|
requests.nvidia.com/gpu 1 1
|
|
----
|
|
|
|
. Using the following command, attempt to create a second GPU pod in the `nvidia` namespace. This is technically available on the node because it has 2 GPUs:
|
|
+
|
|
[source,terminal]
|
|
----
|
|
$ oc create -f gpu-pod.yaml
|
|
----
|
|
+
|
|
.Example output
|
|
|
|
[source, terminal]
|
|
----
|
|
Error from server (Forbidden): error when creating "gpu-pod.yaml": pods "gpu-pod-f7z2w" is forbidden: exceeded quota: gpu-quota, requested: requests.nvidia.com/gpu=1, used: requests.nvidia.com/gpu=1, limited: requests.nvidia.com/gpu=1
|
|
----
|
|
+
|
|
This `Forbidden` error message occurs because you have a quota of 1 GPU and this pod tried to allocate a second GPU, which exceeds its quota.
|
|
|
|
== Quota scopes
|
|
|
|
Each quota can have an associated set of _scopes_. A quota only measures usage for a resource if it matches the intersection of enumerated scopes.
|
|
|
|
Adding a scope to a quota restricts the set of resources to which that quota can apply. Specifying a resource outside of the allowed set results in a validation error.
|
|
|
|
[cols="3a,8a",options="header"]
|
|
|===
|
|
|
|
|Scope |Description
|
|
|
|
|`Terminating`
|
|
|Match pods where `spec.activeDeadlineSeconds >= 0`.
|
|
|
|
|`NotTerminating`
|
|
|Match pods where `spec.activeDeadlineSeconds` is `nil`.
|
|
|
|
|`BestEffort`
|
|
|Match pods that have best effort quality of service for either `cpu` or `memory`.
|
|
|
|
|`otBestEffort`
|
|
|Match pods that do not have best effort quality of service for `cpu` and `memory`.
|
|
|===
|
|
|
|
A `BestEffort` scope restricts a quota to limiting the following resources:
|
|
|
|
- `pods`
|
|
|
|
A `Terminating`, `NotTerminating`, and `NotBestEffort` scope restricts a quota to tracking the following resources:
|
|
|
|
* `pods`
|
|
* `memory`
|
|
* `requests.memory`
|
|
* `limits.memory`
|
|
* `cpu`
|
|
* `requests.cpu`
|
|
* `limits.cpu`
|
|
* `ephemeral-storage`
|
|
* `requests.ephemeral-storage`
|
|
* `limits.ephemeral-storage`
|
|
|
|
[NOTE]
|
|
====
|
|
Ephemeral storage requests and limits apply only if you enabled the ephemeral storage technology preview. This feature is disabled by default.
|
|
====
|