openshift/openshift-docs
1
0
Fork 0
mirror of https://github.com/openshift/openshift-docs.git synced 2026-02-05 12:46:18 +01:00
Code Issues Releases Activity

OBSDOCS-1478: Ability to modify UWM Prometheus scrape interval

Browse Source
This commit is contained in:
Eliska Romanova
2024-12-02 11:33:17 +01:00
committed by openshift-cherrypick-robot
parent 79892428b0
commit b35e704e5d
5 changed files with 35 additions and 31 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
modules/monitoring-limiting-scrape-samples-in-user-defined-projects.adoc
View File

@@ -20,6 +20,7 @@ can use the following measures to control the impact of unbound metrics attribut
* Limit the number of samples that can be accepted per target scrape in user-defined projects
* Limit the number of scraped labels, the length of label names, and the length of label values
* Configure the intervals between consecutive scrapes and between Prometheus rule evaluations
* Create alerts that fire when a scrape sample threshold is reached or when the target cannot be scraped
[NOTE]

57
modules/monitoring-setting-scrape-sample-and-label-limits-for-user-defined-projects.adoc → modules/monitoring-setting-scrape-and-evaluation-intervals-limits-for-user-defined-projects.adoc
View File

@@ -3,10 +3,16 @@
// * observability/monitoring/configuring-the-monitoring-stack.adoc
:_mod-docs-content-type: PROCEDURE
[id="setting-scrape-sample-and-label-limits-for-user-defined-projects_{context}"]
= Setting scrape sample and label limits for user-defined projects
[id="setting-scrape-and-evaluation-intervals-limits-for-user-defined-projects_{context}"]
= Setting scrape intervals, evaluation intervals, and enforced limits for user-defined projects
You can limit the number of samples that can be accepted per target scrape in user-defined projects. You can also limit the number of scraped labels, the length of label names, and the length of label values.
You can set the following scrape and label limits for user-defined projects:
* Limit the number of samples that can be accepted per target scrape
* Limit the number of scraped labels
* Limit the length of label names and label values
You can also set an interval between consecutive scrapes and between Prometheus rule evaluations.
[WARNING]
====
@@ -34,7 +40,7 @@ endif::openshift-dedicated,openshift-rosa[]
$ oc -n openshift-user-workload-monitoring edit configmap user-workload-monitoring-config
----
. Add the `enforcedSampleLimit` configuration to `data/config.yaml` to limit the number of samples that can be accepted per target scrape in user-defined projects:
. Add the enforced limit and time interval configurations to `data/config.yaml`:
+
[source,yaml]
----
@@ -46,31 +52,28 @@ metadata:
data:
config.yaml: |
prometheus:
enforcedSampleLimit: 50000 <1>
enforcedSampleLimit: 50000 # <1>
enforcedLabelLimit: 500 # <2>
enforcedLabelNameLengthLimit: 50 # <3>
enforcedLabelValueLengthLimit: 600 # <4>
scrapeInterval: 1m30s # <5>
evaluationInterval: 1m15s # <6>
----
<1> A value is required if this parameter is specified. This `enforcedSampleLimit` example limits the number of samples that can be accepted per target scrape in user-defined projects to 50,000.
. Add the `enforcedLabelLimit`, `enforcedLabelNameLengthLimit`, and `enforcedLabelValueLengthLimit` configurations to `data/config.yaml` to limit the number of scraped labels, the length of label names, and the length of label values in user-defined projects:
<2> Specifies the maximum number of labels per scrape.
The default value is `0`, which specifies no limit.
<3> Specifies the maximum character length for a label name.
The default value is `0`, which specifies no limit.
<4> Specifies the maximum character length for a label value.
The default value is `0`, which specifies no limit.
<5> Specifies the interval between consecutive scrapes. The interval must be set between 5 seconds and 5 minutes.
The default value is `30s`.
<6> Specifies the interval between Prometheus rule evaluations. The interval must be set between 5 seconds and 5 minutes.
The default value for Prometheus is `30s`.
+
[source,yaml]
----
apiVersion: v1
kind: ConfigMap
metadata:
name: user-workload-monitoring-config
namespace: openshift-user-workload-monitoring
data:
config.yaml: |
prometheus:
enforcedLabelLimit: 500 <1>
enforcedLabelNameLengthLimit: 50 <2>
enforcedLabelValueLengthLimit: 600 <3>
----
<1> Specifies the maximum number of labels per scrape.
The default value is `0`, which specifies no limit.
<2> Specifies the maximum length in characters of a label name.
The default value is `0`, which specifies no limit.
<3> Specifies the maximum length in characters of a label value.
The default value is `0`, which specifies no limit.
[NOTE]
====
You can also configure the `evaluationInterval` property for Thanos Ruler through the `data/config.yaml/thanosRuler` field. The default value for Thanos Ruler is `15s`.
====
. Save the file to apply the changes. The limits are applied automatically.

4
observability/monitoring/configuring-the-monitoring-stack.adoc
View File

@@ -248,9 +248,9 @@ include::modules/monitoring-choosing-a-metrics-collection-profile.adoc[leveloffs
* See xref:../../nodes/clusters/nodes-cluster-enabling-features.adoc[Enabling features using feature gates] for steps to enable Technology Preview features.
endif::openshift-dedicated,openshift-rosa[]
// Managing scrape sample limits for user-defined projects
// Managing scrape and evaluation intervals and enforced limits for user-defined projects
include::modules/monitoring-limiting-scrape-samples-in-user-defined-projects.adoc[leveloffset=+1]
include::modules/monitoring-setting-scrape-sample-and-label-limits-for-user-defined-projects.adoc[leveloffset=+2]
include::modules/monitoring-setting-scrape-and-evaluation-intervals-limits-for-user-defined-projects.adoc[leveloffset=+2]
ifndef::openshift-dedicated,openshift-rosa[]
include::modules/monitoring-creating-scrape-sample-alerts.adoc[leveloffset=+2]

2
observability/monitoring/troubleshooting-monitoring-issues.adoc
View File

@@ -37,7 +37,7 @@ include::modules/monitoring-determining-why-prometheus-is-consuming-disk-space.a
.Additional resources
* xref:../../observability/monitoring/accessing-third-party-monitoring-apis.adoc#about-accessing-monitoring-web-service-apis_accessing-monitoring-apis-by-using-the-cli[Accessing monitoring APIs by using the CLI]
* xref:../../observability/monitoring/configuring-the-monitoring-stack.adoc#setting-scrape-sample-and-label-limits-for-user-defined-projects_configuring-the-monitoring-stack[Setting a scrape sample limit for user-defined projects]
* xref:../../observability/monitoring/configuring-the-monitoring-stack.adoc#setting-scrape-and-evaluation-intervals-limits-for-user-defined-projects_configuring-the-monitoring-stack[Setting scrape intervals, evaluation intervals, and scrape limits for user-defined projects]
* xref:../../support/getting-support.adoc#support-submitting-a-case_getting-support[Submitting a support case]
// Resolving the KubePersistentVolumeFillingUp alert firing for Prometheus

2
support/troubleshooting/investigating-monitoring-issues.adoc
View File

@@ -31,7 +31,7 @@ include::modules/monitoring-determining-why-prometheus-is-consuming-disk-space.a
[role="_additional-resources"]
.Additional resources
* See xref:../../observability/monitoring/configuring-the-monitoring-stack.adoc#setting-scrape-sample-and-label-limits-for-user-defined-projects_configuring-the-monitoring-stack[Setting a scrape sample limit for user-defined projects] for details on how to set a scrape sample limit and create related alerting rules
* xref:../../observability/monitoring/configuring-the-monitoring-stack.adoc#setting-scrape-and-evaluation-intervals-limits-for-user-defined-projects_configuring-the-monitoring-stack[Setting scrape and evaluation intervals and enforced limits for user-defined projects]
// Resolving the KubePersistentVolumeFillingUp alert firing for Prometheus
include::modules/monitoring-resolving-the-kubepersistentvolumefillingup-alert-firing-for-prometheus.adoc[leveloffset=+1]