mirror of
https://github.com/openshift/openshift-docs.git
synced 2026-02-07 09:46:53 +01:00
217 lines
8.4 KiB
Plaintext
217 lines
8.4 KiB
Plaintext
// Module included in the following assemblies:
|
|
//
|
|
// * observability/monitoring/configuring-the-monitoring-stack.adoc
|
|
|
|
:_mod-docs-content-type: PROCEDURE
|
|
[id="configuring_remote_write_storage_{context}"]
|
|
= Configuring remote write storage
|
|
|
|
[role="_abstract"]
|
|
You can configure remote write storage to enable Prometheus to send ingested metrics to remote systems for long-term storage. Doing so has no impact on how or for how long Prometheus stores metrics.
|
|
|
|
.Prerequisites
|
|
|
|
ifndef::openshift-dedicated,openshift-rosa[]
|
|
* *If you are configuring core {product-title} monitoring components:*
|
|
** You have access to the cluster as a user with the `cluster-admin` cluster role.
|
|
** You have created the `cluster-monitoring-config` `ConfigMap` object.
|
|
* *If you are configuring components that monitor user-defined projects:*
|
|
** You have access to the cluster as a user with the `cluster-admin` cluster role or as a user with the `user-workload-monitoring-config-edit` role in the `openshift-user-workload-monitoring` project.
|
|
** You have created the `user-workload-monitoring-config` `ConfigMap` object.
|
|
endif::openshift-dedicated,openshift-rosa[]
|
|
ifdef::openshift-dedicated,openshift-rosa[]
|
|
* You have access to the cluster as a user with the `dedicated-admin` role.
|
|
* The `user-workload-monitoring-config` `ConfigMap` object exists. This object is created by default when the cluster is created.
|
|
endif::openshift-dedicated,openshift-rosa[]
|
|
* You have installed the OpenShift CLI (`oc`).
|
|
* You have set up a remote write compatible endpoint (such as Thanos) and know the endpoint URL. See the link:https://prometheus.io/docs/operating/integrations/#remote-endpoints-and-storage[Prometheus remote endpoints and storage documentation] for information about endpoints that are compatible with the remote write feature.
|
|
+
|
|
[IMPORTANT]
|
|
====
|
|
Red{nbsp}Hat only provides information for configuring remote write senders and does not offer guidance on configuring receiver endpoints. Customers are responsible for setting up their own endpoints that are remote-write compatible. Issues with endpoint receiver configurations are not included in Red{nbsp}Hat production support.
|
|
====
|
|
* You have set up authentication credentials in a `Secret` object for the remote write endpoint. You must create the secret in the
|
|
ifndef::openshift-dedicated,openshift-rosa[]
|
|
same namespace as the Prometheus object for which you configure remote write: the `openshift-monitoring` namespace for default platform monitoring or the `openshift-user-workload-monitoring` namespace for user workload monitoring.
|
|
endif::openshift-dedicated,openshift-rosa[]
|
|
ifdef::openshift-dedicated,openshift-rosa[]
|
|
`openshift-user-workload-monitoring` namespace.
|
|
endif::openshift-dedicated,openshift-rosa[]
|
|
|
|
+
|
|
[WARNING]
|
|
====
|
|
To reduce security risks, use HTTPS and authentication to send metrics to an endpoint.
|
|
====
|
|
|
|
.Procedure
|
|
|
|
. Edit the `ConfigMap` object:
|
|
ifndef::openshift-dedicated,openshift-rosa[]
|
|
** *To configure remote write for the Prometheus instance that monitors core {product-title} projects*:
|
|
.. Edit the `cluster-monitoring-config` `ConfigMap` object in the `openshift-monitoring` project:
|
|
+
|
|
[source,terminal]
|
|
----
|
|
$ oc -n openshift-monitoring edit configmap cluster-monitoring-config
|
|
----
|
|
|
|
.. Add a `remoteWrite:` section under `data/config.yaml/prometheusK8s`.
|
|
|
|
.. Add an endpoint URL and authentication credentials in this section:
|
|
+
|
|
[source,yaml]
|
|
----
|
|
apiVersion: v1
|
|
kind: ConfigMap
|
|
metadata:
|
|
name: cluster-monitoring-config
|
|
namespace: openshift-monitoring
|
|
data:
|
|
config.yaml: |
|
|
prometheusK8s:
|
|
remoteWrite:
|
|
- url: "https://remote-write-endpoint.example.com" <1>
|
|
<endpoint_authentication_credentials> <2>
|
|
----
|
|
+
|
|
<1> The URL of the remote write endpoint.
|
|
<2> The authentication method and credentials for the endpoint.
|
|
Currently supported authentication methods are AWS Signature Version 4, authentication using HTTP in an `Authorization` request header, Basic authentication, OAuth 2.0, and TLS client.
|
|
See _Supported remote write authentication settings_ for sample configurations of supported authentication methods.
|
|
|
|
.. Add write relabel configuration values after the authentication credentials:
|
|
+
|
|
[source,yaml]
|
|
----
|
|
apiVersion: v1
|
|
kind: ConfigMap
|
|
metadata:
|
|
name: cluster-monitoring-config
|
|
namespace: openshift-monitoring
|
|
data:
|
|
config.yaml: |
|
|
prometheusK8s:
|
|
remoteWrite:
|
|
- url: "https://remote-write-endpoint.example.com"
|
|
<endpoint_authentication_credentials>
|
|
<your_write_relabel_configs> <1>
|
|
----
|
|
<1> The write relabel configuration settings.
|
|
+
|
|
For `<your_write_relabel_configs>` substitute a list of write relabel configurations for metrics that you want to send to the remote endpoint.
|
|
+
|
|
The following sample shows how to forward a single metric called `my_metric`:
|
|
+
|
|
[source,yaml]
|
|
----
|
|
apiVersion: v1
|
|
kind: ConfigMap
|
|
metadata:
|
|
name: cluster-monitoring-config
|
|
namespace: openshift-monitoring
|
|
data:
|
|
config.yaml: |
|
|
prometheusK8s:
|
|
remoteWrite:
|
|
- url: "https://remote-write-endpoint.example.com"
|
|
writeRelabelConfigs:
|
|
- sourceLabels: [__name__]
|
|
regex: 'my_metric'
|
|
action: keep
|
|
|
|
----
|
|
+
|
|
See the link:https://prometheus.io/docs/prometheus/latest/configuration/configuration/#relabel_config[Prometheus relabel_config documentation] for information about write relabel configuration options.
|
|
|
|
** *To configure remote write for the Prometheus instance that monitors user-defined projects*:
|
|
endif::openshift-dedicated,openshift-rosa[]
|
|
.. Edit the `user-workload-monitoring-config` `ConfigMap` object in the `openshift-user-workload-monitoring` project:
|
|
+
|
|
[source,terminal]
|
|
----
|
|
$ oc -n openshift-user-workload-monitoring edit configmap user-workload-monitoring-config
|
|
----
|
|
|
|
.. Add a `remoteWrite:` section under `data/config.yaml/prometheus`.
|
|
|
|
.. Add an endpoint URL and authentication credentials in this section:
|
|
+
|
|
[source,yaml]
|
|
----
|
|
apiVersion: v1
|
|
kind: ConfigMap
|
|
metadata:
|
|
name: user-workload-monitoring-config
|
|
namespace: openshift-user-workload-monitoring
|
|
data:
|
|
config.yaml: |
|
|
prometheus:
|
|
remoteWrite:
|
|
- url: "https://remote-write-endpoint.example.com" <1>
|
|
<endpoint_authentication_credentials> <2>
|
|
----
|
|
+
|
|
<1> The URL of the remote write endpoint.
|
|
<2> The authentication method and credentials for the endpoint.
|
|
Currently supported authentication methods are AWS Signature Version 4, authentication using HTTP an `Authorization` request header, basic authentication, OAuth 2.0, and TLS client.
|
|
See _Supported remote write authentication settings_ below for sample configurations of supported authentication methods.
|
|
|
|
.. Add write relabel configuration values after the authentication credentials:
|
|
+
|
|
[source,yaml]
|
|
----
|
|
apiVersion: v1
|
|
kind: ConfigMap
|
|
metadata:
|
|
name: user-workload-monitoring-config
|
|
namespace: openshift-user-workload-monitoring
|
|
data:
|
|
config.yaml: |
|
|
prometheus:
|
|
remoteWrite:
|
|
- url: "https://remote-write-endpoint.example.com"
|
|
<endpoint_authentication_credentials>
|
|
<your_write_relabel_configs> <1>
|
|
----
|
|
<1> The write relabel configuration settings.
|
|
+
|
|
For `<your_write_relabel_configs>` substitute a list of write relabel configurations for metrics that you want to send to the remote endpoint.
|
|
+
|
|
The following sample shows how to forward a single metric called `my_metric`:
|
|
+
|
|
[source,yaml]
|
|
----
|
|
apiVersion: v1
|
|
kind: ConfigMap
|
|
metadata:
|
|
name: user-workload-monitoring-config
|
|
namespace: openshift-user-workload-monitoring
|
|
data:
|
|
config.yaml: |
|
|
prometheus:
|
|
remoteWrite:
|
|
- url: "https://remote-write-endpoint.example.com"
|
|
writeRelabelConfigs:
|
|
- sourceLabels: [__name__]
|
|
regex: 'my_metric'
|
|
action: keep
|
|
|
|
----
|
|
+
|
|
See the link:https://prometheus.io/docs/prometheus/latest/configuration/configuration/#relabel_config[Prometheus relabel_config documentation] for information about write relabel configuration options.
|
|
|
|
. Save the file to apply the changes. The pods affected by the new configuration restart automatically.
|
|
ifndef::openshift-dedicated,openshift-rosa[]
|
|
+
|
|
[NOTE]
|
|
====
|
|
Configurations applied to the `user-workload-monitoring-config` `ConfigMap` object are not activated unless a cluster administrator has enabled monitoring for user-defined projects.
|
|
====
|
|
endif::openshift-dedicated,openshift-rosa[]
|
|
+
|
|
[WARNING]
|
|
====
|
|
Saving changes to a monitoring `ConfigMap` object might redeploy the pods and other resources in the related project. Saving changes might also restart the running monitoring processes in that project.
|
|
====
|