diff --git a/microshift_ai/microshift-rhoai.adoc b/microshift_ai/microshift-rhoai.adoc index bbc2b977ff..c35644235f 100644 --- a/microshift_ai/microshift-rhoai.adoc +++ b/microshift_ai/microshift-rhoai.adoc @@ -64,6 +64,8 @@ include::modules/microshift-rhoai-servingruntimes-ex.adoc[leveloffset=+1] include::modules/microshift-rhoai-inferenceservice-ex.adoc[leveloffset=+1] +include::modules/microshift-rhoai-export-metrics-otel.adoc[leveloffset=+2] + include::modules/microshift-inferenceservice-more-options.adoc[leveloffset=+2] include::modules/microshift-rhoai-model-serving-rt-verify.adoc[leveloffset=+1] diff --git a/modules/microshift-inferenceservice-more-options.adoc b/modules/microshift-inferenceservice-more-options.adoc index 1f949fa906..dae7ff2b58 100644 --- a/modules/microshift-inferenceservice-more-options.adoc +++ b/modules/microshift-inferenceservice-more-options.adoc @@ -2,9 +2,9 @@ // // * microshift_ai/microshift-rhoai.adoc -:_mod-docs-content-type: CONCEPT +:_mod-docs-content-type: REFERENCE [id="microshift-rhoai-inferenceservice-more-options_{context}"] -= More InferenceService CRD options += More InferenceService CR options The inference service YAML file can include many different options. For example, you can include a `resources` section that is passed first to the deployment and then to the pod, so that the model server gets access to your hardware through the device plugin. diff --git a/modules/microshift-rhoai-export-metrics-otel.adoc b/modules/microshift-rhoai-export-metrics-otel.adoc new file mode 100644 index 0000000000..2070ed9132 --- /dev/null +++ b/modules/microshift-rhoai-export-metrics-otel.adoc @@ -0,0 +1,39 @@ +// Module included in the following assemblies: +// +// * microshift_ai/microshift-rhoai.adoc + +:_mod-docs-content-type: PROCEDURE +[id="microshift-rhoai-export-metrics-otel_{context}"] += Exporting model-server metrics by using Open Telemetry + +You can export model-server metrics by using Open Telemetry if you installed the `microshift-observability` RPM for {microshift-short}. + +[NOTE] +==== +You can alternatively get the Prometheus-format metrics of the model server by making a request on the `/metrics` endpoint. See "Getting the model-server metrics" for more information. +==== + +.Prerequisites + +* You configured the `ServingRuntimes` CR. +* You have root user access to your machine. +* The {oc-first} is installed. +* You installed the `microshift-observability` RPM. +* Your {microshift-short} Open Telemetry configuration includes the Prometheus Receiver. For more information, see link:https://docs.redhat.com/en/documentation/openshift_container_platform/4.18/html/red_hat_build_of_opentelemetry/configuring-the-collector#prometheus-receiver_otel-collector-receivers[Prometheus Receiver]. + +.Procedure + +* Add the following Open Telemetry annotation to your `InferenceService` custom resource: ++ +.Example `InferenceService` object with Open Telemetry +[source,yaml] +---- +apiVersion: serving.kserve.io/v1beta1 +kind: InferenceService +metadata: + name: ovms-resnet50 +#... + annotations: + prometheus.io/scrape: "true" +#... +---- diff --git a/modules/microshift-rhoai-get-model-server-metrics.adoc b/modules/microshift-rhoai-get-model-server-metrics.adoc index 46b3708499..5b7b9e98c2 100644 --- a/modules/microshift-rhoai-get-model-server-metrics.adoc +++ b/modules/microshift-rhoai-get-model-server-metrics.adoc @@ -8,6 +8,11 @@ After making a query, you can get the model server's metrics to identify bottlenecks, optimize resource allocation, and ensure efficient infrastructure utilization. +[NOTE] +==== +You can alternatively configure Open Telemetry for {microshift-short} to get model-server metrics. See "Adding Open Telemetry to an InferenceService custom resource" for more information. +==== + .Prerequisites * The {microshift-short} cluster is running.