mirror of
https://github.com/openshift/openshift-docs.git
synced 2026-02-05 12:46:18 +01:00
76 lines
2.5 KiB
Plaintext
76 lines
2.5 KiB
Plaintext
// Module included in the following assemblies:
|
|
//
|
|
// * microshift_ai/microshift-rhoai.adoc
|
|
|
|
:_mod-docs-content-type: PROCEDURE
|
|
[id="microshift-rhoai-servingruntimes-ex_{context}"]
|
|
= Creating a ServingRuntime CR for use in {microshift-short}
|
|
|
|
Create a `ServingRuntime` custom resource (CR) based on installed manifests and release information. The included steps are an example of reusing the included `microshift-ai-model-serving` manifest files to re-create the {ovms} ({ov}) model-serving runtime in the workload namespace.
|
|
|
|
[NOTE]
|
|
====
|
|
This approach does not require a live node, so it can be part of CI/CD automation.
|
|
====
|
|
|
|
.Prerequisites
|
|
|
|
* Both the `microshift-ai-model-serving` and `microshift-ai-model-serving-release-info` RPMs are installed.
|
|
* You have root user access to your machine.
|
|
* The {oc-first} is installed.
|
|
|
|
.Procedure
|
|
|
|
. Extract the image reference of the `ServingRuntime` CR you want to use from the {microshift-short} release information file by running the following command:
|
|
+
|
|
[source,terminal]
|
|
----
|
|
$ OVMS_IMAGE="$(jq -r '.images | with_entries(select(.key == "ovms-image")) | .[]' /usr/share/microshift/release/release-ai-model-serving-"$(uname -i)".json)" <1>
|
|
----
|
|
<1> In this example, the image reference for the {ov} model-serving runtime is extracted.
|
|
|
|
. Copy the original `ServingRuntime` YAML file by running the following command:
|
|
+
|
|
[source,terminal]
|
|
----
|
|
$ cp /usr/lib/microshift/manifests.d/050-microshift-ai-model-serving-runtimes/ovms-kserve.yaml ./ovms-kserve.yaml
|
|
----
|
|
|
|
. Add the actual image reference to the `image:` parameter field value of the `ServingRuntime` YAML by running the following command:
|
|
+
|
|
[source,terminal]
|
|
----
|
|
$ sed -i "s,image: ovms-image,image: ${OVMS_IMAGE}," ./ovms-kserve.yaml
|
|
----
|
|
|
|
. Create the `ServingRuntime` object in a custom namespace using the YAML file by running the following command:
|
|
+
|
|
[source,terminal,subs="+quotes"]
|
|
----
|
|
$ oc create -n _<ai_demo>_ -f ./ovms-kserve.yaml <1>
|
|
----
|
|
<1> Replace `_<ai_demo>_` with the name of your namespace.
|
|
|
|
[IMPORTANT]
|
|
====
|
|
If the `ServingRuntime` CR is part of a new manifest, set the namespace in the `kustomization.yaml` file, for example:
|
|
|
|
.Example Kustomize manifest namespace value
|
|
[source,yaml]
|
|
----
|
|
apiVersion: kustomize.config.k8s.io/v1beta1
|
|
kind: Kustomization
|
|
namespace: ai-demo
|
|
resources:
|
|
- ovms-kserve.yaml
|
|
#...
|
|
----
|
|
====
|
|
|
|
.Next steps
|
|
|
|
* Create the `InferenceService` object.
|
|
* Verify that your model is ready for inferencing.
|
|
* Query the model.
|
|
* Optional: examine the model metrics.
|