mirror of
https://github.com/openshift/openshift-docs.git
synced 2026-02-05 12:46:18 +01:00
78 lines
2.0 KiB
Plaintext
78 lines
2.0 KiB
Plaintext
// Module included in the following assemblies:
|
|
//
|
|
// * microshift_ai/microshift-rhoai.adoc
|
|
|
|
:_mod-docs-content-type: PROCEDURE
|
|
[id="microshift-rhoai-query-model_{context}"]
|
|
= Querying your AI model
|
|
|
|
Make an inference request against the AI model server that is using the `ovms-resnet50` model.
|
|
|
|
.Prerequisites
|
|
|
|
* {microshift-short} is running.
|
|
* You configured the model-serving runtime.
|
|
* You uploaded your AI model to {microshift-short}.
|
|
|
|
.Procedure
|
|
|
|
* Make an inference request against the model server that is using the `ovms-resnet50` model by running the following command:
|
|
+
|
|
[source,terminal]
|
|
----
|
|
$ curl \
|
|
--data-binary "@./request.json" \
|
|
--header "Inference-Header-Content-Length: ${HEADER_LEN}" \
|
|
"${DOMAIN}/v2/models/ovms-resnet50/infer" \
|
|
--connect-to "${DOMAIN}::${IP}:" > response.json
|
|
----
|
|
+
|
|
.Example inferencing output, saved to a `response.json`
|
|
+
|
|
[source,json]
|
|
----
|
|
{
|
|
"model_name": "ovms-resnet50",
|
|
"model_version": "1",
|
|
"outputs": [{
|
|
"name": "1463",
|
|
"shape": [1, 1000],
|
|
"datatype": "FP32",
|
|
"data": [ ....... ] <1>
|
|
}]
|
|
}
|
|
----
|
|
<1> The contents of `.outputs[0].data` were omitted from the example for brevity.
|
|
|
|
.Verification
|
|
|
|
. To determine the model's prediction, get the index of the highest element in the `.outputs[0].data` to determine the model's predicted value by using the following Python script:
|
|
+
|
|
[source,python]
|
|
----
|
|
import json
|
|
with open('response.json') as f:
|
|
response = json.load(f)
|
|
data = response["outputs"][0]["data"]
|
|
argmax = data.index(max(data))
|
|
print(argmax)
|
|
----
|
|
+
|
|
.Example output
|
|
[source,text]
|
|
----
|
|
309 <1>
|
|
----
|
|
<1> In this example, the element labeled `309` is the model's response.
|
|
|
|
. Validate the output against link:https://github.com/openvinotoolkit/model_server/blob/main/client/common/resnet_input_images.txt[resnet's input data], for example:
|
|
+
|
|
[source,text]
|
|
----
|
|
../../../../demos/common/static/images/bee.jpeg 309
|
|
----
|
|
|
|
.Next steps
|
|
|
|
* Optional. Query the AI model using other images available in the resnet input data.
|