Deploying a model on a multi-model server

OpenShift AI multi-model servers can host several models at once. You create a new model server and deploy your model to it.

Prerequisites

  • A user with admin privileges has enabled the multi-model serving platform on your OpenShift cluster.

Procedure

  1. In the OpenShift AI dashboard, navigate to the project details page and click the Models tab.

    Models
    Depending on how model serving has been configured on your cluster, you might see only one model serving platform option.

  2. In the Multi-model serving platform tile, click Add model server.

  3. In the form, provide the following values:

    1. For Model server name, type a name, for example Model Server.

    2. For Serving runtime, select OpenVINO Model Server.

    3. Leave the other fields with the default settings.

      Create model server form

  4. Click Add.

  5. In the Models and model servers list, next to the new model server, click Deploy model.

    Create model server form

  6. In the form, provide the following values:

    1. For Model deployment name, type fraud.

    2. For Model framework (name - version), select onnx-1.

    3. For Existing connection, select My Storage.

    4. Type the path that leads to the version folder that contains your model file: models/fraud

    5. Leave the other fields with the default settings.

      Deploy model from for multi-model serving

  7. Click Deploy.

Verification

Notice the loading symbol under the Status section. The symbol changes to a green checkmark when the deployment completes successfully.

Deployed model status
Next step

Testing the model API