Deploying a model on a multi-model server

OpenShift AI multi-model servers can host several models at once. You create a new model server and deploy your model to it.

Prerequiste

  • A user with admin privileges has enabled the multi-model serving platform on your OpenShift cluster.

Procedure

  1. In the OpenShift AI dashboard, navigate to the project details page and click the Models tab.

    Models

  2. In the Multi-model serving platform tile, click Add model server.

  3. In the form, provide the following values:

    1. For Model server name, type a name, for example Model Server.

    2. For Serving runtime, select OpenVINO Model Server.

    3. Leave the other fields with the default settings.

      Create model server form

  4. Click Add.

  5. In the Models and model servers list, next to the new model server, click Deploy model.

    Create model server form

  6. In the form, provide the following values:

    1. For Model Name, type fraud.

    2. For Model framework, select onnx-1.

    3. For Existing data connection, select My Storage.

    4. Type the path that leads to the version folder that contains your model file: models/fraud

    5. Leave the other fields with the default settings.

      Deploy model from for multi-model serving

  7. Click Deploy.

Verification

Wait for the model to deploy and for the Status to show a green checkmark.

Deployed model status
Next step

Testing the model API