RHOAI Models-as-a-Service (MaaS) Guide

Guide to deploy Red Hat OpenShift AI 3.4 Models as a Service on OpenShift.

  • Kustomize manifests with status gates between every phase

  • Single automation script for end-to-end deployment

  • CPU-only simulator model for validation without GPUs

Requires OpenShift 4.19+ with cluster-admin access.

This guide is not a replacement for the official Red Hat OpenShift AI Models as a Service documentation. It is a companion resource with opinionated Kustomize manifests and automation scripts to accelerate deployment.

Getting Started

Clone the repository and work from its root directory. All commands and file paths in this guide are relative to the cloned repo.

git clone https://github.com/rh-aiservices-bu/rhoai-maas-guide.git
cd rhoai-maas-guide

You also need:

  • OpenShift 4.19+ cluster with oc CLI authenticated as cluster-admin

  • envsubst, curl, jq available on PATH

Phases

Each phase has step-by-step instructions, status gates, and troubleshooting.

Disconnected Setup

Phase Description Time

0. Disconnected Setup (air-gapped only)

Mirror MaaS operators and images, ITMS for modelcars, WASM shim patch, GPU driver setup

30-60 min

Installation Guide

Phase Description Time

1. Prerequisites

Operator subscriptions (RHOAI, RHCL, cert-manager, LWS)

5-10 min

2. Platform Configuration

Kuadrant/Authorino, User Workload Monitoring, GatewayClass, Gateway

5-10 min

3. MaaS Platform

PostgreSQL database and secrets

5 min

4. RHOAI Configuration

DataScienceCluster, DSCInitialization, Dashboard settings

5-10 min

Model Deployment & Verification

Phase Description Time

5. Model Deployment

Deploy and register LLM models with MaaS

1-15 min

6. Verification

End-to-end checks (API keys, inference, rate limiting)

5 min

Observability

Phase Description Time

7. Observability (optional)

COO subscription + Gateway telemetry dashboards

5 min

External Models

Phase Description Time

8. External Models (optional)

Expose third-party LLM APIs (OpenAI, Bedrock, Gemini) through the MaaS gateway

5-10 min

Automated Setup

For end-to-end deployment using a single script, see the Automated Setup page.

Available Models

Model GPU Required VRAM Use Case

simulator

No

None

Testing/demo (CPU-only)

granite-tiny-gpu

Yes

< 40 GiB

Small GPU (T4, L4, A10)

gpt-oss-20b

Yes

>= 40 GiB

Large GPU (L40S, A100, H100)

Documentation