ODH - Models as a Service with Policy Management
Our goal is to create a comprehensive platform for Models as a Service with real-time policy management.
📦 Technology Stack
- OpenShift: Kubernetes platform
- Gateway API: Traffic routing and management (OpenShift native implementation)
- Kuadrant/Authorino/Limitador: API gateway and policy engine
- KServe: Model serving platform
- React: Frontend framework
- Go: Backend frameworks
📋 Prerequisites
- OpenShift cluster (4.19.9+) with kubectl/oc access
- Kuadrant v1.4.2+ (ODH) or RHCL v1.3+ (RHOAI) - Required for MaaS v0.1.0+
- PostgreSQL database (for production ODH/RHOAI deployments)
⚠️ Important Version Requirements
Kuadrant 1.4.2+ Required (MaaS v0.1.0+)
MaaS v0.1.0 and later requires Kuadrant 1.4.2+ (ODH) or RHCL 1.3+ (RHOAI).
Why Kuadrant 1.4.2+ is required:
MaaS v0.1.0 requires the authorization header stripping capability added in Authorino v0.23.1 (shipped with Kuadrant 1.4.2) to protect user credentials from potential exfiltration to model backends.
Security Context:
When a user makes an inference request with their OpenShift token or API key, that credential must be validated by Authorino but should NOT be forwarded to model backends (whether internal KServe models or external providers). Kuadrant 1.4.2+ allows Authorino to:
- Validate the incoming user credential (OpenShift token or MaaS API key)
- Strip/replace the Authorization header before forwarding to model backends
- Optionally inject model-specific credentials from Kubernetes Secrets (credentialRef) for ExternalModel resources
This prevents credential exfiltration where a malicious or compromised model service could capture and misuse user tokens.
Migration Notes:
- The deployment script (
scripts/deploy.sh) automatically installs Kuadrant 1.4.2 for new deployments
- For existing deployments, upgrade Kuadrant/RHCL before upgrading to MaaS v0.1.0+
For detailed version compatibility, see Version Compatibility.
Database Required for Production
MaaS requires a PostgreSQL database for API key management. For production ODH/RHOAI deployments, you must create a Secret with the database connection URL before enabling modelsAsService.
See Database Prerequisites for details.
Note: The scripts/deploy.sh script creates a development PostgreSQL instance automatically.
🚀 Quick Start
Deploy Infrastructure
Use the unified deployment script for all deployment scenarios. The script installs prerequisites (policy engine, Gateway, PostgreSQL, Authorino TLS) and deploys maas-controller, which then deploys maas-api automatically via its Tenant reconciler:
# Deploy ODH (default)
./scripts/deploy.sh
# Deploy RHOAI
./scripts/deploy.sh --operator-type rhoai
# Deploy via Kustomize
./scripts/deploy.sh --deployment-mode kustomize
For detailed instructions, see the Installation Guide or the Deployment Options section below.
🛠️ Deployment Options
Basic Deployment
./scripts/deploy.sh [OPTIONS]
Key Options
| Flag | Values | Default | Description |
|---|
--deployment-mode | operator, kustomize | operator | Deployment method |
--operator-type | odh, rhoai | odh | Which operator to install |
--enable-tls-backend | flag | enabled | TLS for Authorino ↔ MaaS API |
--disable-tls-backend | flag | false | Disable TLS backend |
--namespace | string | auto | Target namespace |
--verbose | flag | false | Enable debug logging |
--dry-run | flag | false | Show plan without executing |
--help | flag | - | Display full help |
Advanced Options (PR Testing)
| Flag | Description | Example |
|---|
--operator-catalog | Custom operator catalog/index image | quay.io/opendatahub/catalog:pr-456 |
--operator-image | Custom operator image (patches CSV) | quay.io/opendatahub/operator:pr-456 |
--channel | Operator channel override | fast-3, stable-3.x |
Environment Variables
| Variable | Description | Example |
|---|
MAAS_API_IMAGE | Custom MaaS API container image (passed to the Tenant reconciler via RELATED_IMAGE_ODH_MAAS_API_IMAGE) | quay.io/user/maas-api:pr-123 |
MAAS_CONTROLLER_IMAGE | Custom MaaS controller container image | quay.io/user/maas-controller:pr-123 |
OPERATOR_CATALOG | Custom operator catalog | quay.io/opendatahub/catalog:pr-456 |
OPERATOR_IMAGE | Custom operator image | quay.io/opendatahub/operator:pr-456 |
OPERATOR_TYPE | Operator type (rhoai/odh) | odh |
LOG_LEVEL | Logging verbosity | DEBUG, INFO, WARN, ERROR |
Note: TLS backend is enabled by default. Use --disable-tls-backend to disable.
Note: The policy engine is auto-determined based on operator type (rhcl for RHOAI, kuadrant for ODH/kustomize) and does not need to be set manually.
Deployment Examples
Standard Deployments
# Deploy RHOAI
./scripts/deploy.sh --operator-type rhoai
# Deploy ODH
./scripts/deploy.sh --operator-type odh
Testing PRs
# Test MaaS API PR #123
MAAS_API_IMAGE=quay.io/myuser/maas-api:pr-123 \
./scripts/deploy.sh --operator-type odh
# Test ODH operator PR #456 with custom manifests
./scripts/deploy.sh \
--operator-type odh \
--operator-catalog quay.io/opendatahub/opendatahub-operator-catalog:pr-456 \
--operator-image quay.io/opendatahub/opendatahub-operator:pr-456
Minimal Deployments
# Deploy without TLS backend (HTTP for Authorino to maas-api)
./scripts/deploy.sh --disable-tls-backend
📚 Documentation
Online Documentation: https://opendatahub-io.github.io/models-as-a-service/
🤝 Contributing
We welcome contributions! Please:
- Fork the repository
- Create a feature branch
- Make your changes
- Submit a pull request
📝 License
This project is licensed under the Apache 2.0 License.
📞 Support
For questions or issues:
