Model Gateway overview
Learn how Bob on-premises uses the model-gateway.yaml and config.yaml files to securely configure model endpoints, credentials, and TLS certificates for Model Gateway access.
To enable access to external and provider-hosted models, Bob on-premises uses a two-file configuration approach that separates model routing settings from sensitive credentials and certificates. The model-gateway.yaml file defines model endpoints, provider configurations, and references to credential and certificate variables, while config.yaml stores the corresponding secret values. This separation improves security, simplifies credential management, and supports consistent deployment across environments.
Understanding configuration files
Configuring model access for Bob on-premises requires two configuration files that work together. This approach separates model configuration from sensitive information, helping you manage credentials and certificates securely.
The model-gateway.yaml file defines model endpoints and references the names of variables that store credentials and certificates. The actual secret values — API keys, access tokens, passwords, and TLS certificates — are stored separately in config.yaml.
| File | Purpose | Contents | When to use |
|---|---|---|---|
model-gateway.yaml (Model Routing Specification) | Defines model routing and connection settings for the Model Gateway. | Model endpoints and provider routing blocks; variable name pointers for credentials (for example, api_key: env.BOB_AZURE_API_KEY, access_key_id: env.AWS_ACCESS_KEY); variable name pointer for TLS CA certificates (for example, ca_cert_pem: env.CA_CERT). Environment variable names can be any name, provided the same name is defined in bob.modelGateway.secrets. Do not store actual secrets in this file. | Create from config-model-gateway-template.yaml to specify model endpoints and the variable names for credentials and TLS certificates. |
config.yaml (Main Install & Secrets) | Stores the actual secret values used by the Model Gateway. | API keys, access tokens, passwords, and PEM-encoded TLS certificates mapped to variable names defined in model-gateway.yaml. | Populate this file with the required credentials and certificate values before deployment under bob.modelGateway.secrets. |
How secrets are managed
You do not need to create or export environment variables on your local workstation.
When you run the Bob installation:
- Bob reads the values defined in
bob.modelGateway.secretswithinconfig.yaml. - The installation process creates a Kubernetes secret in the OpenShift cluster.
- The secret values are injected into the Bob Inference service.
- The Model Gateway resolves references such as
env.<VAR_NAME>from the runtime environment inside the pod.
This approach ensures that sensitive information is stored securely and remains separate from model configuration settings.
Recommended configuration workflow
Configure model-gateway.yaml
Create model-gateway.yaml from the provided template and define:
- Model endpoints
- Provider settings
- Credential variable references
- TLS certificate variable references
For example: access_key: env.AWS_ACCESS_KEY
Configure config.yaml
Create config.yaml from the provided template and add the corresponding secret values under bob.modelGateway.secrets. For example:
bob:
modelGateway:
secrets:
AWS_ACCESS_KEY: "<actual-access-key>"Deploy the configuration
Install or update Bob using the configured files. During deployment, Bob applies the infrastructure configuration from config.yaml and loads the Model Gateway configuration from model-gateway.yaml.
The Bob Operator manages connectivity between Bob and your model endpoints. Provisioning, deploying, and operating the underlying model-serving infrastructure is outside the scope of the Bob Operator.
Upgrade
Update an existing IBM Bob on-premises deployment to a newer release while preserving the current configuration.
Required and supported models
Learn about the models required by Bob on-premises, including supported core inference models, guardrail models, and provider-specific safety features used to enable AI-powered capabilities.