|
| 1 | +--- |
| 2 | +title: "Using GoModel with Oracle Generative AI" |
| 3 | +description: "Configure Oracle's OpenAI-compatible Generative AI endpoint in GoModel, including the required OCI policy and model fallback." |
| 4 | +--- |
| 5 | + |
| 6 | +GoModel works with Oracle Generative AI through Oracle's OpenAI-compatible |
| 7 | +endpoint. |
| 8 | + |
| 9 | +Flow: |
| 10 | + |
| 11 | +`Client -> GoModel -> Oracle Generative AI` |
| 12 | + |
| 13 | +## Before you start |
| 14 | + |
| 15 | +- Create an Oracle Generative AI API key. |
| 16 | +- Add an OCI IAM policy for `generativeaiapikey`. |
| 17 | +- Choose a supported Oracle region and model. |
| 18 | +- Prefer YAML configuration for Oracle so you can set `models:`. |
| 19 | + |
| 20 | +## 1. Add the OCI policy |
| 21 | + |
| 22 | +For a simple test setup, this tenancy-level policy is enough: |
| 23 | + |
| 24 | +```text |
| 25 | +Allow any-user to use generative-ai-family in tenancy where ALL {request.principal.type='generativeaiapikey'} |
| 26 | +``` |
| 27 | + |
| 28 | +This allows Oracle Generative AI bearer API keys to call the inference APIs. |
| 29 | + |
| 30 | +For production, narrow this policy to a specific compartment, API key, or model |
| 31 | +instead of leaving it tenancy-wide. |
| 32 | + |
| 33 | +## 2. Set the Oracle endpoint and API key |
| 34 | + |
| 35 | +Use Oracle's OpenAI-compatible inference base URL for your region. For Chicago: |
| 36 | + |
| 37 | +```bash |
| 38 | +export ORACLE_BASE_URL="https://inference.generativeai.us-chicago-1.oci.oraclecloud.com/20231130/actions/v1" |
| 39 | +export ORACLE_API_KEY="..." |
| 40 | +``` |
| 41 | + |
| 42 | +## 3. Configure Oracle in GoModel |
| 43 | + |
| 44 | +Use a YAML provider block and set `models:` explicitly: |
| 45 | + |
| 46 | +```yaml |
| 47 | +providers: |
| 48 | + oracle: |
| 49 | + type: oracle |
| 50 | + base_url: "${ORACLE_BASE_URL}" |
| 51 | + api_key: "${ORACLE_API_KEY}" |
| 52 | + models: |
| 53 | + - openai.gpt-oss-120b |
| 54 | +``` |
| 55 | +
|
| 56 | +Why `models:` matters: |
| 57 | + |
| 58 | +- Oracle inference works through `chat/completions` and `responses` |
| 59 | +- Oracle's `/models` endpoint may not be available for this API-key flow |
| 60 | +- GoModel can fall back to the configured model list when `/models` is |
| 61 | + unavailable |
| 62 | + |
| 63 | +For Oracle, YAML is the recommended path. Environment-only auto-discovery is |
| 64 | +not enough if you need the configured-model fallback. |
| 65 | + |
| 66 | +## Current status |
| 67 | + |
| 68 | +What is integrated today: |
| 69 | + |
| 70 | +- Oracle's OpenAI-compatible inference endpoints |
| 71 | +- manual model configuration through `models:` |
| 72 | +- GoModel `/v1/models` from the configured-model fallback |
| 73 | + |
| 74 | +What is not yet validated as reliable: |
| 75 | + |
| 76 | +- Oracle's OpenAI-compatible `/models` endpoint for automatic model discovery |
| 77 | + |
| 78 | +What is not integrated yet: |
| 79 | + |
| 80 | +- native Oracle model auto-discovery through OCI APIs |
| 81 | +- automatic population of the Oracle model inventory without `models:` |
| 82 | + |
| 83 | +If Oracle later exposes a reliable `/models` endpoint for this API-key flow, or |
| 84 | +GoModel adds a separate OCI-native discovery path, this manual `models:` |
| 85 | +requirement can be relaxed. |
| 86 | + |
| 87 | +## 4. Start GoModel |
| 88 | + |
| 89 | +```bash |
| 90 | +go run ./cmd/gomodel |
| 91 | +``` |
| 92 | + |
| 93 | +## 5. Verify the model registry |
| 94 | + |
| 95 | +```bash |
| 96 | +curl -s http://localhost:8080/v1/models |
| 97 | +``` |
| 98 | + |
| 99 | +Expected result: |
| 100 | + |
| 101 | +- a `200 OK` |
| 102 | +- an Oracle-owned model such as `oracle/openai.gpt-oss-120b` |
| 103 | + |
| 104 | +## 6. Verify Responses |
| 105 | + |
| 106 | +```bash |
| 107 | +curl -s http://localhost:8080/v1/responses \ |
| 108 | + -H "Content-Type: application/json" \ |
| 109 | + -d '{ |
| 110 | + "model": "openai.gpt-oss-120b", |
| 111 | + "input": "Reply with the single word ok." |
| 112 | + }' |
| 113 | +``` |
| 114 | + |
| 115 | +Expected result: |
| 116 | + |
| 117 | +- a `200 OK` |
| 118 | +- final output text containing `ok` |
| 119 | + |
| 120 | +## 7. Verify Chat Completions |
| 121 | + |
| 122 | +```bash |
| 123 | +curl -s http://localhost:8080/v1/chat/completions \ |
| 124 | + -H "Content-Type: application/json" \ |
| 125 | + -d '{ |
| 126 | + "model": "openai.gpt-oss-120b", |
| 127 | + "messages": [{"role": "user", "content": "Reply with the single word ok."}], |
| 128 | + "max_tokens": 80 |
| 129 | + }' |
| 130 | +``` |
| 131 | + |
| 132 | +Use a high enough `max_tokens` budget. Some Oracle-backed reasoning models can |
| 133 | +spend short completions on reasoning content before emitting final assistant |
| 134 | +text. |
| 135 | + |
| 136 | +## Troubleshooting |
| 137 | + |
| 138 | +- `404 Authorization failed or requested resource not found` |
| 139 | + Usually means the Generative AI API key policy is missing, the region is |
| 140 | + wrong, or the model is not available to the account. |
| 141 | +- `model registry has no models` |
| 142 | + Add `models:` to the Oracle provider config so GoModel can use the fallback. |
| 143 | +- OCI CLI works but Oracle bearer requests fail |
| 144 | + These are different auth flows. OCI CLI uses API signing keys; Oracle |
| 145 | + Generative AI inference uses the Generative AI bearer API key. |
| 146 | + |
| 147 | +## References |
| 148 | + |
| 149 | +- Oracle API keys overview: https://docs.oracle.com/en-us/iaas/Content/generative-ai/api-keys.htm |
| 150 | +- Oracle API key permissions: https://docs.oracle.com/en-us/iaas/Content/generative-ai/add-api-permission.htm |
| 151 | +- Oracle OpenAI-compatible endpoint: https://docs.oracle.com/en-us/iaas/Content/generative-ai/oci-openai.htm |
0 commit comments