fix(catalog): warned on LiteLLM metadata fallback

Logged one actionable warning per LiteLLM management base when rich metadata discovery fails, while keeping missing 404 routes silent.

Documented metadata-route permissions and covered forbidden versus absent endpoints.

Fixes #5801
This commit is contained in:
roboomp
2026-07-17 07:38:06 +00:00
parent 0f9fceeea4
commit 63cac8dfd5
4 changed files with 102 additions and 7 deletions
+3 -1
View File
@@ -300,7 +300,9 @@ When `litellm` is active (for example through `LITELLM_API_KEY` or stored auth),
- base URL: explicit provider `baseUrl` / `models.yml` config, otherwise `LITELLM_BASE_URL`, otherwise `http://localhost:4000/v1`
- auth mode: `LITELLM_API_KEY` or stored LiteLLM auth when the proxy requires a key
Runtime discovery probes LiteLLM management metadata first: `GET /model_group/info`, then `GET /v2/model/info`, then falls back to the OpenAI-compatible `GET /models` list. Rich metadata maps `max_input_tokens`, `max_output_tokens`, `supports_vision`, and `supports_reasoning`; bare fallback ids are enriched against bundled reference metadata when available.
Runtime discovery probes LiteLLM management metadata in order: `GET /model_group/info`, `GET /v2/model/info`, `GET /model/info`, and `GET /v1/model/info`. The configured key must be authorized to read at least one of these routes; on deployments that restrict management endpoints, grant the route through LiteLLM's `allowed_routes` access controls or use a master/admin key for discovery.
If every metadata route is unavailable, discovery falls back to the OpenAI-compatible `GET /models` list. A forbidden or failed metadata request is logged once with its endpoint and status; `404` is treated as an absent route. Rich metadata maps per-model context and capability fields, while bare fallback ids are enriched against bundled reference metadata when available. Models absent from the bundled catalog can therefore have unknown context and pricing after fallback.
### Explicit provider discovery