feat(cloud): manage LLM providers in database
Tests / Test passed: 664

This commit is contained in:
2026-07-14 00:31:47 +08:00
parent b613a315ff
commit a166ffd8a4
35 changed files with 2432 additions and 204 deletions
+26 -9
View File
@@ -379,10 +379,22 @@ planning decision through the Cloud API's
host-scoped bearer credential used for heartbeat/claim/renew/result). In this
mode:
- **Credentials move to the Cloud API.** Configure `AI_PLANNER_PROVIDER`,
`AI_PLANNER_MODEL`, `AI_PLANNER_TIMEOUT_SECONDS`, and
`ANTHROPIC_API_KEY`/`OPENAI_API_KEY` on the Cloud API process instead of the
Host Agent -- edge hosts no longer need provider keys at all.
- **Provider configuration lives in the Cloud database.** Set
`CLOUD_LLM_PROVIDER_ENCRYPTION_KEY` from the deployment secret manager, then
sign in to `/console/` as an administrator and create an active entry under
**LLM providers**. Provider API keys are encrypted in the database and are
never returned by the API or Console. Edge Hosts do not hold Provider keys.
The Cloud API does not read `AI_PLANNER_PROVIDER`, `AI_PLANNER_MODEL`,
`AI_PLANNER_TIMEOUT_SECONDS`, `ANTHROPIC_API_KEY`, or `OPENAI_API_KEY`.
- **Profile types:** choose **Anthropic** for native Anthropic tool use, or
**OpenAI-compatible** for the OpenAI Chat Completions tool-calling protocol.
Leave its Base URL blank for official OpenAI, or provide the compatible
provider's absolute HTTP(S) `/v1` endpoint. Providers requiring another
request schema or custom authentication are not supported by this path.
- **Activation is immediate:** a newly activated enabled profile becomes the
Provider/model for the next Cloud-proxy planner decision. A Cloud-planner
request fails closed until one enabled profile is active; it never falls back
to a Cloud API environment credential.
- **Trade-offs to accept before enabling:**
- *Latency*: every planning step now makes a round trip to the Cloud API in
addition to the LLM provider call.
@@ -420,11 +432,16 @@ Hosts reporting `AI_PLANNER_TRANSPORT=direct` are explicitly shown as
**unmetered**. Cloud cannot enforce or verify their provider token use. Do not
interpret an unmetered Host's absence of usage events as budget compliance.
Roll out in this order: migrate the Cloud database, deploy the Cloud API,
switch a pilot Host to Cloud transport, configure a budget above the reservation
ceiling, then review its usage events before enabling budgets fleet-wide. A
rollback to direct transport requires valid provider credentials on that Host;
preserve usage and policy rows rather than deleting accounting history.
Roll out in this order: migrate the Cloud database, provision
`CLOUD_LLM_PROVIDER_ENCRYPTION_KEY`, deploy the Cloud API, create and activate
a Provider profile in `/console/`, then switch a pilot Host to Cloud transport.
Configure a budget above the reservation ceiling and review usage events before
enabling budgets fleet-wide. Rotate a Provider API key by editing that profile;
the encryption master key is deployment-managed and must be preserved with the
database backups. A rollback to an older Cloud API requires restoring its
legacy Provider environment configuration, while a rollback to `direct`
transport requires valid provider credentials on that Host; preserve usage and
policy rows rather than deleting accounting history.
## Operational Limitations