179 lines
9.2 KiB
Markdown
179 lines
9.2 KiB
Markdown
## Context
|
|
|
|
`cloud-planner-proxy` centralizes an LLM call for Hosts that opt into
|
|
`AI_PLANNER_TRANSPORT=cloud`, but `cloud.planner_config` currently resolves
|
|
provider, model, and credentials from Cloud API environment variables. The
|
|
proxy route builds a client for every decision request, so it already has a
|
|
natural per-request configuration boundary. The Cloud Control Plane also has a
|
|
durable SQLAlchemy repository, Alembic migrations, session-authenticated
|
|
administrator Console, CSRF protection, and auth audit records that this
|
|
change can reuse.
|
|
|
|
The existing runtime clients instantiate provider SDKs with no arguments and
|
|
therefore read keys from their environment. Database-backed credentials and a
|
|
custom base URL require those clients to accept explicit values while
|
|
preserving the unchanged direct-transport default.
|
|
|
|
## Goals / Non-Goals
|
|
|
|
**Goals:**
|
|
- Let administrators manage a Cloud-wide catalog of `anthropic` and
|
|
`openai-compatible` planner profiles from the Cloud Console.
|
|
- Encrypt every stored Provider API key and never return or log its plaintext.
|
|
- Switch the provider/model used by all cloud-transport Host Agents without a
|
|
Host configuration change, Cloud API restart, or local credential.
|
|
- Support custom base URLs for Anthropic native tool use and OpenAI-compatible
|
|
Chat Completions tool calling without changing their respective protocols.
|
|
- Remove Cloud API environment-based planner Provider configuration so every
|
|
cloud-transport decision is governed by the active database profile.
|
|
|
|
**Non-Goals:**
|
|
- Per-Host, per-user, per-task, weighted, or failover Provider selection.
|
|
- Provider usage billing, connection-test endpoints, model discovery, or
|
|
support for protocols other than Anthropic native tool use and OpenAI Chat
|
|
Completions tool calling.
|
|
- Custom request headers or different Anthropic/OpenAI-compatible parameter
|
|
dialects.
|
|
- Automatic encryption-master-key rotation. Provider API key rotation is
|
|
supported by updating a profile; master-key rotation remains an operational
|
|
migration until a key-ring design is separately proposed.
|
|
|
|
## Decisions
|
|
|
|
### D1: Persist a profile catalog and a singleton active-profile setting
|
|
|
|
Add `cloud_llm_provider_profiles` with immutable id, unique normalized name,
|
|
provider type, model, optional base URL, timeout, encrypted API key, enabled
|
|
state, revision, and timestamps. Add a one-row
|
|
`cloud_llm_provider_settings` record with `database_management_enabled`,
|
|
`active_profile_id`, revision, and timestamp. The profile table represents
|
|
the operator's choices; the singleton holds the one Cloud-wide choice that is
|
|
active.
|
|
|
|
Activating an enabled profile updates the singleton in the same transaction
|
|
and sets `database_management_enabled=true`. An active profile cannot be
|
|
disabled or deleted before another enabled profile is activated. This avoids
|
|
the ambiguity and cross-database portability problems of a partial unique
|
|
"one active row" index, and gives PostgreSQL and SQLite one repository
|
|
contract.
|
|
|
|
Alternative considered: an `is_active` column on profiles. Rejected because
|
|
atomic replacement and uniqueness semantics differ between SQLite and
|
|
PostgreSQL, while a singleton pointer is explicit and makes initial
|
|
environment-fallback state representable.
|
|
|
|
### D2: Resolve the active profile on every planner decision
|
|
|
|
The internal planner endpoint resolves the active profile from the repository
|
|
at the start of each request, decrypts its API key, and builds the existing
|
|
tool-calling client with the resolved values. It uses that same resolved
|
|
profile for token-usage metadata, preventing a model switch during the call
|
|
from recording a mismatched provider/model. No application-level cache is
|
|
used; the one indexed settings lookup and symmetric decryption are negligible
|
|
relative to an LLM request and make a successful activation effective on the
|
|
next decision.
|
|
|
|
The resolver fails a planner request with the existing structured
|
|
`planner_unavailable` response when the active profile is missing, disabled,
|
|
corrupt, or undecryptable. It never reads legacy `AI_PLANNER_*` or Provider
|
|
API-key environment variables.
|
|
|
|
Alternative considered: move profile configuration into Host Agent heartbeat
|
|
responses. Rejected because it would replicate API keys to edge hosts, delay
|
|
changes until heartbeat, and duplicate the Cloud proxy's existing decision
|
|
boundary.
|
|
|
|
### D3: Use Fernet encryption with an environment-held master key
|
|
|
|
`cryptography` is added to `device-cloud-platform`. A small secret-box port
|
|
uses `cryptography.fernet.Fernet` with
|
|
`CLOUD_LLM_PROVIDER_ENCRYPTION_KEY`; only Fernet ciphertext is stored in the
|
|
database. The key remains environment or secret-manager configuration and is
|
|
not stored in the database. Profile write requests accept a plaintext API key
|
|
only to encrypt it; profile read responses expose `has_api_key` and
|
|
`key_last_rotated_at` rather than a key value.
|
|
|
|
The Cloud API can list profile metadata without the encryption key, but
|
|
creating, rotating, activating, or resolving a profile without a valid key
|
|
fails with a controlled configuration error that excludes the supplied secret.
|
|
Provider mutation audit events retain profile id and action only.
|
|
|
|
Alternative considered: plaintext database keys. Rejected because database
|
|
backup or read access would expose external-provider credentials. Alternative
|
|
considered: a database-held master key. Rejected because it does not create a
|
|
separate protection boundary.
|
|
|
|
### D4: Treat each provider wire protocol as a concrete contract
|
|
|
|
Profiles have `provider_type` of `anthropic` or `openai-compatible`. Either
|
|
profile may omit `base_url` to use its official provider endpoint, or supply
|
|
an absolute HTTP(S) base URL for a compatible proxy or provider service.
|
|
The runtime `OpenAIToolCallingClient` gains optional `api_key` and `base_url`
|
|
constructor arguments and constructs `OpenAI` explicitly when supplied.
|
|
`AnthropicToolCallingClient` gains optional explicit API key and `base_url`
|
|
arguments and constructs `Anthropic` explicitly when supplied. Absent optional
|
|
arguments retain current SDK environment behavior for direct transport.
|
|
|
|
Compatibility means the endpoint accepts OpenAI Chat Completions requests
|
|
with the tool/function-calling fields emitted by the existing client and
|
|
returns a single compatible tool call. It does not claim compatibility with
|
|
providers requiring a different request schema or custom authentication.
|
|
|
|
Alternative considered: create a new cloud-only HTTP adapter. Rejected because
|
|
it duplicates response parsing and would drift from the direct provider path.
|
|
|
|
### D5: Expose a dedicated administrator API and Console view
|
|
|
|
Add a public `/v1/planner/providers` router composed by the Cloud API. It
|
|
requires a new `llm-providers:admin` scope, which the existing admin role
|
|
receives via its wildcard. All writes use the existing session CSRF validator
|
|
and record non-secret auth audit events. The Console shows only for principals
|
|
with that scope and provides profile creation, editing/key rotation,
|
|
enable/disable, activation, and deletion of inactive profiles. Responses use
|
|
revisions for optimistic-concurrency conflicts rather than overwriting an
|
|
administrator's newer edit.
|
|
|
|
Alternative considered: reuse a generic configuration endpoint. Rejected
|
|
because Provider records carry secrets and need substantially stricter output,
|
|
audit, and activation invariants.
|
|
|
|
## Risks / Trade-offs
|
|
|
|
- [Risk] The Cloud database and encryption key become part of the planner hot
|
|
path. -> Mitigation: resolver failures use the established fail-fast
|
|
response, and readiness/configuration documentation makes the required
|
|
active profile explicit.
|
|
- [Risk] An administrator can direct Cloud traffic to an arbitrary compatible
|
|
base URL. -> Mitigation: only `llm-providers:admin` can write a profile;
|
|
deployments must treat that role as privileged network-configuration access.
|
|
- [Risk] Losing the master key makes stored Provider credentials unusable. ->
|
|
Mitigation: document secret-manager backup and keep a legacy deployment
|
|
rollback path while adopting the feature.
|
|
- [Risk] Some services marketed as OpenAI-compatible diverge on parameters or
|
|
tools. -> Mitigation: document the exact Chat Completions tool-calling
|
|
contract and surface provider failures instead of retrying against another
|
|
profile.
|
|
- [Risk] Concurrent Console administrators can race to edit or activate a
|
|
profile. -> Mitigation: revision checks on profile/settings writes and a
|
|
transaction for activation.
|
|
|
|
## Migration Plan
|
|
|
|
1. Apply the migration and provision `CLOUD_LLM_PROVIDER_ENCRYPTION_KEY` from
|
|
the deployment secret manager before routing cloud-transport planner work
|
|
to the new Cloud API version.
|
|
2. Create and activate a profile through the administrator Console. Until an
|
|
active profile exists, cloud planner requests fail closed.
|
|
3. Verify profile metadata and token-usage provider/model labels, then remove
|
|
`AI_PLANNER_*`, `ANTHROPIC_API_KEY`, and `OPENAI_API_KEY` from the Cloud API
|
|
deployment configuration.
|
|
4. To roll back, restore a previous Cloud API version together with its legacy
|
|
environment configuration. Do not remove the encryption key or database
|
|
rows before a planned migration.
|
|
|
|
## Open Questions
|
|
|
|
- None for the initial Cloud-wide selection. Per-Host assignment, cost-aware
|
|
routing, and master-key rotation need separate designs because they change
|
|
the selection and operational contracts.
|