> ## Documentation Index
> Fetch the complete documentation index at: https://docs.factorize.sh/llms.txt
> Use this file to discover all available pages before exploring further.

# Credential proxy cutover and recovery

> Operator and user procedure for retiring the separate credential proxy and reconnecting credentials for the per-run backend proxy.

# Credential proxy cutover and recovery

This is a breaking migration from the separate per-owner proxy and user-held
unlock key to the Factorize backend proxy. The release removes the old proxy,
unlock, pairing and run-bearer endpoints. Guest access requires a fresh vendor
signature bound to the exact run VM; there is no Factorize VM token fallback.

Credentials are encrypted with the server's `CREDENTIAL_ENCRYPTION_KEY`. The
backend can migrate its own managed credentials, but cannot recover secrets
inside the old user encryption. Those records require owner re-entry.

Do this during a maintenance window. Migration `0052_retire_credential_proxy`
refuses nonterminal runs or existing credential records without operator
preparation. This document describes deployment work; repository tests do not
mean the production cutover has already happened.

## Before the cutover

1. Announce a maintenance window and pause schedules, webhook events, and
   other dispatch sources for affected triggers. Prevent new runs from starting
   while the inventory is being prepared.
2. Inventory each tenant's triggers, selected portable integration IDs, model
   account IDs, and the owner of each credential. Keep this inventory private;
   it must contain identifiers and owners, never secret values.
3. Let active runs finish. For runs that cannot finish, stop them through the
   Factorize run controls and verify that each is terminal before proceeding.
   Do not stop a shared execution app or VM to drain one tenant's runs.
4. Pause credential writes as well as all dispatch. With the pre-cutover
   schema (through migration 0051), run the read-only preparation plan:

   ```sh theme={null}
   node packages/api/scripts/prepare-credential-cutover.mjs > private-cutover-plan.json
   ```

   Supply `DATABASE_URL` and `CREDENTIAL_ENCRYPTION_KEY` through the operator's
   secret environment. The report contains identifiers and trigger selections, not
   secret values. Confirm every recorded `proxy_identity` against vendor
   inventory; distinguish proxy-only apps from execution resources.
5. Review the plan, then run the same command with `--apply`, preserving its
   report privately. This disables triggers selecting legacy user ciphertext,
   leaves their credential IDs and execution configuration intact, tombstones
   those unreadable credentials, clears the retired change feed and revokes
   old proxy control/enrollment state. Managed active credentials, GitHub and
   Linear connections, Fly connections and execution configurations survive.
6. Keep dispatch and writes paused. Stop and verify deletion of only the
   inventoried old proxy Machines/VMs, shared proxy integrations and proxy-only
   apps through documented vendor controls. Record resource identifiers and
   absence checks. The preparation script does not delete vendor resources.
7. Apply migration 0052 and deploy the backend and static application together
   following the package migration release gates. Do not deploy this release
   over an undrained database. The migration checks for credential writes after
   preparation and drops the obsolete proxy, enrollment, unlock and generation
   tables. Remove obsolete proxy image/build secrets and model browser
   capability signing keys from deployment configuration.
8. Verify live canary runs on exe.dev and Fly: exact run identity, selected and
   unselected credential behavior, empty scopes, cancellation, artifacts and
   cleanup. exe.dev requires unique per-run WIF attachment and inventory access;
   Fly requires organization OIDC discovery/JWKS and guest Unix-socket tokens.
   Fix failed canaries before resuming dispatch.

If any run remains active, or a resource identity cannot be established, keep
maintenance in place. Never stop a shared execution app to retire one proxy.

## Reconnect credentials and repair triggers

1. Keep existing GitHub and Linear connections; their upstream permissions do
   not expand. Owners reconnect unavailable model accounts using an OpenAI-compatible
   provider URL and optional access key, and re-enter unreadable custom bearer
   credentials. New IDs may need explicit trigger repair. Existing server-managed
   records retain their IDs and do not require unnecessary reconnection.
2. Compare the new connection metadata with the private inventory. Update each
   affected trigger to select the corresponding new credential ID. The cutover
   must not select every tenant credential automatically. A trigger that cannot
   be mapped confidently stays disabled until its owner repairs it.
3. Verify each saved selection belongs to that trigger's tenant and that the
   selected account is connected. Run one canary per repaired trigger and confirm
   its outbound requests use only those selections. Confirm Factorize API
   scopes match the trigger's configured scopes.
4. Keep triggers with no portable selections unchanged. Do not add a credential to
   a trigger merely because it was available to the retired proxy.
5. Resume dispatch only after the affected trigger inventory is reconciled. If a
   credential cannot be reconnected, leave that selection unavailable and the
   trigger disabled; do not substitute another credential silently.

## Recovery

If a canary fails, keep dispatch paused and repair the new backend or the
specific credential selection. After migration 0052, the old backend is not a
compatible rollback target: it depends on removed tables and credentials.
Never restore old credentials or user ciphertext to revive access. A rollback
requires a separately reviewed deployment/database recovery plan with dispatch
paused; ordinary recovery proceeds forward on the managed architecture.

## Completion record

Record the maintenance window, terminal-run count, retired proxy app/Machine
identifiers, repaired trigger count, triggers left disabled for owner repair, and
canary results. Keep secret values, bearer tokens, unlock keys, and decrypted
credential material out of the record.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.