Human approval for Test-to-Live promotion

Availability depends on the evaluation, run and promotion services enabled for your organization. These operations do not run a judge or automatically publish an agent. Test Compute runs and assessments use real credits.

A promotion evidence receipt records a human owner’s approval of an exact proposal, quality policy and verified output set. A calibration certificate with criteria_met is necessary, but it is only one input. Paid judge scores and calibration certificates keep their uncalibrated status after approval.

Use a signed-in human owner session. You must currently own both the source Test environment and destination Live environment in the same organization. API keys, service accounts and workload credentials cannot perform these operations.

Declare the policy before human review

Prepare the exact promotion request: source agent version, destination, proposed Live definition, resource mappings, model dependencies, selected grants and funded limits. The declaration’s request.evidence is the empty reference {"id":"","version":0,"integrity":""}; a receipt does not exist yet.

Call promotionEvidence.declare in TypeScript or promotion_evidence.declare in Python with the candidate campaign’s exact ID and integrity, the policy, explicit baseline selection and acknowledge: true. Supply your own idempotency key.

Every criterion requires its minimum case count, minimum resolved human labels, minimum label coverage and minimum human pass rate. Confidence and every threshold are mandatory; the service does not choose them. Thresholds expressed in parts per million use 1,000,000 for 100 percent.

The policy must be declared before any human assignment or label exists in either campaign, including an expired assignment. One candidate campaign permits one declaration. Existing judge scores may already exist, so this is a predeclared human quality policy, not a blind preregistered model experiment.

For an initial Live publication, set the baseline and every criterion’s regression field explicitly to null. For an upgrade, pin a distinct historical Test agent version and baseline campaign corresponding to the exact previous Live version. Declare minimum paired cases, minimum pair coverage and maximum allowed human pass rate loss for each criterion. A baseline cannot be omitted from an upgrade.

Review all held-out cases

Complete independent human review and seal the candidate campaign. An upgrade also needs a separately reviewed baseline campaign. Both certificates must meet their own declared calibration criteria. Judge comparison using another certificate’s labels does not replace independent review of different agent outputs.

The two campaigns must use the same exact cases and rubric. Campaigns are bounded to 100 pinned held-out dataset versions and 1,000 unique input cases. Development and held-out input roots and hashes cannot overlap. Shared expected-answer bytes alone are not treated as an input overlap.

For every case, identify an actual successful managed Test Compute run of the pinned agent version. Its admitted input must match the case’s exact approved input reference. Its final generated artifact must match the candidate output’s hash, size and media type. A simulated run or a manually supplied completion record is not sufficient. This correspondence does not change the candidate’s recorded user_supplied origin.

Inspect the evidence preview

Call prepare with the declaration ID and integrity, candidate certificate and complete case-to-run mapping. Supply the equivalent baseline arm for an upgrade, or explicit null for initial publication. Preparation never starts more runs.

The immutable preview contains exact score, assessment, admission, decision and artifact digests; human counts; confidence bounds; and the earliest output or parent artifact expiry. It contains references and metadata rather than customer content.

Report value	Meaning
insufficient_evidence	Required case, resolved-label or paired-case evidence is missing
criteria_failed	A configured coverage, quality or regression bound failed
criteria_met	Every configured human-quality criterion met its bound

Missing human labels and human abstentions have separate counts. An undefined rate is null, never zero. Pass-rate loss uses a conservative bound on paired outcomes; unpaired cases do not inflate the comparison. Reported intervals account for the number of criterion comparisons. Meeting a permissive threshold does not establish useful statistical power or live judge quality.

Inspect the exact target and all thresholds before approving. A changed target, current dependency, previous Live head, revoked source or reviewer, Test reset, or expired run output can make the recorded preview unusable for publication.

Approve and use the receipt

Call approve with the exact preview ID and integrity and acknowledge: true. The service rechecks authority, policy, complete evidence, dependencies and expiration. A successful receipt has state human_approved.

Use its receipt ID, version 1 and returned record integrity as the promotion request’s evidence reference. The promotion workflow still requires its own exact review and activation. Activation checks current grants, funding, dependencies, head version and evidence expiry; an old approval does not restore revoked access.

Use listDeclarations, listPreviews and listReceipts to find records you created in the selected Test/Live pair. Python uses list_declarations, list_previews and list_receipts. Pass the organization ID and a scope object containing source_environment_id and destination_environment_id. Each page contains at most 25 rows; pass its next_cursor unchanged as after to continue. The encrypted cursor is bound to both current environment authorities, your session and the record kind. It expires after 15 minutes and can become invalid after a service restart.

Directories show identifiers, integrity values, parent IDs, creator, timestamps, state and expiry. They do not include customer content, source references or score reports. Entries from an earlier Test reset generation are hidden. An expired or revoked output can still have historical directory metadata; the entry is not proof that its evidence remains usable.

getDeclaration, getPreview and getReceipt open a record by ID and recheck current evidence authority. Python uses get_declaration, get_preview and get_receipt. An unavailable full record must not be displayed as ready merely because its directory entry has a recorded passing state.

After a lost or uncertain mutation response, retain the exact request and original idempotency key. Retry only that unchanged pair to recover the operation. Both SDKs require caller-supplied keys and make no automatic retry. A changed account or scope requires reloading and reviewing the current state before another mutation.

The product API reference describes the routes and closed request and response fields. See evaluations for approved content, paid assessments and human review prerequisites.

Find and review a promotion draft

After obtaining the approved evidence reference, use environmentPromotions.create / environment_promotions.create to record the exact Test-to-Live proposal. Inspect its target and current funding, then use review to prepare the exact signed target and activate for the separate publication decision. Each mutation needs your own idempotency key; keep its reviewed body unchanged during recovery.

environmentPromotions.list(orgID, scope, after?, limit?) and Python environment_promotions.list(org_id, scope, after=None, limit=25) find drafts you created. The scope contains source_environment_id and destination_environment_id. The directory requires a fresh human owner session in both environments and returns at most 25 rows per page in the current Test reset generation.

Each entry contains its ID, integrity, creator, creation time and evidence_attached. That boolean indicates only that the draft records an evidence reference. It does not mean the evidence remains valid, the draft is ready, or publication occurred. Open the draft and perform the current review checks before enabling an activation decision.

Pass next_cursor unchanged as after; the encrypted cursor is bound to both current authorities and your session, expires after 15 minutes and may become invalid on service restart. The directory does not expose the draft’s source references or request body.

After publication, controlled rollback can restore an eligible prior approved Live definition as a new immutable version. It still requires current dependencies, grants, funding and an explicit human review.