Review release health after a deploy

Assemble trusted deploy, error-case, and product-event evidence into a human release review without collapsing unlike signals into a misleading score.

Updated July 10, 2026 · published May 26, 2026

difficulty intermediate · time to value 10 minutes · execution human review required

Review the latest trusted production deploy. Show SHA-attributed and timing-only error cases separately, check whether real post-deploy traffic exists, compare only approved product signals, and prepare a release decision brief for human review.

Why this matters

Post-deploy review combines unlike evidence: trusted release facts, error cases, traffic, and product outcomes. Turning those into one opaque score can make an unexercised release look healthy or a normal traffic dip look like a product regression.

Keep each source visible. A trusted deploy establishes what shipped. A matching SHA strengthens error attribution. Product-event volume shows activity, not conversion or causality by itself.

What you get

The latest trusted deploy and rollout boundary
Post-deploy cases separated by attribution quality
Real traffic evidence for the exact release
Selected product-signal movement with the right denominator and caveats
A reviewed decision brief, not an automatic operational action

Walk through it

You

Is the latest production release healthy enough to keep rolling out?

Agent

I will keep release facts, errors, traffic, and product outcomes separate and will stop at a human decision brief.

GET /v1/projects/:project_id/deploys?limit=3
POST /v1/projects/:project_id/query-blueprints/errors_after_last_deploy/run
POST /v1/projects/:project_id/query-blueprints/errors_by_deploy/run
POST /v1/projects/:project_id/query-blueprints/analytics_events_by_deploy/run

Fetch each blueprint’s current params and response schema first. Use errors_after_last_deploy for case rows and attribution labels; errors_by_deploy for aggregate error volume; and analytics_events_by_deploy only as traffic context.

If a particular product outcome matters, run its approved blueprint or a bounded custom query with an explicit numerator, denominator, exact periods, and rollout caveats. Do not infer funnel conversion from total analytics event volume.

The output

Release review — <sha> — <environment> — <rollout bounds>

Trusted release fact
- <deploy record and proof source>

Error evidence
- SHA-matched cases: <rows>
- timing-only cases: <rows and uncertainty>
- aggregate error volume: <value>

Traffic evidence
- approved real-traffic sample: <value/readiness>
- rollout/comparability caveat: <note>

Product evidence
- <approved metric with numerator, denominator, periods, sample>

Human decision
- keep watching | pause rollout | investigate | consider rollback/hotfix
- owner and evidence required for the choice

No single row grants rollback or deploy authority. If a case dominates, inspect GET /v1/cases/:case_id before proposing a code change.

Setting it up

Record deploys only from successful CI/provider post-deploy automation using AGENTRY_CI_API_KEY and a stable Idempotency-Key. Keep runtime error and analytics writes in their public/server credential boundaries. Include the release SHA on runtime signals when the app can prove it.

Select one real product event as a traffic sample and separately define any business outcome expected to move. Verify both through the saved signal map and live install proof. During a gradual rollout, compare like traffic and disclose the rollout fraction.

Variations

“Compare this release with the previous three trusted deploys.”
“Review checkout conversion with an explicit denominator and exact periods.”
“Show why the release is not yet decision-ready.”
“Prepare an oncall handoff for the top SHA-matched case.”

Adapt this workflow in your own agent.

Ask your agent to map the starter prompt to your saved signal map, live events, and the current HTTP API before answering.

Agentry gives coding agents production context from errors, product analytics, and deploys through one HTTP API. Fetch and read https://agentry.sh/install.md completely, then install Agentry's full first-time telemetry baseline in this repository. I authorize device-code login, saving the owner key only to ~/.agentry/credentials.json, creating one Agentry project for this repository, and read-only inspection of shipped source plus hidden CI/provider configuration. Deeply analyze the product's business logic and complete value flow before choosing telemetry. Save the complete source-backed plan and show me its exact contents and hash. Until I explicitly approve that plan or request a replacement, do not edit app/CI source, place runtime/CI credentials, commit, exercise product traffic, start proof, push, or deploy. After approval, I authorize only the reviewed targets: place the required scoped browser/server/CI credentials through the established environment or secret mechanism, preserve existing telemetry, implement and test the baseline, commit it, push that reviewed commit when the shipped CI/provider path requires it, exercise safe proof paths with test/non-customer data, and perform one deployment through the reviewed shipped CI/provider path. Ask first if proof would charge money, contact a third party, change real customer data, or require new external access. After the plan is saved, immediately before every onboarding state-changing POST, GET current onboarding state, perform only its single returned next_action, then read state again; do not batch or infer later stages. Continue until status is verified, installation_complete is true, and next_action is null. Keep all secrets, source snapshots, proof markers, and scratch files outside the repository.

+ Full access

5.5 Extra High

1. Open your repo in Codex, Claude Code, Cursor etc.
2. Paste the install prompt.
3. Your agent reads the install doc and shows you an implementation plan for approval.