Why this matters
Most teams ship and then manually check three places: deploy logs, error monitoring, and product metrics. The scorecard turns that into a single post-deploy ritual the agent can run after every release.
What you get
- New and regressed fingerprints since the deploy
- Affected-user count for the top cases
- Movement in selected product events or funnels
- A clear recommendation: keep watching, roll back, hotfix, or investigate one case
Walk through it
Build a release health scorecard for the latest deploy.
I’ll pull the deploy, compare error fingerprints, then check key product events around the release window.
GET /v1/projects/:project_id/deploys { project_id: "default", limit: 1 }POST /v1/projects/:project_id/recipes/:recipe_id/run {
recipe_id: "errors_after_last_deploy",
project_id: "default",
params: {}
}POST /v1/projects/:project_id/analytics/query {
project_id: "default",
hogql: "SELECT event, count() AS count FROM events WHERE timestamp > now() - INTERVAL 2 HOUR GROUP BY event ORDER BY count DESC LIMIT 20"
}The output
The agent returns a release note for engineering: deploy SHA, new cases, user impact, metric changes, and the one next action. If one case dominates, it should call GET /v1/cases/:case_id before recommending a fix.
Setting it up
Post deploys from CI with POST /v1/deploys/ and AGENTRY_DSN, and keep error + analytics ingest enabled. No extra event names are required beyond the metrics your team wants in the scorecard.
Variations
- “Compare this deploy to the previous three production deploys.”
- “Only include checkout and signup metrics.”
- “Send the scorecard to the release Slack channel after every deploy.”