Detect a sudden spike in negative reviews

Why this matters

Negative review spikes are one of those signals that’s catastrophic if you miss it and trivial if you catch it early:

• Coordinated review-bomb: a competitor, a disgruntled customer with a Twitter following, or a flash mob (some product appearance on TV inviting “go review this and tank it”). The longer it runs, the more it sets the App Store / Trustpilot baseline.
• Quality regression: a deploy broke a checkout flow, customers paid, didn’t get their order, are now 1-starring you. The window between “first 1-star comes in” and “fix shipped” is the entire damage radius.

Both look identical: a 1-2 star rate that’s 3-10× the normal rate. Both need a human eyeball within 15 minutes, not 15 hours.

Agentry’s Routines + raw HogQL is the simplest way to build this. No Datadog, no PagerDuty config files — just a scheduled prompt that checks the rate and pings Slack when it spikes.

What you get

• A scheduled Routine running every 15 minutes
• A clean baseline (7-day rolling avg, ignoring outliers)
• A Slack message when the current 15-minute window exceeds 3× baseline, with offending sellers/listings included

Walk through it

agentry_register_webhook {
  project_id: "default",
  url: "https://hooks.slack.com/services/T0000/B0000/XXXX",
  description: "Review-bomb detector",
  events: ["routine.review_bomb_alert"]
}

{
  "webhook_id": "wh_rb1",
  "signing_secret": "agws_…(shown once)…",
  "active": true
}

/schedule "*/15 * * * *" "
Step 1: Compute baseline. Run agentry_analytics_query:
  SELECT countIf(properties.rating <= 2) / (7 * 24 * 4) AS baseline_per_15min
  FROM events WHERE event = 'review_submitted'
  AND timestamp BETWEEN now() - INTERVAL 7 DAY AND now() - INTERVAL 15 MINUTE

Step 2: Compute current 15-min window:
  SELECT countIf(properties.rating <= 2) AS current_window,
         groupArray((properties.seller_id, properties.rating, properties.comment)) AS offending
  FROM events WHERE event = 'review_submitted' AND timestamp > now() - INTERVAL 15 MINUTE

Step 3: If current_window > 3 * baseline_per_15min AND current_window >= 5, POST a Slack message
to webhook wh_rb1 with current rate, baseline, top offending sellers, and a link to investigate.
Otherwise silent.
"

{ "routine_id": "rt_rb1", "next_run_at": "2026-05-16T13:30:00Z" }

The output

A Slack message like:

🚨  Negative review spike

  Last 15 min:    42 reviews with rating ≤ 2
  7-day baseline: 3.8 / 15 min
  Multiplier:     11.1× ★

  Top affected sellers (last 15 min):
    seller_8421   "Order never arrived"      ×12
    seller_3110   "Item not as described"    ×8
    seller_8421   "Charged but no order"     ×6

  Likely: either coordinated bomb on seller_8421, OR
          a fulfillment / order-state regression.

  Investigate:
    https://agentry.sh/recipes/dispute-rate-monitoring
    https://agentry.sh/recipes/customer-investigation

And as a console output when the Routine first runs:

review_bomb_detector Routine (rt_rb1)

  ✓ Webhook registered:   wh_rb1
  ✓ Cron schedule:        */15 * * * *
  ✓ Floor threshold:      ≥ 5 events in current window
  ✓ Multiplier threshold: > 3× baseline
  ✓ First check at:       2026-05-16T13:30:00Z

  Baseline (computed at register time):
    Last 7 days low-rating reviews:    2,560
    Average per 15-min slot:           3.81
    P95 per 15-min slot:                12

Setting it up

1. Emit review_submitted with rating. The agent groups by seller_id to surface offenders — include it if you have it:

await fetch(`https://api.agentry.sh/v1/analytics/${PROJECT_ID}/`, {
  method: "POST",
  headers: {
    "Authorization": `Bearer ${process.env.AGENTRY_DSN}`,
    "Content-Type": "application/json",
    "User-Agent": "marketplace/1.0",  // REQUIRED — Cloudflare 403s default UAs
  },
  body: JSON.stringify({
    event: "review_submitted",
    distinct_id: reviewer.id,
    properties: {
      rating: review.rating,           // 1-5
      seller_id: review.seller_id,
      listing_id: review.listing_id,
      comment: review.comment?.slice(0, 200),  // capped — don't ship full essays through analytics
    },
  }),
});

2. Webhook verification (optional but recommended). Agentry signs all webhook payloads — verify in your Slack-side proxy:

async function verifyAgentryWebhook(req: Request): Promise<boolean> {
  const sig = req.headers.get("x-agentry-signature") ?? "";
  const body = await req.text();
  const key = await crypto.subtle.importKey(
    "raw",
    new TextEncoder().encode(process.env.AGENTRY_WEBHOOK_SECRET!),
    { name: "HMAC", hash: "SHA-256" },
    false,
    ["sign"],
  );
  const mac = await crypto.subtle.sign("HMAC", key, new TextEncoder().encode(body));
  const expected = Array.from(new Uint8Array(mac))
    .map((b) => b.toString(16).padStart(2, "0"))
    .join("");
  return sig === expected;
}

3. Tune the threshold. Default 3× over baseline + floor of 5 is conservative — alerts ~once per month on a healthy marketplace. If you’re paging too often, raise to 5×; too rarely, lower to 2×.

Variations

• “Add a per-seller spike detector too: any single seller getting >5 negative reviews in 1 hour, page seller-support.”
• “Cross-reference the spike against recent deploys — if a deploy was in the last 4h, link to the deploy in the Slack message so engineering can roll back.”
• “Run the same logic but for dispute_opened events — fraudulent buyers sometimes mass-dispute too.”
• “During the spike, auto-mark the affected listings as ‘in review’ and stop new payments until a human looks.”