Triage alerts from your tools
An alert fires at 03:00, and the first half hour goes to the same questions: which service, what changed, is it real. Let a deputy answer them before anyone opens a laptop. Every critical alert starts an investigation, and the on-call engineer gets a diagnosis, the suspect change and a proposed next step.
Available when your administrator enables deputies and webhooks.
The result
- A head start on every alert. The investigation is underway seconds after the alert arrives, and the findings are waiting when a person looks.
- Evidence, not guesses. The run reads the code and the recent merged pull requests, and points at files, lines and commits.
- One investigation per alert. Your alert pipeline can retry as often as it likes. A repeated delivery with the same
idstarts the mission once. - A person decides on changes. The deputy opens an issue with its findings, and asks before it proposes a fix.
How it works
Your alert pipeline posts each alert to a Marshal webhook endpoint named alerts, signed with the endpoint’s secret and the event name alert.fired. Marshal checks the signature and records the delivery. The event trigger on the Alert Triage deputy matches it, checks the condition event.severity == "critical", and maps the alert’s title, service and link into the mission’s inputs. The run starts in a fresh space with your code. It reads the alert, searches acme/web for the code behind the failing service, checks the pull requests merged in the last day, and writes up the cause it finds with the evidence. It opens a GitHub issue with the findings, then asks the on-call engineer whether to prepare a fix. The answer arrives from Home or a Slack button, and the run carries on.
What you use
| Piece | Its role in this story |
|---|---|
| Webhook endpoint | alerts: a delivery URL and a signing secret for your alert pipeline. |
| Event trigger | Starts the mission on alert.fired for critical alerts only. |
| Input map | Passes the alert’s title, service and link into the prompt. |
| Deputy | Alert Triage, owned by the platform team, with its own manifest and memory. |
| Manifest | claude-code, github and your toolchain, with a team GitHub token. |
| Run | One investigation per alert, with a transcript and an audit trail. |
| Home and notifications | Where the on-call engineer answers the deputy’s question, on the web or in Slack. |
Build it
- Create the endpoint. On Personal settings > Webhooks, select Add endpoint, name it
alerts, set the Owner to a team you lead, and choose a Signature format. Copy the Delivery URL and the Signing secret. See Create a webhook endpoint. - Point your alerts at it. A sender that signs with a supported format posts straight to the URL. Otherwise, add a short step to your alert pipeline that signs each alert with Marshal’s format and sends
X-Event-Type: alert.fired. See Sign a request. - Create the manifest and the deputy. Add
claude-code,githuband your toolchain, with your Claude credential and a team GitHub token. Then create the “Alert Triage” deputy for your team. See Create a deputy and a mission. - Write the mission. Create
triage-alertwith the prompt below. In the Inputs panel, makealert,serviceandlinkoptional, each with a default such asunknown. The trigger then fills them from the event. Select Dry run, then Activate. - Add the trigger. Open the mission, select Event under Triggers, set Source to
Custom webhook, Endpoint toalertsand Event toalert.fired. In When, enterevent.severity == "critical"and select Test against real events. Then select Create. - Map the alert into the inputs. With an API key, list the deputy’s triggers with
GET /api/deputies/alert-triage/triggers, then add the map to the new one. See Map event data into mission inputs. - Send a test alert. A run appears on the deputy page with the trigger
webhookand the eventalert.fired.
An example alert body:
{
"id": "al_4821",
"severity": "critical",
"service": "checkout",
"title": "Checkout error rate above 5%",
"link": "https://alerts.acme.dev/al_4821"
}The input map for step 6:
curl -X PATCH https://marshal.example.com/api/deputies/alert-triage/triggers/<trigger-id> \
--header "Authorization: Bearer $MARSHAL_API_KEY" \
--header "Content-Type: application/json" \
--data '{"inputs": {"alert": {"path": "title"}, "service": {"path": "service"}, "link": {"path": "link", "default": "none"}}}'The mission prompt:
A critical alert fired: "{{.alert}}" on the {{.service}} service.
Alert link: {{.link}}
1. Clone acme/web. Find the code that serves {{.service}}.
2. List the pull requests merged in the last 24 hours with gh, and read
the ones that touch that code.
3. Write the cause you find, the evidence (files, lines, commits) and a
suggested fix.
4. Open a GitHub issue titled "Alert: {{.alert}}" with your findings.
5. Ask me with the choices "Prepare a fix" and "Leave it to on-call".
If I pick "Prepare a fix", push a branch and open a pull request.
Never merge.Make it yours
- Widen the net. Use the condition
event.severity in ["critical", "high"], or bind toalert.*to take every alert event. - Read your monitoring data. On a personal deputy, add a monitoring package such as
sentry-mcp,datadog-mcporposthog-mcp, so the run reads stack traces, logs and traces too. Their sign-in acts as you. See Integrations. - Choose who is asked. A mission can name the people it asks, so the question goes to the on-call group. See Who is asked.
- Poll instead of push. Swap the trigger for a schedule every 15 minutes, and have the run read new alerts through a monitoring package. See More recipes.
Next
Create a webhook endpoint for your alerts.
Related