ShipStation's Slack IT agent kept 812 of 1,221 chats out of Jira

AWS published a customer post on 13 October 2026 describing Deckhand, an IT help desk agent that ShipStation Global's engineers built for Slack on Amazon Bedrock AgentCore. The post was co-written with ShipStation staff, and the numbers below are theirs. Deckhand launched in April 2026, and in its first three months it handled 1,935 messages and started 1,221 agent conversations, according to the post. Only 409 of those conversations became Jira Service Management tickets.

The problem and the build

ShipStation says its help desk Slack channel drew more than 1,500 threads in the 12 months before launch, with volume growing. More than a fifth of questions arrived outside US business hours, and answers were scattered across Confluence articles, intranet pages, PDFs and old Slack threads.

Deckhand has four parts, per the post. A Slack service on AWS App Runner receives events and attaches the requester's profile (name, email, time zone, manager). A Strands Agents agent runs on AgentCore runtime using Anthropic's Claude on Amazon Bedrock. A single Lambda function behind AgentCore Gateway exposes Model Context Protocol (MCP) tools for classifying requests, creating and updating tickets, and analysing screenshots. A Bedrock knowledge base indexes Confluence, the intranet, Slack and marketing pages. The model ID is an environment variable, so the team can switch models without changing code.

Mining old Slack threads

The post calls resolved Slack threads the most valuable source. The team used DSPy's Recursive Language Model to classify each exported thread as resolved, unresolved or escalated, counting a thread as resolved only when the full fix is visible in it. For each resolved thread, sub-calls extract a summary, the resolution, whether the fix is self-service, the steps and any error messages, validated against a Pydantic schema with retries on failure. The output goes to S3 as markdown for the knowledge base.

Rules enforced in code, and when to stay quiet

The post says prompt rules do not replace access controls. Ticket creation requires the employee to press a Create Ticket button. Classification is a separate Bedrock call whose output is validated before it reaches Jira. The screenshot tool may only download from Slack's file host, at most five images of 5 MB each. Credentials sit in AWS Secrets Manager, with a scoped IAM role per service and tool.

Deckhand's first decision on every message is whether to answer. It outputs SKIP for announcements, social messages and conversations between people. In threads it replies only to the original poster, stays silent once a human helper has replied, and ignores messages that start by @naming a person.

The numbers

• Volume: about 30 messages a day at launch, up to 100 by early summer; 1,935 messages in the first three months.
• Coverage: Deckhand replied to 97.5% of messages. About a third of questions arrived after hours, and the post says Deckhand answered 94% of those.
• Quality: positive ratings went from 82% in April to 96% in June, after the team revised the system prompt and knowledge base in response to complaints.
• Deflection: of 1,221 agent conversations, 409 became Jira tickets and 812 were handled in chat. At ShipStation's figure of about 30 minutes to file, work and close a ticket, that is roughly 406 hours over three months, or about 1,600 hours a year.

What the post leaves out

• Cost. There is no figure for model, AgentCore, App Runner or knowledge-base spend, so the saved hours cannot be set against a running cost.
• Correctness. The 96% figure is employees' thumbs-up rate, not an audit of whether answers were right, and the post gives no count of wrong or harmful answers.
• The ticket maths. The 812 chats are counted as avoided tickets, but the post does not say how many users would have filed a ticket anyway, and the 30-minute handling time is ShipStation's own estimate.
• A definition. The introduction says Deckhand resolved 97.5% of queries, while the outcome section says it replied to 97.5% of messages. A reply is not a resolution, and the 409 tickets show about a third of conversations still needed a human.

The practical lesson is in the design more than the headline: a bot in a shared channel needs an explicit rule for staying quiet, side effects such as ticket creation sit behind a confirmation, and the thumbs-down data is what moved the rating from 82% to 96%.