Skip to main content

Ghostwriter’s performance review

Ghostwriter is an AI teammate that helps our customers build their agents. It works alongside them in Slack and Teams, watches how their agents perform, and suggests improvements that help them serve their customers better.

Like any teammate, Ghostwriter gets a performance review, so we asked five customers for feedback. One theme runs through all of their answers: by the time someone sits down to look at a problem, Ghostwriter has usually found it, understood the cause, or drafted a change to fix it.

Finding problems before anyone reports them

Most teams learn about problems with their agent when a customer complains. Ghostwriter reviews conversations, releases, and metrics continuously against the goals each team sets, raising issues in the team’s channel. At ShipStation Global, that has cut down how much the team reviews by hand.

Quote from Karissa Wright at ShipStation Global

Because Ghostwriter is always watching, it catches problems even when no one is around to look. One morning at Carta, it posted that the chat agent was failing to create support cases. It had already found the likely cause and listed the affected customers, and it later confirmed when things recovered.

Quote from Colleen Lowder at Carta

Getting to the root cause

When something looks off, the slow part is often working out why. Ghostwriter traces the problem to its source, whether that’s how the journey was built or a system the agent depends on, and lays out what it found in the thread. At Tilt, a question sent from a phone was enough to get an issue diagnosed and handed off.

Quote from Dela Diaz at Tilt

For more on how Tilt built its agent, starting with voice, read Behind the Build: How Tilt made its hardest channel the test for everything else.

Drafting the fix

Ghostwriter doesn’t stop at the diagnosis. It drafts the change, tests it, and tags the right person to review it, whether it spotted the problem itself or someone on the team asked. For Airtable, the result is speed.

Quote from Sean Simons at Airtable

Because each fix arrives already tested, the ShipStation Global team spends less time digging through journeys and settings to work out why the agent did what it did.

Quote from Gina Alexander at ShipStation Global

Checking the result

After a change goes live, Ghostwriter keeps track of each experiment and release and reports back on whether it made a difference. For Imprint, which partners with some of the largest brands in the world, that’s how they maintain quality even as more people are working on their agent.

Quote from Jonathan Hart at Imprint

Taken together, that’s a strong first performance review, and it’s only the start. The longer Ghostwriter works with a team, the more it learns how that team works, so we’re even more excited for the next one.

Subscribe to the Sierra blog

Get notified about new product features, customer updates, and more.

Discover what Sierra can do for you

Find out how Sierra can help you deliver better outcomes with AI.