At Sierra, we enable companies to build their own branded, customer-facing AI agents for customer service, commerce, and more. Sierra’s agents are intricate software systems that simulate a human type of reasoning, decision-making, and creativity. Our agents can solve a wide range of real-world problems, assisting people with tasks that range from the mundanely logistical (like exchanging too-tight shoes) to the surprisingly complex (such as navigating the acoustic idiosyncrasies of setting up premium speakers).
There's an enormous difference between building a demo and deploying an AI agent at scale. An engineer can build a demo showcasing the potential of agents over the course of an hour. In fact, ChatGPT can do so with a fairly well-directed prompt. But enabling agents to operate at scale consistently - across millions of conversations - proves much more challenging.
We’ve found that building and shipping delightful agents requires not only a new software development approach, but also a new type of software engineer.
Meet the agent engineer
At Sierra, we’ve built a team of agent engineers – individuals who work with our customers to design, build, and ship agents using Sierra’s platform: Agent OS.
We see our agent engineers as a part of the broader AI Engineering sub-discipline. These engineers work at the forefront of what’s possible with AI and are amongst the first to productionize LLMs to solve consumer problems at scale.
I, myself, am one of these agent engineers. While building and shipping agents, I’ve had the opportunity to see many businesses up close and apply frontier AI technology to their thorniest problems. There’s a particular thrill in the moments that you see the AI you've designed solve a problem so efficiently that it leaves a customer delighted.
Taking an example from my own work with Sonos to accelerate “time to music,” I helped build an AI agent that can troubleshoot speaker connection issues. The process involved comprehensive scoping with our technical counterparts to understand the details of the Sonos product portfolio. Then, we translated our understanding of the problem into an AI agent that could solve it repeatedly, for nearly any customer.
Other agent engineers at Sierra have designed and deployed agents that tackled similarly complex problems. For example:
- Refreshing your SiriusXM radio, which requires sending a signal from a satellite to your vehicle.
- Helping you choose a pair of OluKai shoes to wear to your next event or on your next vacation.
- Scheduling a furniture delivery, navigating door codes and the complexity of fitting an expensive sofa through your apartment door.
Agent engineers need to be strong software engineers, but their role differs from other engineering roles in key ways. AI agents are nondeterministic and built on tools uncommon in modern software stacks. In addition to programming, Agent engineers must master a modern AI stack, developing expertise in:
- Large Language Models: Frontier models like GPT-4o, Claude 3.5 Sonnet, and Gemini, alongside smaller, more specialized models.
- Vector Databases: Data stores for semantic retrieval of relevant context.
- Prompt Engineering: Standard techniques like few-shot prompting, in-context learning, and self-consistency.
- Agent Architecture: Tool-calling optimizations like Toolformer or reasoning improvements from processes like Reflexion (authored by Sierra’s own Noah Shinn).
- AI Orchestration Engines: Tools like LangChain, Flowise, or Sierra’s Agent SDK, which help coordinate different tools and models.
Agent engineers hone the performance of agents with the rigorous training and selection of underlying models, the authoring of thoughtful and domain-specific prompts, the implementation of agent orchestration layers, and the integration of relevant tools and external APIs.
Beyond technical competence, agent engineers need to possess innate curiosity and respect for the complexity of their customers’ challenges. Their role requires understanding and applying expertise in a given domain to the development of an AI agent.
Agent engineering in practice
To capture the types of technical challenges agent engineers face in building and deploying agents, we’ve detailed below three areas our agent engineers think about frequently: Interacting with Systems, Agent Supervision, and Agent Extensibility. These are the tip of the iceberg for the evolving role of Agent Engineer, but they provide a flavor for the type of work our team does.
Interacting with systems: “How can my agent know where a customer’s package is?”
Simplistic AI agents in the market today are built solely on Retrieval-Augmented Generation (RAG), an approach that relies on selected customer documents to respond to a user’s query. At its simplest, RAG helps an agent answer questions like: “What is your return policy?” using a company’s actual written return policy.
But regardless of how advanced a RAG solution or an underlying LLM might be, a solution without system integrations can’t return your package or cancel your account. Without access to the underlying APIs and systems of record necessary to complete the transaction, a service experience ends at question and answer. You can’t change your flight, return your shoes, or change your subscription plan.
But even with access to systems via APIs, agents with simplistic architectures don’t achieve the consistency necessary to deploy agents in production. In practice, most workflows require agents to orchestrate multiple API calls in a particular order to achieve a desired outcome. Function calling and more advanced techniques like ReAct remain the public state of the art, but they don’t perform sufficiently for production workflows.
To address these shortcomings, we developed the Sierra Agent SDK. The Agent SDK is a powerful framework that enables Agent Engineers to construct agents by stacking composable skills and enforcing deterministic API Interactions. The Agent SDK powers the orchestration necessary to manage agent control flow, ensuring that Sierra’s agents access the right systems of record at the right time.
This configurability extends all the way to the agent responding to complex user-specific rules, such as regional regulations or plan-specific policies. In many customer service settings, a user from the UK might be subject to different rules or require the use of different APIs than one from the US. Agent engineers work with our customers to understand the relevant policies across regions and interact with the right systems of record.
Agent supervision: “How do I make the agent safe to deploy at scale?”
One challenge of deploying AI agents at scale is the unbounded nature of language. In production, a customer could say anything to the agent. The challenge is ensuring that the agent always responds appropriately given the context.
One answer to the problem of unbounded inputs and nondeterministic outputs is to layer more AI on top of the existing agent.
To explore a simplified example, imagine that an agent outputs the correct answer 90% of the time, and in the remaining 10% of the time, the agent gets something wrong in its response. If we have a second model, a supervisor, that can verify the output of the first with 90% accuracy and revise, the combined accuracy of the two models can skyrocket to 99%.
At Sierra, we have developed built-in supervisors to ensure that agents perform appropriately. Depending on the context, agent engineers may build custom supervisors for customers, particularly in heavily regulated environments like healthcare or financial services.
In addition to real-time agent oversight, Sierra’s Agent SDK gives agent engineers the flexibility to tune agent creativity and determinism according to the context. In certain cases, for example canceling an account or offering a promotion, higher levels of determinism may be warranted.
For example, particular regulatory requirements might necessitate that in certain scenarios, an agent responds with precise language. In such cases, Sierra’s Agent SDK makes it possible to selectively bypass an LLM to achieve the necessary consistency.
Agent extensibility: “How can I increase the speed of developing agents?”
Some approaches to agent development lead to more flexibility than others. For example, fine-tuning an LLM to a particular use case creates a model that may be well-suited to a particular task, but which also lacks the flexibility or extensibility most agents require.
In contrast, our agent engineers use our Agent SDK to define agent behavior, composing skills that are expressed using a declarative programming language. By building abstractions on common agent behavior, agent engineers create higher order components to further accelerate agent development.
In other words, we’re building with lego bricks instead of pouring concrete. This composability makes agents quicker to build and deploy. It also makes agents much more durable. Over time, new components are developed, added, and extended to modify the agent, enabling it to handle a greater variety of scenarios and tasks.
On the agent engineering team, we build canonical versions of these higher order components and extend them to meet our needs. Our goal is to establish the world-class implementation of each skill to accelerate our customer deployments and build agents of exceptional quality.
Building the future
The role of agent engineering is dynamic. Each day is less a repeat performance and more a re-invention of what it means to build and deploy agents in production. The work itself blends theory with engineering practice – agent engineers must apply frontier technologies to our customers’ most important business challenges.
When successfully developed and deployed, an AI agent becomes more than a tool; it morphs into an integral part of a company’s identity, a cornerstone of its brand, and a key element of its competitive advantage. Tactically, Sierra’s AI agents often double or even triple the resolution rate of existing solutions while increasing customer satisfaction. But this is only the start - agents are able to create delightful, personalized experiences that were the realm of sci-fi only a few years ago.
For those interacting with Sierra’s agents each day, we may be powering their first interaction with this generation of AI. Agent engineers build systems that can surprise and delight customers, delivering the magic that many of us felt when we first interacted with ChatGPT.
A view from the arena
The role of agent engineer isn’t one size fits all. I myself came to the role of agent engineer through an interdisciplinary approach to software engineering. I’ve worked as an infrastructure engineer at Palantir, founded a company, and graduated from business school.
Members of our team have similarly varied experiences and backgrounds. My colleagues have founded companies, led teams as staff engineers, or spent time perfecting their skiing. While non-traditional experiences are not a prerequisite for success, a combination of engineering expertise, customer obsession, and business curiosity define some of the most successful agent engineers.
We are participating in the advent of a new type of engineering, and we expect the role of agent engineer to grow in importance over the coming decade.
If our agent engineers sound like you, and you’re interested in deploying AI agents at scale, we’re hiring and we’d love to hear from you.
