How to Write an AI Agent Specification That Developers Can Actually Build

An AI agent specification that developers can build has five sections: the agent's goal stated as a measurable outcome, the tools and data sources the agent can access, the decision boundaries (what the agent decides alone versus what requires human approval), the error handling strategy (what happens when the agent fails or is uncertain), and the success criteria with specific metrics. Most agent specs fail because they describe personality ("be helpful and professional") instead of behaviour ("query the CRM, check inventory, draft a quote, and route to sales manager for approval if the discount exceeds 15%").

The difference between a spec that gets built and a spec that gets rewritten three times is specificity. Engineers do not need to know what the agent should sound like. They need to know what systems it touches, what decisions it makes, and what happens when something goes wrong.

Why do most AI agent specs get thrown away?

They describe the agent in marketing terms, not engineering terms. "An AI assistant that helps customers find products" gives engineers nothing to build against. There is no defined input, no defined output, no boundary on what "helps" means, and no specification for what "find" involves.

Compare that to: "An agent that accepts a customer query, searches the product catalog by category and price range, returns the top 3 matches with inventory status, and escalates to a human agent if the customer mentions a return or complaint." That is buildable. Every verb maps to an API call, a database query, or a routing decision.

The second pattern fails because product managers and founders write specs the way they think about the product (outcomes and feelings) rather than the way engineers build it (inputs, outputs, conditions, and error states). Both perspectives are necessary. The spec is where they meet.

What goes in the goal section?

One sentence. A measurable outcome. "Reduce average quote generation time from 4 hours to 15 minutes by automating data gathering and draft creation." Not "help the sales team be more efficient."

The goal determines every other decision in the spec. If the goal is vague, every downstream choice (which tools to connect, what decisions the agent should make, what metrics to track) becomes a guess. Engineers will fill in the blanks with their own assumptions, and those assumptions will not match yours.

A good goal has three properties: it names the current state ("4 hours"), the target state ("15 minutes"), and the mechanism ("automating data gathering and draft creation"). The mechanism constrains the scope. Without it, the engineering team may build something that achieves the target state through a completely different approach than you intended.

Examples of goals that work:

• "Reduce inbound support ticket response time from 2 hours to under 5 minutes for the 60% of tickets that match known resolution patterns."
• "Automate lead qualification by scoring inbound leads against ICP criteria and routing qualified leads to the assigned sales rep within 10 minutes of form submission."
• "Generate first-draft proposals for deals under $25K by pulling client history from the CRM, matching to relevant case studies, and producing a scoped document the account manager reviews before sending."

How do you define tool access?

List every system the agent can read from and write to. For each system, specify what data it can access, what actions it can take, what authentication it uses, and what rate limits apply. This is the contract between the agent and your infrastructure.

A tool access table looks like this:

The constraints column is what prevents disasters. Without explicit constraints, a well-intentioned agent with CRM write access could bulk-update deal stages, merge duplicate contacts, or send emails to your entire database. Constraints are not limitations on the agent's capability. They are guardrails that protect your business data.

What are decision boundaries and why do they matter?

Decision boundaries are the single most important section of the spec. They define what the agent can do on its own and what requires a human in the loop. Getting this wrong causes one of two problems: the agent is so restricted it provides no value, or the agent makes decisions it should not be making.

For every action the agent can take, define the autonomy level:

Notice the pattern: low-risk, reversible actions get autonomy. High-value, irreversible actions require human approval. The dollar threshold and the action type both matter. An agent should always be able to escalate to a human. It should never be able to close a deal or issue a refund without approval.

How do you specify error handling?

Error handling is where most specs stop being useful. The happy path is easy to describe. The failure modes are what separate a production agent from a prototype.

Every spec needs to cover three scenarios:

Scenario 1: The agent cannot find the information it needs.

Define the fallback. Does the agent ask the user for the missing information? Does it queue the task for human handling? Does it use a default value? The answer depends on the context. A customer-facing agent should ask. A back-office agent should queue. A pricing agent should never use a default.

Scenario 2: The agent is uncertain about its output.

Define a confidence threshold. Below that threshold, the agent escalates instead of acting. For a lead qualification agent, the threshold might be: "If the ICP score is between 40 and 60 (out of 100), flag the lead for manual review instead of auto-routing." The threshold should be set based on the cost of a wrong decision, not an arbitrary number.

Scenario 3: An external system is unavailable.

Define the retry strategy and the timeout. If the CRM is down, does the agent retry 3 times over 5 minutes, then queue the task? Or does it use cached data from the last successful read? If the email service is down, does the agent hold the message and retry, or skip the email and log a warning? Each external dependency needs its own failure plan.

What success criteria should you include?

Success criteria are how you know the agent is working. Without them, the agent ships and nobody can tell whether it is doing its job or slowly making things worse.

Include both quantitative and qualitative metrics:

Quantitative:

• Task completion rate (what percentage of assigned tasks does the agent complete without human intervention?)
• Average time per task (is the agent faster than the manual process it replaced?)
• Error rate (how often does the agent produce an incorrect output, as measured by human review of a random sample?)
• Escalation rate (what percentage of tasks does the agent escalate to a human? Too high means the agent is not useful. Too low may mean it is making decisions it should not.)

Qualitative:

• User satisfaction, measured by a short survey after agent interactions (not inferred from usage data)
• Output accuracy, measured by human review of a 10% sample each week during the first month, then 5% monthly

Baseline: document what these metrics are before the agent exists. If you do not know the current task completion time, error rate, or escalation rate for the manual process, you cannot measure whether the agent improved anything. Baselining is boring. It is also the only way to prove the agent's value after launch.

Madgeek builds AI agents for production business operations, and every engagement starts with a specification built to this standard. The cost of building an AI agent depends directly on the complexity in the spec: how many tools, how many decision boundaries, and how many failure modes the agent needs to handle. For a deeper look at how production agents are structured, read our resource on AI agent architecture for production systems.