What is an AI agent? How to build one, in plain English

What an AI agent is

A large language model (Claude, GPT, Gemini) predicts text from text. On its own it only produces words; it can't take actions.

An AI agent is a model running in a loop with tools, working toward a goal. You give it the goal. It decides the steps, uses tools, looks at the results and keeps going until it's done.

A chatbot answers questions. An agent carries out tasks: it can search, read a database, draft an email and check its own result.

Agent or workflow?

In a workflow, you script the steps: summarize, then classify, then email. The model fills in the blanks, but the path is fixed. In an agent, the model decides the path while it runs.

Most real business solutions are workflows with an agent step or two, because they are more predictable and cheaper. A good rule of thumb: start with a single prompt, then a workflow, then an agent, then several agents. Each step up adds cost and unpredictability.

How an agent works: the loop

Think, act, observe, repeat.

1. The model reads the goal and the context.
2. It picks a tool and calls it.
3. The result is fed back in.
4. It repeats until it decides it is finished.

The model never runs the tool itself. It writes a structured request, your software runs it, and the answer goes back to the model. That is called tool calling, and it's what turns a chat model into an agent.

The building blocks

• System prompt: the instructions that set the agent's role, rules and tone.
• Context window: how much the model can see at once. Stuffing too much in makes it slower, costlier and worse.
• Tools: the actions it can take, such as search, read a database or draft an email. The model chooses tools by reading their descriptions, so describe each one clearly.
• MCP (Model Context Protocol): an open standard for plugging tools and data sources into AI apps, often described as USB-C for AI tools.
• Skills: packaged instructions an agent loads when a task needs them.
• Structured output: forcing the model to return a fixed format such as JSON, so software can use it reliably.
• RAG: fetch the relevant documents first, then give them to the model with the question, so it answers from your own data.
• Memory: short-term is the current conversation; long-term is saved facts that get reloaded later.

Any term you don't know is in the AI glossary.

Where to build one: three paths

You don't have to write code to build an agent. The concepts are the same on every path: a job, a prompt, tools, guardrails and evals. What changes is who runs the loop and who it serves.

1. Inside an AI assistant (no code). Use the assistant itself as the agent. Projects hold standing instructions and files, skills package a repeatable procedure, connectors plug in tools like email, Jira or Stripe, and scheduled tasks run work without you. The assistant already runs the loop; you supply the job, the instructions and the tools. Best for your own work, internal tools and prototypes. The limit: it runs under your account and isn't embedded in a product.
2. A coding agent working on your files. An agent that works in a project folder or terminal: it reads and edits files, runs commands, and uses skills and connectors. Good for automations, apps and data jobs that touch many files. Best for builders who direct the agent and review the result.
3. An agent SDK or API (standalone). You write a small program that runs the loop on your own server, so it runs without anyone opening an assistant: a ticket arrives, a webhook fires, a timer goes off. You control login, logging, cost caps and what it can touch. Best for production agents, customer-facing features and anything that must live inside your own product.

Where to start: inside an AI assistant. Move to the SDK only when the agent must run with no person involved, live inside your product, serve other people's customers, or need controls you can't get in the app. Many useful business agents never need to leave the first two paths.

How to build one: nine steps

The steps are the same on every path.

1. Define one narrow job with a clear "done", for example "triage support emails and draft replies".
2. Pick a model. Start with a capable one and optimize cost later.
3. Write the system prompt: role, rules, output format, and when to stop or ask a person.
4. Give it the fewest tools it needs, and describe each one clearly.
5. Run the loop: call the model, run the tool it asks for, return the result, repeat until it finishes. Inside an assistant this is built in. In code, SDKs and frameworks such as the Claude Agent SDK, OpenAI Agents SDK and LangGraph handle it for you.
6. Add your own documents with RAG if it needs them.
7. Add guardrails: read-only first, human approval for anything irreversible, and a cap on steps and cost.
8. Write evals: 20 to 50 realistic test cases with expected results, and rerun them after every change.
9. Trace and iterate: read the failed runs, then fix the prompt, the tools or the data.

How agents fail, and how to test for it

• Hallucination: confident but wrong output. Ground answers in your own documents and require sources.
• Prompt injection: malicious text hidden in content the agent reads, such as "ignore your instructions and...". A major security risk for agents that read email or web pages. Limit what the agent can do and keep a person approving risky steps.
• Quieter failures to know: a small early mistake that carries through a long chain of reasoning, agreeing too easily when you push back, material in the middle of a long input counting for less, and facts after the training date missing entirely.

Test it like software. Unit tests check one piece, integration tests check the handoffs, end-to-end tests check the whole journey, regression tests re-check old behavior after a change, and edge cases probe the boundaries. A test suite nobody runs is false security, so write tests in the same pass as the build and run them.

Evals are the unit tests for AI, and the skill employers ask about most. See how to explain RAG and evals in interviews. Log every step and tool call too, so you can see what the agent did when something goes wrong.

The four human skills that matter most

• Delegation: deciding what to hand to AI and what to keep. Hand off implementation, never judgment.
• Description: saying what you want clearly and completely, so the need survives each step from person to AI.
• Discernment: judging what comes back. Is it accurate, does it solve the right problem, is it good?
• Diligence: owning the outcome. Check the work, say where AI helped, and be willing to say no.

These follow the 4D framework in Anthropic's free AI Fluency courses; the wording here is original. The free AI courses page links to them.

What this means for AI Solutions Engineer roles

From a sample of job postings, the day-to-day work of an AI Solutions Engineer looks like this:

• Build and ship: turn approved business use cases into production AI apps, agents, automations and APIs.
• RAG and document workflows: chunking, embeddings, retrieval, citations and validation.
• Integrate: connect AI to existing systems such as ERP, CRM, SharePoint and data platforms.
• Quality and operations: evals, logging, monitoring and error handling.
• Governance: security review, documentation and reusable patterns.
• Stakeholders: discovery, demos, and explaining it to non-technical people.

Pre-sales, executive roadmaps and hiring are usually not required for engineer roles.

Titles you'll see: Associate AI Solutions Engineer, AI Solutions Engineer, then II, Senior, Principal and Director. Variants include AI Full Stack Solutions Engineer, Forward Deployed AI Solutions Engineer (embedded with the customer), Agentic AI Solutions Engineer and AI Enablement. On the architect side: AI Solutions Architect, then Principal or Staff.

Engineers build and ship. Architects usually split two ways: customer-facing pre-sales work (discovery, reference architectures, proofs of concept) or enterprise and program work (roadmaps, governance, steering committees). This is a read of a sample of postings, not a census.

Where to go next

• Take a free course from the major AI providers: free AI courses.
• Build something small you can show: portfolio projects for an AI Solutions Engineer.

If an interviewer asks what an agent is, here is a one-sentence answer you can adapt: an agent is a model using tools in a loop to reach a goal; you decide whether a task needs one or just a fixed workflow, give it the fewest tools possible, and prove it works with evals.