Course · 10 modules · 90 lessons · 645 min

AI Agent Concepts

Foundations of autonomous AI agents — reasoning, planning, memory, tool use, multi-agent systems, and safety.

Foundational Concepts

·Action Space DesignThe action space defines everything an agent can do — the set of tools, their parameters, and their constraints — and its design is a critical balancing act between expressiveness (enough tools to be useful) and tractability (few enough tools for the LLM to select correctly).9 min→·The Agent LoopThe agent loop is the observe-think-act cycle that drives all AI agent behavior — a repeated process where the agent perceives its environment, reasons about the next step, executes an action, and feeds the result back into the next iteration.7 min→·Agent State ManagementAgent state management is the practice of tracking, organizing, and maintaining all information an agent needs across turns — including conversation history, working memory, task progress, and environmental snapshots — because the LLM itself retains nothing between calls.9 min→·Agent vs. WorkflowAgents and workflows represent two fundamentally different approaches to automating tasks — agents use LLM reasoning to dynamically decide each step, while workflows follow predetermined coded paths — and choosing between them is one of the most consequential architectural decisions in building AI systems.8 min→·The Autonomy SpectrumAI agent systems range from human-driven copilots that suggest completions to fully autonomous agents that independently execute multi-step tasks, and choosing the right level of autonomy is a core design decision with direct implications for safety, cost, and user trust.8 min→·Determinism vs. StochasticityLLM-based agents are inherently non-deterministic — the same input can produce different outputs across runs due to temperature sampling, floating-point arithmetic, and tool output variability — and managing this stochasticity requires deliberate strategies while recognizing that some randomness is actually beneficial.10 min→·Environment and ObservationsAn agent's environment encompasses everything it can perceive and interact with — tool outputs, API responses, file systems, user messages, and system state — and the quality of an agent's behavior depends heavily on how observations from this environment are processed, filtered, and presented to the LLM.9 min→·Goal SpecificationGoal specification is the process of translating human intent into machine-actionable objectives — through system prompts, user instructions, success criteria, and multi-turn refinement — and the gap between what the human means and what the agent understands is the single largest source of agent failure.9 min→·LLM as Reasoning EngineThe large language model serves as the "brain" of an AI agent — providing language understanding, common-sense reasoning, planning, and few-shot learning — but it is a stateless, probabilistic engine with fundamental limitations that the agent architecture must compensate for.10 min→·What Is an AI Agent?An AI agent is a software system that uses a language model to autonomously perceive its environment, reason about goals, and take actions — going beyond simple question-answering into sustained, goal-directed behavior.7 min→

Reasoning And Planning

·Chain-of-Thought in AgentsChain-of-thought reasoning within agent loops provides structured scratchpad space where agents reason through complex decisions before acting, reducing errors in multi-step tasks by making the reasoning process explicit and auditable.7 min→·Error Detection and RecoveryError detection and recovery is the agent's ability to recognize when actions fail or produce incorrect results, classify the type of failure, and apply appropriate recovery strategies ranging from simple retries to full replanning.8 min→·Inner MonologueInner monologue is the agent's private reasoning process -- hidden thoughts that guide decision-making without being exposed to the user, enabling more honest deliberation, safety checks, and complex multi-step reasoning behind the scenes.7 min→·MetacognitionMetacognition is the agent's ability to reason about its own reasoning -- knowing what it knows, what it does not know, when to ask for help, and how to route tasks based on honest self-assessment of its capabilities and limitations.8 min→·Plan-and-ExecutePlan-and-Execute separates strategic planning (deciding what steps to take) from tactical execution (carrying out each step), enabling agents to handle complex multi-step tasks with structured oversight and adaptive replanning.6 min→·ReAct PatternReAct interleaves chain-of-thought reasoning with concrete actions, enabling agents to think about what to do, do it, observe the result, and adjust their reasoning accordingly.7 min→·Reflection and Self-CritiqueReflection enables agents to evaluate their own outputs and actions after the fact, learn from mistakes through verbal self-critique, and improve performance on subsequent attempts without weight updates.7 min→·Task DecompositionTask decomposition breaks complex, ambiguous goals into atomic, executable subtasks, transforming vague instructions like "research topic X" into concrete sequences of retrievable, verifiable actions.7 min→·Tree Search and BranchingTree search explores multiple solution paths simultaneously, allowing agents to consider alternative actions like a chess player evaluating candidate moves, backtracking from dead ends, and selecting the most promising path forward.7 min→·World ModelsA world model is the agent's internal representation of its environment's state, enabling it to predict consequences of actions, maintain awareness of what has changed, and simulate outcomes before committing to irreversible operations.8 min→

Memory Systems

·Conversation ManagementConversation management tracks dialogue state across multiple turns, deciding when to keep full history versus summarize, how to structure system/user/assistant message roles, and how to maintain coherent multi-turn context within token constraints.8 min→·Episodic MemoryEpisodic memory stores records of past interactions, task attempts, and experiences, enabling agents to learn from previous successes and failures and to retrieve relevant episodes that inform current decision-making.7 min→·Long-Term Persistent MemoryLong-term persistent memory enables agents to store and retrieve information across sessions using vector stores, databases, and files, overcoming the ephemeral nature of context windows through systems like MemGPT's hierarchical memory management.7 min→·Memory Architecture OverviewAgent memory architectures mirror cognitive science's division of memory into sensory (raw input buffering), working (active processing), and long-term (persistent storage), with each type serving a distinct purpose and implemented through different technical mechanisms.7 min→·Memory CompressionMemory compression reduces the token footprint of stored information through summarization, hierarchical compression, and selective forgetting, enabling agents to maintain longer effective memories within fixed context window budgets.9 min→·Memory Retrieval StrategiesMemory retrieval strategies determine how agents find the right memories at the right time, combining recency (recent is relevant), relevance (semantic similarity), and importance (scored by impact) into hybrid scoring functions that surface the most useful information.8 min→·Semantic MemorySemantic memory stores factual knowledge -- entities, relationships, concepts, and structured data -- that agents access and update beyond what is encoded in the LLM's weights, implemented through knowledge graphs, databases, and entity stores.7 min→·Short-Term Context MemoryShort-term context memory is the agent's working memory implemented through the LLM context window, a finite buffer of tokens that holds the current conversation, recent observations, and active reasoning, constrained by the fundamental limit of context window size.7 min→

Tool Use And Integration

·API IntegrationAPI integration connects AI agents to external services via REST and GraphQL endpoints, handling authentication, rate limiting, pagination, and error recovery to enable real-world action.6 min→·Browser AutomationBrowser automation enables agents to navigate websites, fill forms, click buttons, and extract information from web pages — turning the entire internet into a tool surface through either visual (screenshot) or structural (DOM) approaches.7 min→·Code Generation and ExecutionAgents that write and execute code gain a universal tool — computation, data manipulation, visualization, and system interaction all become possible through generated programs run in sandboxed environments.7 min→·Dynamic Tool CreationDynamic tool creation enables agents to write, validate, and register new tools at runtime when existing tools are insufficient, turning the agent from a tool user into a tool maker that extends its own capabilities.7 min→·File and System OperationsFile and system operations give agents the ability to read, write, search, and manage files and execute system commands — turning the local file system into the agent's workspace like a desk where it can organize, review, and modify its materials.7 min→·Function CallingFunction calling enables LLMs to generate structured JSON objects that invoke external tools, bridging the gap between natural language understanding and programmatic action.6 min→·Model Context Protocol (MCP)MCP is an open standard that provides a unified protocol for connecting AI models to external tools, data sources, and services through a client-server architecture with three core primitives: resources, tools, and prompts.7 min→·Structured Output for ActionsStructured output techniques — JSON mode, constrained decoding, and schema validation — ensure that LLM-generated actions conform to precise, machine-readable formats, eliminating the fragility of parsing free-text responses.7 min→·Tool ChainingTool chaining is the practice of using the output of one tool as the input to the next, enabling agents to complete complex tasks through sequential multi-step tool invocations that build toward a goal.7 min→·Tool Selection and RoutingTool selection is the process by which an agent examines available tools, matches them against the user's intent, and picks the right one to invoke — a decision that becomes harder as the tool catalog grows.6 min→

Multi Agent Systems

·Agent Debate and VerificationAgent debate uses multiple agents in adversarial or collaborative verification roles — one proposes, another critiques — to catch errors, reduce hallucination, and improve output quality through structured disagreement.7 min→·Agent DelegationAgent delegation is the process by which a manager agent decomposes a complex task into subtasks, assigns each to a specialist sub-agent with a defined scope, collects their results, and synthesizes a final output.7 min→·Consensus and VotingConsensus and voting mechanisms use multiple agents (or multiple samples from one agent) to produce answers, then aggregate them through majority vote, weighted voting, or structured debate to improve reliability — exploiting the statistical principle that independent errors cancel out.8 min→·Hierarchical Agent SystemsHierarchical agent systems organize agents into multi-level structures where higher-level agents decompose tasks and supervise lower-level agents, creating recursive delegation chains with escalation paths — mirroring how organizations manage complex projects through management layers.7 min→·Inter-Agent CommunicationInter-agent communication defines how agents in a multi-agent system exchange information — through direct message passing, shared memory (blackboard), or event-based (pub/sub) patterns — with protocol design determining whether agents use structured formats or natural language.7 min→·Multi-Agent ArchitecturesMulti-agent architectures define how multiple AI agents are organized and coordinated — pipeline, debate, hierarchy, swarm, and blackboard patterns each suit different problem types, much like different team structures suit different organizations.7 min→·Role-Based SpecializationRole-based specialization assigns distinct personas, expertise domains, and behavioral guidelines to different agents in a multi-agent system, improving output quality through focused competence — just as a team of specialists outperforms a team of generalists on complex projects.7 min→·Swarm and Emergent BehaviorSwarm architectures give agents simple individual rules and let complex collective behavior emerge from their interactions — inspired by ant colonies and bird flocks — with OpenAI's Swarm framework demonstrating lightweight agent handoffs, though debugging emergent behavior remains a fundamental challenge.8 min→

Knowledge And Retrieval

·Agentic RAGAgentic RAG puts the AI agent in control of retrieval decisions, dynamically choosing when, what, and how to retrieve information rather than blindly fetching documents for every query.6 min→·Document UnderstandingDocument understanding enables agents to extract structured, actionable information from complex document formats -- PDFs, tables, images, spreadsheets, and scanned documents -- through multi-modal parsing, layout analysis, and structured data extraction.7 min→·Dynamic Retrieval DecisionsDynamic retrieval decisions determine when an agent should fetch external information versus relying on its own parametric knowledge, using confidence-based triggers and retrieval budgets to optimize for both accuracy and efficiency.6 min→·Hybrid Search StrategiesHybrid search combines semantic search (embeddings), keyword search (BM25), and structured queries (SQL/graph) to overcome the individual weaknesses of each approach, using fusion techniques to deliver more robust retrieval results.7 min→·Knowledge Base MaintenanceKnowledge base maintenance is the ongoing operational work of keeping agent knowledge stores accurate, current, and performant through updating stale information, deduplication, versioning, incremental indexing, and contradiction resolution.7 min→·Knowledge Graph NavigationKnowledge graph navigation enables agents to traverse structured entity-relationship networks for multi-hop reasoning, answering questions that require connecting facts across multiple nodes in ways unstructured search cannot reliably achieve.6 min→·Query ReformulationQuery reformulation improves retrieval quality by iteratively transforming user queries into more effective search queries through expansion, decomposition, and hypothetical document generation techniques.6 min→·Source VerificationSource verification ensures agent reliability by cross-referencing retrieved information across multiple sources, detecting contradictions, scoring confidence, and providing citations to prevent the propagation of incorrect or outdated facts.7 min→

Safety And Control

·Agent GuardrailsAgent guardrails are programmable safety layers that intercept agent inputs, outputs, and actions to detect and block harmful, unsafe, or policy-violating behavior through multi-layer defense including input guards, output guards, and action guards.7 min→·Agent SandboxingAgent sandboxing constrains the execution environment of AI agents using container isolation, network restrictions, and filesystem limits to ensure that even if an agent behaves unexpectedly, the damage it can cause is bounded.7 min→·Alignment for AgentsAlignment for agents ensures that AI agents faithfully pursue their intended goals and follow their instructions without gaming specifications, finding loopholes, or optimizing for metrics at the expense of the actual objective, while balancing safety constraints with practical helpfulness.7 min→·Authorization and PermissionsAuthorization and permissions control what resources and actions an AI agent can access, applying the principle of least privilege through scope-based permissions, credential management, and dynamic access control to minimize the damage from agent errors or compromise.7 min→·Human-in-the-LoopHuman-in-the-loop patterns require agent actions to be approved by a human before execution, creating safety checkpoints for destructive, costly, or irreversible operations while balancing safety with usability.7 min→·Monitoring and ObservabilityMonitoring and observability provide real-time visibility into agent behavior through tracing, metrics, anomaly detection, and dashboards, enabling operators to detect problems, understand failures, and maintain production agent reliability.7 min→·Prompt Injection DefensePrompt injection defense protects AI agents from adversarial inputs that attempt to override system instructions, using multi-layer defenses including input sanitization, instruction hierarchy, output monitoring, and architectural isolation to prevent both direct and indirect injection attacks.7 min→·Resource LimitsResource limits prevent runaway agent execution by enforcing token budgets, time limits, cost caps, and iteration maximums, acting as circuit breakers that ensure agents fail safely rather than consuming unbounded resources.7 min→·Rollback and UndoRollback and undo mechanisms enable the reversal of agent actions through version control, database transactions, compensating actions, and checkpoint strategies, ensuring that agent mistakes are recoverable rather than permanent.7 min→·Trust BoundariesTrust boundaries define different trust levels for different data sources entering an agent system -- from high-trust system instructions to low-trust retrieved documents -- and use these levels to govern how the agent processes, weights, and acts on information from each source.7 min→

Evaluation And Testing

·Agent BenchmarksAgent benchmarks are standardized evaluation suites -- including SWE-bench for coding, WebArena for web tasks, GAIA for general assistance, and others -- that provide reproducible task sets with defined metrics, enabling meaningful comparison of agent capabilities and tracking of state-of-the-art progress.7 min→·Agent Evaluation MethodsAgent evaluation methods measure agent performance through end-to-end task completion assessment, step-by-step trajectory analysis, human evaluation, automated metrics, and LLM-as-judge approaches, each addressing different aspects of the fundamental challenge that agents are non-deterministic multi-step systems.7 min→·Cost-Efficiency MetricsCost-efficiency metrics measure agent performance relative to resource consumption -- cost per task completion, tokens consumed, API calls made, and time elapsed -- revealing the Pareto frontier where cheaper approaches with more retries can outperform expensive single-shot attempts.7 min→·Latency and PerformanceLatency and performance metrics measure the time characteristics of agent execution -- time-to-first-action, end-to-end completion time, thinking versus action time -- and navigate the fundamental tradeoff where more reasoning steps produce better quality but slower responses.7 min→·Regression TestingRegression testing for agents ensures that changes to prompts, tools, models, or configurations do not degrade previously working capabilities, using test suites of known-good task completions run through CI/CD pipelines to detect regressions from any source of change.7 min→·Reliability and ReproducibilityReliability and reproducibility measure an agent's consistency across repeated runs, quantifying variance through multi-run success rate distributions, deterministic testing strategies, and the critical insight that a 90% success rate means 1 in 10 production failures.7 min→·Task Completion MetricsTask completion metrics measure agent success through binary (pass/fail), graded (partial credit), and comparative (vs baseline) scoring systems, with domain-specific metrics for coding, research, and customer service tasks, addressing the fundamental challenge of defining what "done" means for diverse agent tasks.7 min→·Trajectory EvaluationTrajectory evaluation assesses the quality of an agent's sequence of actions rather than just its final output, measuring process efficiency, error recovery, and reasoning quality to distinguish good outcomes achieved through sound process from lucky successes masking poor decision-making.7 min→

Infrastructure And Operations

·Agent DeploymentDeploying agents to production involves containerization, scaling strategies, version management of prompts and tools, and operational practices that account for agents being fundamentally harder to deploy than traditional APIs.7 min→·Agent OrchestrationAgent orchestration is the coordination layer that manages execution flow, step scheduling, and state transitions across an agent's lifecycle.6 min→·Context Window ManagementContext window management is the art of selecting, prioritizing, and compressing information to fit an LLM's limited input capacity while preserving the context most critical for the current reasoning step.7 min→·Cost OptimizationAgent cost optimization reduces operating expenses through model routing, caching, prompt compression, batching, and token budgets -- turning agents from expensive experiments into economically viable products.7 min→·Error Handling and RetriesRobust agent execution requires systematic error classification, intelligent retry policies, circuit breakers, and graceful degradation to handle the inevitable failures in multi-step, multi-service agent workflows.7 min→·Event-Driven ArchitecturesEvent-driven architectures enable reactive agents that respond to external triggers -- webhooks, file changes, schedules, user messages -- rather than running continuous polling loops.6 min→·Logging, Tracing, and DebuggingObservability for agents requires capturing structured traces of thought-action-observation chains, enabling developers to replay, diagnose, and optimize multi-step agent behavior.6 min→·State Machines and GraphsState machines and directed graphs provide deterministic control flow structures that make agent behavior predictable, debuggable, and reliable.6 min→

Advanced And Frontier

·Agent DistillationAgent distillation trains smaller, faster, cheaper models on successful trajectories collected from expensive frontier agents, trading inference-time compute for training-time compute to reduce production costs by 10-100x.7 min→·Agent Operating SystemsAgent operating systems provide OS-like abstractions -- process management, memory management, I/O management, and scheduling -- that treat AI agents as first-class computing entities requiring systematic resource coordination.7 min→·Autonomous Coding AgentsAutonomous coding agents write, test, debug, and iterate on code with minimal human intervention, representing the most mature and practically impactful category of AI agents today.7 min→·Computer Use AgentsComputer use agents interact with graphical user interfaces by understanding screenshots, controlling mouse and keyboard, and navigating arbitrary software -- turning any application into a programmable API.7 min→·Deep Research AgentsDeep research agents perform multi-step investigation by iteratively searching, reading, extracting, synthesizing, identifying gaps, and refining -- producing comprehensive reports that would take a human researcher hours or days.7 min→·Embodied AgentsEmbodied agents are AI systems with physical bodies (robots) that perceive the world through sensors, reason with language models, and act in continuous physical space -- bridging the gap between digital intelligence and the real world.8 min→·Generative AgentsGenerative agents are AI entities with persistent personalities, goals, memories, and social behaviors that produce believable human-like conduct in simulated environments through memory retrieval, reflection, and planning.8 min→·Self-Improving AgentsSelf-improving agents get better over time by learning from user feedback, optimizing their own prompts, acquiring new skills, and refining their tool usage -- without requiring manual re-engineering.7 min→·Simulation EnvironmentsSimulation environments are controlled virtual worlds for testing agents before deployment, enabling safe failure, reproducible evaluation, and rapid iteration without real-world consequences.7 min→·Web Navigation AgentsWeb navigation agents browse the web like humans -- filling forms, clicking links, navigating multi-page workflows, and extracting information -- using either screenshot-based or HTML/accessibility-tree-based approaches.7 min→