Agentic AI

Ng's course describes three practical degrees: low (the developer codes the sequence of LLM calls and tool calls), medium (the LLM chooses which of a fixed set of tools or steps to run and in what order), and high (the LLM plans, may write code or new tools, and adapts when a step fails). Module 5 covers the high end under 'Patterns for highly autonomous agents'. The course's own summary: agentic workflows 'plan multi-step processes, execute them iteratively, and improve outputs through reflection and tool use'.

The comparison comes from Ng's March 2024 letter in The Batch, using the HumanEval benchmark of coding problems. Other benefits the course lists: agentic workflows can run steps in parallel (fetch nine web pages at once and then synthesise), they are modular (swap a component without rewriting the system), and they can exceed human speed on repetitive multi-step jobs. The cost is more LLM calls per task, which is why Module 4 spends time on latency and cost.

Ng's standard illustration is essay writing: research, outline, first draft, self-critique, revision, rather than asking for the essay in one shot. The course extends this to business processes: identify where a human would iterate (reflection), where they would look something up or act (tool use), and where the sequence is not knowable in advance (planning). Steps that are independent should run concurrently. The decomposition also decides your eval strategy: every step boundary is a place to add a component-level check (section 7).

Ng's one-line definitions: Reflection, the LLM examines its own work to find ways to improve it. Tool use, the LLM is given tools such as web search, code execution, or any other function to gather information, take action, or process data. Planning, the LLM comes up with and executes a multi-step plan to achieve a goal. Multi-agent collaboration, more than one AI agent work together, splitting up tasks and discussing and debating ideas. The course builds each from first principles in Python before introducing frameworks, so the learner understands what the framework is hiding.

Ng's canonical flow for code: generate code for task X; prompt 'check the code carefully for correctness, style, and efficiency and give constructive criticism'; feed the critique back and ask for a rewrite; repeat. The Module 2 lesson 'Why not just direct generation?' shows the quality gap, and 'Evaluating the impact of reflection' measures it rather than assuming it. External feedback examples from the course: execute generated SQL against the database and return errors or empty results; run unit tests; render a chart and have the model inspect the image. Reflection can be one model talking to itself or two roles (author and reviewer), which is the bridge to multi-agent designs.

Mechanics from the course and Ng's Batch letter: tool descriptions (name, purpose, argument schema) are placed in the model's context; the model outputs a formatted call such as {tool: web-search, query: ...}; post-processing detects it, runs the function, and appends the result for the next model turn. With hundreds of tools, a retrieval step selects the relevant subset, the same idea as RAG. Code execution lets the model write Python that calls thousands of library functions, which the course recommends over hand-building a tool per edge case. MCP (Model Context Protocol) defines how tools are described and invoked so one server works with many agents; ByteByteGo's summary is that MCP handles tool access while agent-to-agent protocols handle agent communication.

Course guidance in order: (1) define what success looks like, (2) build a mini-eval of 20 to 50 examples for fast signal, (3) pick objective checks (code compares to ground truth) where an answer is verifiable and subjective checks (an LLM judge with a structured rubric of discrete yes/no criteria such as 'has a clear title', 'correct axis labels') where it is not. End-to-end evals are expensive and tell you whether the system works; component-level evals isolate the underperforming step and direct effort precisely. Keep eval data separate from any examples used in prompts. Rerun the suite on every prompt, model, or tool change.

Method from the course: build the workflow end to end quickly, run it on a small set, and manually inspect each run's trace, comparing each component's output to what a competent person would have produced. Tag the first failing component. Aggregate the tags to find the component that fails most often, then apply component-level evals to that step so improvement can be measured. The four remedies for an LLM component are refining the prompt, switching models, decomposing the step into smaller steps, and fine-tuning. Tool components fail differently (bad inputs, timeouts, missing permissions) and are fixed in code.

The course's two modes: build (create the end-to-end system quickly, accept rough edges) and analyse (examine outputs, read traces, run small evals on 10 to 20 samples). The cycle tightens over time as evals grow and components stabilise. On performance: latency typically matters more than cost because it degrades the user experience and bottlenecks the run; parallel execution of independent tasks (the nine-page fetch example) is the first optimisation. Cost levers, in order of quality risk: fewer unnecessary calls, caching, smaller models for easy components, quantised or distilled models (see Module 2, section 11).

In the course's framing, planning is the LLM autonomously deciding the sequence of steps for a larger task, adapting when things do not go as expected. It is less mature than reflection and tool use and produces less predictable behaviour, so use it where the decomposition cannot be specified in advance. The course suggests combining LLM planning with hard-coded discrete steps so component-level error analysis remains possible. Planning with code execution: the model writes a short program that calls library functions (pandas for data, date libraries for calendars), which handles edge cases the team never enumerated; the program runs in a sandbox and errors are fed back for a rewrite (reflection again).

Ng's three reasons to split work across agents: ablation studies (for example in the AutoGen paper) show multiple agents outperform one; an LLM prompted to focus on one thing at a time performs better; and roles are a useful abstraction for decomposing a problem, like a manager organising a team. His software example staffs a software engineer, a product manager, a designer, and a QA engineer, each with its own system prompt. In the course, specialists are exposed as callable tools so the manager can plan across them, producing non-linear execution. Communication patterns range from sequential handoffs to hierarchical delegation to group chat; the further toward free interaction, the more evaluation and tracing you need. Protocols: MCP for tools, A2A and similar for agent-to-agent messages.