M0.1 — How LLMs Actually Behave

ACT 1: HOOK & ANCHOR — Slides 1–3

_header: "" _footer: ""

_notes: (1) KEY CONCEPTS: Welcome to Module 0.1 — the first module in the Agentic AI Mastery curriculum. Before you touch a single line of agent code, you need a correct mental model of the engine underneath. This module installs that model. Most engineers who struggle with agentic systems are fighting a ghost — their mental model says "deterministic lookup" but the system they built says "probabilistic prediction." Every guard rail, every eval harness, every retry policy you will add across this curriculum flows from understanding what we cover in the next seven hours. The industry signal is strong: "Hands-on experience using LLM APIs in production systems: prompt engineering, structured outputs, function calling, cost management, and evaluation" (Checkr, 2026 JD, jd-002). Twenty-eight of thirty real 2026 job descriptions require this foundation. (2) EMPHASIZE: This module is not about LLM theory. It is about the practical mental model that explains why agents behave the way they do — and how to design around that behavior. (3) MISCONCEPTION: None yet — this is the orientation slide. However, the most common pre-module misconception we anticipate is the Deterministic-Software Misconception: the belief that an LLM, like a traditional function, returns the same output for the same input. We will name and correct this misconception in Act 3. For now, simply note that if you believe LLMs are deterministic, this module is especially important for you. (4) REFERENCES: - [Checkr 2026 JD — jd-002](https://greenhouse.io) "Hands-on experience using LLM APIs in production systems" - [Coverage Matrix — req-002 LLM API integration](../../hiring_map/coverage_matrix.json) 28/30 JDs - [Hiring Alignment Map](../../hiring_map/hiring_alignment_map.md) §3.1 AI Engineer archetype

_notes: (1) KEY CONCEPTS: This is the orientation slide for the entire curriculum's visual language. We are intentional about conventions because the cohort demographic is wide — from career-switchers seeing their first agent code to senior engineers transitioning into ML. Inconsistent visual cues create friction. Every emoji, color, and callout shape in this deck has a defined meaning, and that meaning is preserved across all 20+ modules. (2) EMPHASIZE: Three things to call out aloud: (a) the notes are not optional — they contain the citations, the misconception warnings, and the deeper context that the slide title cannot fit; (b) every 🟪 career anchor maps to a specific 2026 job description requirement we will surface in the notes; (c) if any code block is hard to read on a learner's screen, we patch the theme — we do not ask learners to squint. (3) MISCONCEPTION: "Conventions slides are throat-clearing — skip ahead." No. Skipping this slide is the #1 reason cohort learners get lost in later modules: they miss what 🟨 means, then ignore a misconception warning, then debug the wrong layer for two hours. (4) REFERENCES: - [Career-Forge Slide System Design Spec v1.1](../../slide_system_design_spec.md) §3 Visual Conventions - [Career-Forge Slide Theme v1.1 CSS](../../theme/career-forge-v1.1.css) — canonical color and contrast values

_notes: (1) KEY CONCEPTS: This scenario — the 2 a.m. agent debugging story — is not hypothetical. It is the most common failure mode for engineers new to LLM-powered systems. The code looks correct because, in traditional software terms, it is. But the engineer's mental model is wrong: they are treating the LLM like a database query that returns a deterministic result given a deterministic key. The LLM is not a database. It is a probability distribution over possible next tokens, sampled fresh on every call. "Same prompt, different decision" is not a bug — it is the expected behavior at temperature > 0. This module will make that statement precise and actionable. (2) EMPHASIZE: The root cause of the problem is not a code bug — it is a mental model mismatch. Fixing the mental model fixes the diagnosis, the design, and the debugging approach. (3) MISCONCEPTION: The Temperature-Determinism Misconception (MCN-003): "I set temperature=0 so the output is always identical." This is nearly true but not perfectly true — hardware floating-point non-determinism and batch scheduling mean temperature=0 is near-deterministic, not guaranteed identical. We cover this precisely in Segment 2. For now, plant the seed that temperature=0 is not a guarantee. (4) REFERENCES: - [Russell & Norvig 2020 — Artificial Intelligence: A Modern Approach, 4th ed.](https://aima.cs.berkeley.edu) Chapter 22 on language models and uncertainty - [Anthropic Building Effective Agents 2024-12](https://www.anthropic.com/research/building-effective-agents) §"Model behavior is probabilistic"

_notes: (1) KEY CONCEPTS: Let us anchor why you should invest 7 hours in this module before writing agent code. These numbers come from 30 real 2026 job descriptions harvested from Greenhouse on 2026-06-08 — they are not estimates. LLM API integration appears in 28 of 30 JDs. That is essentially universal. Cost optimization appears in 16 of 30. These are the two signals this module directly plants. Notice that M0.1's coverage scores are 1 and 2, not 3 — this is accurate. M0.1 does not produce a portfolio artifact (that happens from M0.2 onward). What it does is install the mental model without which all subsequent portfolio artifacts would be built on shaky ground. Think of M0.1 as the investment in your conceptual foundation, not the portfolio evidence itself. (2) EMPHASIZE: The 28/30 JD frequency for LLM API integration means this is a universal requirement — not a niche or specialty. Every single AI Engineer, Agentic Systems Engineer, and FDE role expects you to understand how LLM APIs work, what the response metadata means, and how to control output quality. (3) MISCONCEPTION: None anticipated on this slide — this is a data slide. However, a learner might incorrectly assume that coverage score 1 means this module is unimportant. Correct that: score 1 means the concept is foundational to everything that follows, even if M0.1 alone does not produce portfolio evidence. The coverage score measures portfolio evidence, not conceptual importance. (4) REFERENCES: - [Coverage Matrix — coverage_matrix.json](../../hiring_map/coverage_matrix.json) req-002 M0.1=2, req-001 M0.1=1, req-021 M0.1=1, req-006 M0.1=1 - [Hiring Alignment Map §3.1](../../hiring_map/hiring_alignment_map.md) AI Engineer archetype avg 2.8/3.0 - [JD harvest — jd-002 Checkr](https://greenhouse.io) req-002 evidence

ACT 2: PREREQS & BRIDGE — Slides 4–6

_notes: (1) KEY CONCEPTS: This module assumes three things. If any of these are shaky, the labs will be confusing rather than illuminating — you will spend your cognitive budget troubleshooting environment setup instead of building the LLM mental model. Please be honest with yourself. If "create a virtual environment" sounds unfamiliar, spend 30 minutes on the HTTP/JSON primer first. The remedial decks listed are planned but not yet authored — in the interim, the Python docs on venv and the Anthropic quickstart guide both cover these prerequisites adequately. Note that "async/await basics" is also useful background but not strictly required for M0.1 — the labs do not use async. We list it here because M1.2 and M2.1 will need it, so if you know you are weak there, now is a good time to address it. (2) EMPHASIZE: The three prerequisites are binary gates, not nice-to-haves. If you cannot make an authenticated API call before this module, you cannot complete the labs. (3) MISCONCEPTION: Learners sometimes assume "I know Python" is sufficient. The specific requirement is HTTP/JSON fluency plus API key management via .env. A learner who knows Python well but has never worked with REST APIs will struggle with the lab setup even if the Python syntax is familiar. (4) REFERENCES: - [Python venv documentation](https://docs.python.org/3/library/venv.html) - [Anthropic quickstart guide](https://docs.anthropic.com/en/api/getting-started) - [OpenAI quickstart guide](https://platform.openai.com/docs/quickstart) - [python-dotenv README](https://github.com/theskumar/python-dotenv)

_notes: (1) KEY CONCEPTS: Here is where M0.1 sits in the full curriculum. The four milestones form a progression: M0 builds the mental model and toolchain; M1 grounds LLM outputs in external knowledge (RAG and tools); M2 orchestrates multiple LLM calls into agent loops; M3 makes those agents production-grade. M0.1 is genuinely foundational — not because it is first, but because the single idea "LLM outputs are probabilistic" is the premise that motivates every technique in every subsequent module. RAG exists because the model cannot know current facts. Tools exist because the model cannot take actions. Evals exist because the model's outputs are not guaranteed correct. Temperature control exists because the model is not deterministic by default. Once you internalize the probabilistic core, the rest of the curriculum is a series of engineering responses to it. ALT TEXT: A linear diagram showing M0.1 → M0.2 → M1.x Retrieval. A vertical arrow labeled "Mental model (probabilistic)" points up at M0.1. A vertical arrow labeled "Structured output (controlled output)" points up at M0.2. The diagram communicates that each module layer builds on the mental model planted in M0.1. (2) EMPHASIZE: The single insight from M0.1 is: "LLMs produce probability distributions over tokens, not deterministic lookups." Everything else is a consequence. (3) MISCONCEPTION: None anticipated — this is an orientation diagram. But note that some learners may expect a "hello world agent" in this module. M0.1 deliberately does NOT build an agent — it builds the mental model you need before you can reason about what an agent is doing. The agent architecture is introduced in M2.1. (4) REFERENCES: - [Curriculum.json — m0-1-llm-mental-model](../../../repos/career-foundry/backend/content/agentic-mastery/curriculum.json) - [Russell & Norvig 2020 Chapter 22](https://aima.cs.berkeley.edu) agent architectures and probabilistic reasoning

_notes: (1) KEY CONCEPTS: We introduce five terms here before we need them. This is intentional — following Mayer's Pre-training principle, introducing vocabulary before the concept explanation means the vocabulary itself does not become a cognitive obstacle when you are trying to understand a new idea. Read through these definitions now. You do not need to memorize them — they will recur in context throughout the module. But having encountered them once means when you see "temperature" in a code snippet, you have a mental hook to hang the explanation on. Pay special attention to the definition of "hallucination": not a bug, not a flaw, not something that can be patched — it is a structural consequence of next-token prediction. We will spend considerable time on this in Segment 3. (2) EMPHASIZE: "Hallucination is a structural property, not a bug" — this is the most important framing in this table. If a learner comes away from M0.1 believing hallucination can be solved with a better prompt or a better model, they will build dangerously underprotected systems. (3) MISCONCEPTION: The Anthropomorphism Misconception: learners often interpret "hallucination" as the model "making something up" in a human sense — implying intent or awareness of wrongness. The model has no awareness of what it knows vs. does not know. When it produces a confident wrong answer, it is doing exactly what it was trained to do: predict the most plausible-sounding next token. The word "hallucination" is an unfortunate label that encourages anthropomorphism. A better frame: "out-of-distribution plausible completion." (4) REFERENCES: - [Mayer 2009 Multimedia Learning, Cambridge University Press](https://www.hartford.edu/faculty-staff/faculty/fcld/_files/12%20Principles%20of%20Multimedia%20Learning.pdf) Pre-training principle (#7) - [OpenAI API reference — finish_reason](https://platform.openai.com/docs/api-reference/chat/object) - [Anthropic API reference — stop_reason](https://docs.anthropic.com/en/api/messages)

ACT 3: CONCEPT BUILD-UP — Slides 7–14

_notes: (1) KEY CONCEPTS: The most important thing to internalize about tokens is that they are not words and not characters — they are subword units. The model's tokenizer (Byte-Pair Encoding for most modern models) splits text into chunks that are vocabulary entries in the model's learned vocabulary. Common words like "the", "and", "a" map to a single token each. Rare words, technical terms, and non-English text often split into many tokens. The practical implication: you cannot estimate the cost or the context usage of a prompt by counting words. You need to count tokens. The lab (lab1_tokens.py) will make this concrete. The formula to remember: English text is roughly 3–4 characters per token, so a 1,000-word document is roughly 1,300 tokens. At $3 per million input tokens, that is $0.0039 per document — cheap in isolation, expensive at scale. ALT TEXT: A simple two-row table. Top row: the English string "Hello, world!". Bottom row: the same string split into four tokens shown in separate colored boxes: ["Hello", ",", " world", "!"]. Arrows connect the original string to the token boxes. (2) EMPHASIZE: Tokens are the unit of cost and the unit of context budget. Everything else follows from this. A 200K-token context window is not "unlimited" if your system prompt plus retrieved documents plus conversation history already fills 180K. (3) MISCONCEPTION: The Word-Token Equivalence Misconception: "1 word = 1 token." This is wrong in two ways — short common words like "a" or "I" map to one token, but longer words, technical terms, and non-ASCII text may map to many more. The misconception leads to systematic under-counting of token usage, which leads to unexpected context-window overflows in production. (4) REFERENCES: - [OpenAI tiktoken library](https://github.com/openai/tiktoken) token counting for GPT models - [Anthropic token counting docs](https://docs.anthropic.com/en/api/messages#token-counting) anthropic.count_tokens() - [litellm token_counter](https://docs.litellm.ai/docs/completion/token_usage) provider-neutral token counting - [Karpathy, A. Let's build the GPT Tokenizer (2024)](https://www.youtube.com/watch?v=zduSFxRajkE) visual tokenization walkthrough

_notes: (1) KEY CONCEPTS: The context window is not a limit you rarely hit — it is a budget you actively manage. The breakdown in the code block shows all the components that consume input tokens: the system prompt (often hundreds of tokens if well-written), the entire conversation history in multi-turn sessions, all retrieved documents in RAG pipelines, plus the user's message. On a simple two-turn conversation these numbers are small. In a multi-agent loop that runs 10 iterations, retrieves three documents per iteration, and maintains conversation history, you can hit 100K input tokens per request easily. This is why M2.2 (Memory & State) dedicates an entire module to context-budget management. For now, just internalize the budget model: every component of input competes for the same fixed budget, and the output is whatever is left. ALT TEXT: A budget equation diagram. A rectangle labeled "Context Window (e.g., 200K tokens)" is divided into sections. The input section (shaded blue) shows four layers stacked: system prompt, conversation history, retrieved documents, and user message. The output section (shaded green) is at the end. An arrow points from "total budget" to the sum of all these sections. (2) EMPHASIZE: The context window is a shared budget between input and output. Long inputs leave less room for output. Truncation at `finish_reason=length` is a production bug, not a graceful fallback — treat it as an error. (3) MISCONCEPTION: The Unlimited-Context Illusion: "Claude has 200K tokens, so I don't need to worry about context." This becomes dangerous the moment you add RAG (thousands of tokens of retrieved text), multi-turn history, and tool results to the same call. A 200K window fills faster than most engineers expect. (4) REFERENCES: - [Anthropic API messages reference](https://docs.anthropic.com/en/api/messages) context window limits - [OpenAI API reference](https://platform.openai.com/docs/models) GPT-4o context limits - [litellm docs — context window](https://docs.litellm.ai/docs/exception_mapping) context window error handling - [Anthropic Building Effective Agents 2024-12](https://www.anthropic.com/research/building-effective-agents) context management section

_notes: (1) KEY CONCEPTS: The sampling mechanism is the core insight of this module. When the model has processed all the input tokens, it produces a vector of probabilities — one probability per entry in its vocabulary (tens of thousands of entries). At temperature=0, the model always picks the most probable token at each step, producing near-deterministic output. At temperature=1, it samples from the distribution as-is, which means low-probability tokens can occasionally win. Temperature is a multiplier on the logits (pre-softmax scores): dividing by a low temperature makes the distribution steeper (concentrates probability on the top token); dividing by a high temperature makes it flatter (spreads probability more evenly). The lab will demonstrate this empirically: run the same prompt five times at temperature=0 (all outputs identical or nearly so) and five times at temperature=1 (outputs vary). This is not a quirk — this is the model working as designed. (2) EMPHASIZE: Temperature is a design decision, not a dial to set-and-forget. For every agent component, ask: "What is the cost of false positives versus false negatives here?" Set temperature accordingly. Policy decisions = low temperature. Creative drafts = moderate temperature. (3) MISCONCEPTION: The Temperature-Determinism Misconception (MCN-003): "temperature=0 guarantees identical output." Near-true but not perfectly true — hardware floating-point operations are not perfectly deterministic across batches and hardware configurations. In practice, temperature=0 with the same model version and same inputs will produce the same output 99.9% of the time, but you should not write production code that assumes perfect determinism. Build tests around output *behavior* (does it follow the policy?), not output *text* (is it character-for-character identical?). (4) REFERENCES: - [Vaswani et al. 2017 — Attention Is All You Need](https://arxiv.org/abs/1706.03762) original transformer paper — temperature in the softmax - [Anthropic Building Effective Agents 2024-12](https://www.anthropic.com/research/building-effective-agents) temperature and reliability - [OpenAI API reference — temperature parameter](https://platform.openai.com/docs/api-reference/chat/create#chat-create-temperature) - [Karpathy Neural Networks: Zero to Hero series](https://karpathy.ai/zero-to-hero.html) sampling and temperature explained intuitively

_notes: (1) KEY CONCEPTS: The training cutoff is a hard constraint, not a soft one. The model's weights were frozen on a specific date — typically 6 to 18 months before the model was released. Everything that happened after that date is unknown to the model. More importantly, even within the training window, the model only knows what was publicly available on the internet (or in licensed training data). It does not know your company's refund policy unless that policy was in a public document that happened to be scraped into the training data. For a customer-operations agent, this means the model has zero knowledge of your product catalog, your customers' order histories, your current pricing, or your policy documents — unless you put them in the context window. This motivates the entire M1 milestone: RAG and tools exist to fill the gap between "what the model was trained on" and "what it needs to answer this specific question." (2) EMPHASIZE: The model does not know your business. Never. It can pattern-match on your business's content if you provide it in the context, but it has no persistent knowledge of your data. This is why grounding (RAG + tools) is not optional for production systems — it is the architectural response to the training-cutoff constraint. (3) MISCONCEPTION: The Training-Knowledge Misconception: "The model was trained on so much data, it probably knows [specific fact]." Engineers often underestimate how specific their production use cases are. A model trained on the public internet has no knowledge of your company's Q3 2026 pricing sheet, your custom product codes, or your specific escalation policy. Assume zero knowledge; explicitly provide what is needed. (4) REFERENCES: - [Lewis et al. 2020 — RAG: Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://arxiv.org/abs/2005.11401) foundational RAG paper — the engineering response to knowledge cutoff - [Anthropic Building Effective Agents 2024-12](https://www.anthropic.com/research/building-effective-agents) grounding and tool use - [Schick et al. 2023 — Toolformer](https://arxiv.org/abs/2302.04761) teaching models to use tools as the response to limited knowledge

_notes: (1) KEY CONCEPTS: Hallucination is one of those words that shapes how people think about LLMs, often in misleading ways. The word implies the model is doing something abnormal — having a psychedelic experience. It is not. It is doing exactly what it was trained to do: generate the most plausible continuation of the input sequence. When that input sequence contains a question the model cannot answer from its weights, the "most plausible continuation" happens to be a confident, fluent, wrong answer. There is no error signal inside the model. The model does not feel uncertain. It generates the same way regardless of whether it "knows" the answer or not. The code block above illustrates this: from the model's perspective, the task is always "what is the most probable next token?" The answer to "what was Acme Corp's refund rate" is treated identically to "what is 2+2" — both are just pattern completion. The practical consequence: you cannot trust LLM output for factual claims without grounding and verification. (2) EMPHASIZE: Hallucination is structural, not a bug. The engineering response is grounding (RAG, tools) and verification (output eval, citation enforcement, confidence thresholds). No prompt, no matter how cleverly written, eliminates hallucination risk — it only changes its frequency and type. (3) MISCONCEPTION: The Patch-the-Hallucination Misconception: "If I write a better prompt that says 'be accurate and factual,' the model will stop hallucinating." This is the most dangerous misconception in LLM engineering. Better prompts reduce hallucination frequency on known-good distribution, but they do not eliminate the structural mechanism. The model does not have access to ground truth. It cannot be instructed into factual accuracy when the facts are outside its training distribution. (4) REFERENCES: - [Yao et al. 2023 — ReAct: Synergizing Reasoning and Acting in Language Models](https://arxiv.org/abs/2210.03629) tool use as a response to hallucination - [Lewis et al. 2020 — RAG paper](https://arxiv.org/abs/2005.11401) grounding as hallucination mitigation - [Anthropic Building Effective Agents 2024-12](https://www.anthropic.com/research/building-effective-agents) verification and output safety - [Russell & Norvig 2020 — AIMA 4th ed.](https://aima.cs.berkeley.edu) probabilistic reasoning and knowledge representation

_notes: (1) KEY CONCEPTS: This concept surprises engineers who have used chatbot UIs that feel like they remember everything. Those UIs are managing the messages array for you. The model API itself is completely stateless — every call starts fresh with whatever messages you send. If you do not include previous turns in the messages array, the model has no knowledge of them. This has three practical consequences for agent design. First, every message sent to the model consumes tokens — so you need a strategy for managing growing conversation history (context compaction, summarization, vector-backed retrieval) rather than naively appending all history. Second, "memory" in an agentic system must be explicitly engineered — it does not happen by default. Third, bugs caused by missing history (like an agent that forgets what tool it just called) are extremely common and stem directly from this statelessness. M2.2 (Memory & State) builds the full memory architecture. For now, remember: the messages array IS the memory. Period. ALT TEXT: Two separate boxes side by side labeled "API Call 1" and "API Call 2." Call 1 shows a messages array with one item: "My name is Alex." Call 2 shows a separate messages array with one item: "What is my name?" An X mark between the boxes indicates no data flows between them. A note below shows the correct pattern: Call 2 should include both messages in its array to allow the model to answer the name question. (2) EMPHASIZE: There is no magic link between API calls. The messages array is the complete state of the "conversation" from the model's perspective. If you want continuity, you must engineer it. (3) MISCONCEPTION: The Stateful-LLM Misconception (MCN-004): "The model remembers our previous conversation." This is the most common misconception among users who have interacted with polished chatbot UIs. The chatbot UI is managing the messages array; the model API is stateless. If you build on the API directly and forget to include history, you will see exactly the behavior described in lab3_hallucination.py — the model simply cannot answer questions about prior context. (4) REFERENCES: - [Anthropic API messages reference](https://docs.anthropic.com/en/api/messages) messages array and conversation structure - [Russell & Norvig 2020 — AIMA 4th ed. Chapter 2](https://aima.cs.berkeley.edu) stateless vs. stateful agents - [Anthropic Building Effective Agents 2024-12](https://www.anthropic.com/research/building-effective-agents) memory and state management

_notes: (1) KEY CONCEPTS: The finish_reason field is one of the most underused — and most important — fields in any LLM API response. Let us walk through each value. "stop" / "end_turn" means the model completed its generation naturally and the output is likely valid. "length" means the model hit the max_tokens limit you set — the output may be cut off mid-sentence, mid-JSON, mid-tool-call. This is NOT acceptable for structured output or tool calls; you must raise an error and either increase max_tokens or shorten your input. "tool_calls" / "tool_use" means the model has decided to invoke a tool — your agent loop must extract the tool arguments, call the actual function, and inject the result back into the messages array before calling the model again. This is the core of the tool-use loop you will build in M1.2. "content_filter" means a safety filter blocked the response — you must handle this gracefully, not crash. The lab4_stop_reason.py lab builds a wrapper that makes this explicit: fail loudly when stop_reason is unexpected, rather than silently proceeding. (2) EMPHASIZE: The finish_reason field is the status code of your LLM call. Ignoring it is like ignoring HTTP status codes — you will process 500 errors as if they were 200 OKs and wonder why your system produces garbage. (3) MISCONCEPTION: The Finish-Reason-Optional Misconception: "I only need to read the content, not the metadata." This is wrong as soon as you build anything that loops or handles structured output. A truncated JSON response at finish_reason=length looks like valid content — it has characters, it has brackets — but it is broken. The only way to catch this early is to check finish_reason before reading content. (4) REFERENCES: - [OpenAI API reference — finish_reason](https://platform.openai.com/docs/api-reference/chat/object) - [Anthropic API reference — stop_reason](https://docs.anthropic.com/en/api/messages) - [Anthropic Building Effective Agents 2024-12](https://www.anthropic.com/research/building-effective-agents) stop reason handling - [litellm completion response format](https://docs.litellm.ai/docs/completion/output) provider-neutral finish_reason

ACT 4: FORMATIVE CHECK #1 — Slides 15 (Understand level)

_notes: (1) KEY CONCEPTS: Pause here before we move to code. Formative checks in this curriculum serve a specific function: they surface misconceptions before you spend effort writing code that embeds those misconceptions. If you get question 1 wrong and immediately proceed to the worked example, you will build code around a false mental model. That is much harder to fix later than catching the misconception now. The full check (in the JSON file) has three multiple-choice questions targeting the Understand Bloom level plus one short-answer question. The short answer asks you to explain in your own words why LLM output is probabilistic — if you cannot write that explanation clearly, you are not yet ready for the Apply-level work that follows. Take 8 minutes. Check yourself honestly. The answer and distractor explanations are in the JSON file and the companion Markdown file. (2) EMPHASIZE: This check is formative — it is for you, not for grading. The distractor explanations are the most valuable part: each wrong answer is tagged to a specific misconception, so even wrong answers tell you something useful about your mental model. (3) MISCONCEPTION: Some learners skip formative checks, treating them as optional review. They are not optional in terms of learning value. Empirically, learners who skip formative checks between concept and code sections are more likely to build code that embeds the exact misconceptions the checks are designed to surface. (4) REFERENCES: - [Anderson & Krathwohl 2001 — Bloom's Revised Taxonomy](https://quincycollege.edu/wp-content/uploads/Anderson-and-Krathwohl_Revised-Blooms-Taxonomy.pdf) Understand level: "explain," "interpret," "classify" - [Mayer 2009 Multimedia Learning](https://www.hartford.edu/faculty-staff/faculty/fcld/_files/12%20Principles%20of%20Multimedia%20Learning.pdf) Segmentation principle — checkpoint before new content - [checks/check_after_act_3.json](../checks/check_after_act_3.json) Full formative check with distractor explanations

ACT 5: WORKED EXAMPLE + LIVE CODE — Slides 16–20

_notes: (1) KEY CONCEPTS: This worked example builds the `complete_safe` wrapper from lab4_stop_reason.py. It is the first real production pattern in the curriculum — a thin defensive layer around every LLM call that makes stop_reason handling explicit. The sequence diagram shows the full request-response cycle: your code calls litellm, which forwards to the provider, which returns a response object containing both the generated content and the stop_reason. Your code then branches on stop_reason. This branching is what makes the wrapper "safe" — it ensures that every caller of complete_safe either gets valid content or an explicit exception, never silently truncated output. ALT TEXT: A UML sequence diagram with three participants: Your Code (left), litellm (center), and API Provider (right). An arrow from Your Code to litellm labeled "completion(messages, max_tokens)". An arrow from litellm to API Provider labeled "POST /v1/messages". A dashed return arrow from API Provider to litellm labeled "response (content + stop_reason)". A dashed return arrow from litellm to Your Code labeled "response object". Then a decision fork: if stop_reason is "end_turn", a green arrow labeled "return (content, usage)" with a checkmark; if "length", a red arrow labeled "raise UnexpectedStopReason" with an X; if "tool_use", a blue arrow labeled "execute tool → continue loop." (2) EMPHASIZE: This wrapper is not a toy — it is a pattern you will use in every subsequent module. Getting the stop_reason check right here means every piece of code you write on top of it inherits that safety property. (3) MISCONCEPTION: The Try-Except-All Misconception: "I'll just wrap everything in a try/except Exception and log the error." This swallows the specific information that stop_reason provides. An UnexpectedStopReason exception at "length" tells you to increase max_tokens or shorten input; a generic exception tells you nothing useful. Named, specific exceptions are the professional pattern. (4) REFERENCES: - [litellm docs — completion](https://docs.litellm.ai/docs/completion) - [Anthropic Building Effective Agents 2024-12](https://www.anthropic.com/research/building-effective-agents) reliable LLM call patterns - [Anthropic API messages reference](https://docs.anthropic.com/en/api/messages) stop_reason values

_notes: (1) KEY CONCEPTS: These 10 lines establish the foundation every subsequent lab builds on. Let us be precise about each choice. dotenv / load_dotenv() is the standard Python pattern for local secrets management — the .env file is gitignored, so your API key never reaches version control. In production, you would use a secrets manager (AWS Secrets Manager, GCP Secret Manager, HashiCorp Vault) instead of .env, but the code reads from os.getenv() in both cases, so the interface is identical. The MODEL variable has a default of "gpt-4o-mini" so the code runs for anyone who has an OpenAI API key, but the DEFAULT_LLM_MODEL env var lets you swap to claude-sonnet-4-6 or any litellm-supported model without changing code. The UnexpectedStopReason exception is a named, specific exception — not a generic Exception. This matters because callers can catch it specifically and respond appropriately. (2) EMPHASIZE: The env-var seam pattern — model configuration via environment variable with a sensible default — is a professional pattern worth internalizing. It makes the code configurable and testable without hardcoding provider logic. (3) MISCONCEPTION: The Hardcoded-Key Misconception: "I'll just put the API key in the code for now and clean it up later." In 12 years of security incidents, "clean it up later" almost never happens, and the key almost always ends up committed to git. Use .env from day one. (4) REFERENCES: - [python-dotenv documentation](https://github.com/theskumar/python-dotenv) - [Twelve-Factor App — Config](https://12factor.net/config) configuration via environment variables - [OWASP Secrets Management Cheat Sheet](https://cheatsheetseries.owasp.org/cheatsheets/Secrets_Management_Cheat_Sheet.html)

_notes: (1) KEY CONCEPTS: Here is the heart of the wrapper. The key architectural decision is in line 7: we read finish_reason before we read content. This is the correct order. If finish_reason is "length", the content field may contain a valid-looking but truncated string — a JSON object with a missing closing brace, for example. Processing that as valid output is a bug that will surface hours later in a confusing way. By checking finish_reason first, we fail immediately with a clear error message that tells the engineer exactly what happened and what to do about it. The error message is also important: "Increase max_tokens or shorten your prompt" is actionable. Compare that to a generic "LLM call failed" — one tells the engineer what to do, the other does not. The final return is a tuple of content and usage. Returning usage is important because it lets callers accumulate token costs across a multi-turn loop — a pattern you will use in M1.2 and M2.1. (2) EMPHASIZE: The order matters: check finish_reason before reading content. Always. This is not optional defensive programming — it is the correct program flow for LLM API calls. (3) MISCONCEPTION: The Content-First Misconception: "The content field always has valid output after the API call succeeds." False — the API call "succeeds" (HTTP 200) even when finish_reason is "length" or "content_filter." The HTTP status code tells you about the network call, not about the output quality. finish_reason is the output quality signal. (4) REFERENCES: - [OpenAI API reference — choices[0].finish_reason](https://platform.openai.com/docs/api-reference/chat/object) - [Anthropic API reference — stop_reason](https://docs.anthropic.com/en/api/messages) - [litellm finish_reason documentation](https://docs.litellm.ai/docs/completion/output) - [Anthropic Building Effective Agents 2024-12](https://www.anthropic.com/research/building-effective-agents) error handling patterns

_notes: (1) KEY CONCEPTS: These three failure modes are not hypothetical — they are the three most common reasons agentic systems fail in the first month after deployment. Silent truncation is the most insidious because the system appears to work — it produces output — but that output is corrupted. A JSON response truncated at "length" will pass the HTTP-level success check but fail to parse at the JSON level, producing a confusing exception deep in your application code rather than at the LLM call boundary. Temperature variance causing wrong policy decisions is the failure mode from the 2 a.m. debugging scenario we opened with. Context overflow is a longer-term problem that surfaces as the system ages: conversation history grows, retrieved documents are added, and eventually the model starts ignoring early context because the provider has silently truncated it. The diagnosis patterns — log finish_reason on every call, set temperature=0 for policy decisions, pre-count tokens — are habits you should form from day one. Lab 4 gives you the tools to do all three. (2) EMPHASIZE: All three of these failure modes are preventable with the patterns in this module. Log finish_reason, set temperature deliberately, count tokens proactively. (3) MISCONCEPTION: The Works-In-Dev Misconception: "It passed my tests, so it must be fine." All three of these failure modes are much more common at production scale than in development. Truncation is rare with short prompts; variance is rare when you test with the same 5 prompts; context overflow is impossible on a fresh conversation. Production workloads have long prompts, diverse inputs, and extended conversations. Test for failure modes specifically, not just the happy path. (4) REFERENCES: - [Anthropic Building Effective Agents 2024-12](https://www.anthropic.com/research/building-effective-agents) failure mode taxonomy - [litellm token_counter](https://docs.litellm.ai/docs/completion/token_usage) pre-call token counting - [OpenAI API reference — max_tokens best practices](https://platform.openai.com/docs/guides/text-generation)

ACT 6: FORMATIVE CHECK #2 — Slide 21 (Apply level)

_notes: (1) KEY CONCEPTS: This is the Apply-level check. Instead of asking you to recognize a description of a concept (Understand level), it asks you to diagnose a real code problem. The bug has a specific technical cause that maps to what we covered in this module. Do not just guess — work through it systematically. What does line 8 do? It reads the content without checking finish_reason. What does max_tokens=50 do? It sets a very tight output budget. What happens when the output is longer than 50 tokens? The finish_reason is "length", the output is truncated, but the code at line 8 reads content as if it were valid. The fix is to check finish_reason before reading content — exactly what complete_safe does. There are plausible wrong fixes listed in the JSON file, each tagged to a specific misconception: raising max_tokens (treats the symptom, not the cause), adding a try/except (catches the wrong layer), removing max_tokens entirely (a cost bomb). None of these is the right fix. The right fix is to check finish_reason. (2) EMPHASIZE: The correct diagnosis is: "The code does not check finish_reason, so it silently processes truncated output as valid output." The correct fix is to check finish_reason == 'length' and either raise or retry. This is exactly the complete_safe pattern. (3) MISCONCEPTION: Learners applying the Try-Except Misconception will add a try/except around the content access. This does not fix the bug because there is no exception — the truncated content is a valid string. The bug is not a Python error; it is a semantic error (invalid output treated as valid). (4) REFERENCES: - [Anderson & Krathwohl 2001 — Bloom's Revised Taxonomy](https://quincycollege.edu/wp-content/uploads/Anderson-and-Krathwohl_Revised-Blooms-Taxonomy.pdf) Apply level: "execute," "implement," "use" - [checks/check_after_act_5.json](../checks/check_after_act_5.json) full debug question with common_wrong_fixes - [OpenAI API reference — finish_reason](https://platform.openai.com/docs/api-reference/chat/object)

ACT 7: PRACTICE BRIEF — Slides 22–24

_notes: (1) KEY CONCEPTS: This is the hands-on practice brief. It is not a demonstration to watch — it is a task to execute. The assignment is intentionally simple: run the same prompts at different temperatures, observe the outputs, and write a paragraph explaining what you see. But "simple" does not mean "obvious." Most learners are surprised by the results: the factual prompt is consistent at temperature=0 and variable at temperature=1; the creative prompt is more interesting at temperature=1; the ambiguous prompt can produce radically different answers at any temperature. Writing the explanation paragraph is the highest-value part of this assignment — it forces you to articulate your mental model in words, which surfaces any remaining gaps. The test file (from the lesson) checks that you have produced a CSV with exactly 30 rows and that the factual prompt at temp=0 is consistent. Run those tests before submitting. (2) EMPHASIZE: The paragraph is not optional and not a formality. Write it before you look at the grading criteria. If you cannot explain the variance pattern in 100 words without looking at the answer, you have not yet internalized the concept. (3) MISCONCEPTION: Some learners will run all 30 calls in one function with a single loop and not examine the individual outputs. The point is NOT to produce a CSV mechanically — it is to observe what actually happens when you vary temperature on real prompts and to build intuition. Pause and read the outputs as you go. (4) REFERENCES: - [M0.1 lesson file — HW-0.1 specification](../../../repos/career-foundry/backend/content/agentic-mastery/lessons/M0.1-llm-mental-model.md) §Homework - [litellm docs — temperature parameter](https://docs.litellm.ai/docs/completion/input) - [Anthropic API messages](https://docs.anthropic.com/en/api/messages) temperature parameter

_notes: (1) KEY CONCEPTS: The rubric distinguishes Proficient from Exemplary on each criterion. Most learners should aim for Proficient on the first attempt. Exemplary on "variance analysis" requires a specific insight: prompt ambiguity is a source of output variance independent of temperature. At temperature=0, an ambiguous prompt still produces variable output compared to an unambiguous prompt at the same temperature — because the model's probability distribution is already more diffuse over the vocabulary for ambiguous inputs. This insight goes beyond "higher temperature = more variance" and reflects a genuine understanding of the sampling mechanism. The "stop_reason handling" exemplary criterion — "raises with an actionable message that names the fix" — is training a habit that will serve you across the entire curriculum. An error message that says "try increasing max_tokens" is infinitely more useful than one that says "unexpected stop reason." (2) EMPHASIZE: The rubric criterion on "mental model accuracy" is specifically about avoiding anthropomorphism: "the model decided" versus "the model sampled a token from the distribution." These are different framings with real implications for how you will design systems and debug failures. The first implies intentionality; the second implies mechanism. Mechanism thinking is more useful for engineering. (3) MISCONCEPTION: None anticipated on this slide — it is a rubric. But watch for learners who get Proficient on the CSV but skip the paragraph. The paragraph is where the conceptual learning is evidenced. (4) REFERENCES: - [M0.1 self-assessment rubric](../../pedagogical/self_assessment_rubrics/M0.1_self_assessment.md) companion rubric for self-scoring - [Anderson & Krathwohl 2001](https://quincycollege.edu/wp-content/uploads/Anderson-and-Krathwohl_Revised-Blooms-Taxonomy.pdf) Apply level behavioral indicators

_notes: (1) KEY CONCEPTS: Two to three hours is a realistic estimate for a learner who has the prerequisites but is new to the litellm seam. If you are spending more than 30 minutes on setup issues (API key not loading, module not found errors), that is a signal to check the prerequisites checklist — the issue is probably a missing package or a .env file not in the right location. Until the cohort chat ships, [GitHub Issues](https://github.com/drdgreed/career-foundry/issues) is the interim peer-help venue — file an issue tagged with the module slug (e.g. `module:m0-1`). The in-page AI tutor (PR #17) handles real-time questions; the cohort chat and remedial pointer system are 🚧 in development — track them on [ROADMAP.md](https://github.com/drdgreed/career-foundry/blob/main/ROADMAP.md). If you are doing this entirely self-paced, the reflection prompts in M0.1_reflection.md can help unstick the analytical paragraph — particularly "Sense-Making Prompt 2" which asks you to specify temperature for a real use case you care about. That usually unlocks the thinking needed for the paragraph. (2) EMPHASIZE: The setup check — "can you run lessons/m01/lab1_tokens.py?" — is a binary gate. If yes, your environment is ready. Do not spend time debugging environment issues that lab1 would have already caught. (3) MISCONCEPTION: None anticipated — this is a logistics slide. However, learners may underestimate the 30-call time because API calls sometimes take 3–10 seconds each. At worst, 30 calls × 5 seconds = 150 seconds (2.5 minutes) of waiting. Plan for it — run the calls in the background while you start writing the paragraph. (4) REFERENCES: - [M0.1 reflection prompts](../../pedagogical/reflection_prompts/M0.1_reflection.md) Sense-Making Prompt 2 - [litellm quickstart](https://docs.litellm.ai/docs/) setup guide - [Anthropic quickstart](https://docs.anthropic.com/en/api/getting-started) API key setup

ACT 8: REFLECTION & BRIDGE — Slides 25–26

_notes: (1) KEY CONCEPTS: These three reflection questions follow the structured reflection format that closes every module in the curriculum. They are not rhetorical. Write down your actual answers before starting the lab. The first question forces you to anticipate where your implementation will differ from the worked example — this is prediction-before-action, which research shows significantly improves retention compared to post-hoc reflection. The second question is the professional transfer question: where in your real work does this module's content apply? If you cannot answer this, you have not yet made the knowledge actionable. The third question is explicitly permission to not know things yet. Writing down your confusions at this moment — before the lab distracts you — captures the exact gap in your understanding. After the lab, return to this question and note whether the confusion resolved through doing. If it did not, that is a specific thing to ask the in-page AI tutor or to file as a GitHub Issue against the module. (2) EMPHASIZE: The reflection is not complete until you have written answers to all three questions. Thinking them without writing produces less learning than writing produces. This is documented in the andragogical research on self-directed adult learning. (3) MISCONCEPTION: None anticipated on this slide — it is a reflection prompt. However, learners sometimes skip reflection because it feels like "extra work" that delays the lab. Reframe: the reflection reduces total study time by catching confusion before it embeds in code. A misconception caught in 5 minutes of reflection is far cheaper to fix than one caught in 2 hours of debugging. (4) REFERENCES: - [M0.1 reflection prompts](../../pedagogical/reflection_prompts/M0.1_reflection.md) Post-module integration prompts 1–3 - [M0.1 self-assessment rubric](../../pedagogical/self_assessment_rubrics/M0.1_self_assessment.md) competency self-scoring - [Kolb 1984 — Experiential Learning](https://citt.it.ufl.edu/resources/course-development/the-learning-process/types-of-learners/kolbs-four-stages-of-learning/) Reflective Observation closing the experiential cycle - [Knowles 1980 — The Adult Learner](https://elearningindustry.com/the-adult-learning-theory-andragogy-of-malcolm-knowles) self-directed reflection as metacognitive practice

_notes: (1) KEY CONCEPTS: The closing slide does two things. First, it is honest about M0.1's portfolio signal: this module scores 1 or 2 on coverage, not 3, because it does not yet produce a portfolio artifact. That is correct. M0.1's value is the mental model it installs — the thing that makes every artifact you produce from M0.2 onward more credible and better engineered. When a hiring manager in a technical screen asks "why is LLM output non-deterministic?" — this module is your answer. Second, the "what is next" bridge is a Kolb Concrete Experience seed for M0.2: you have just spent 7 hours understanding why LLM output is probabilistic; M0.2 teaches structured output as the first tool to contain that probabilism. The connection is explicit and motivating. ALT TEXT: A two-column table with headers "Skill demonstrated," "JD frequency," and "Coverage score." Three rows: LLM API integration (28/30 JDs, score 2), Cost/latency/token optimization (16/30 JDs, score 1), Python proficiency (30/30 JDs, score 1). Below the table: "Archetype strongest fit: AI Engineer — Generalist (2.8/3.0)." (2) EMPHASIZE: The portfolio signal from M0.1 is indirect but real. Interviewers probe LLM fundamentals because shallow knowledge of LLMs is one of the most reliable ways to filter out candidates who have used LLMs but do not understand them. M0.1 is your preparation for those questions. (3) MISCONCEPTION: Learners sometimes feel disappointed that M0.1 does not produce a GitHub artifact they can point to. Reframe: your understanding of probabilistic LLM behavior IS the artifact — it is the thing that will show through in every interview answer, every code review comment, every system design decision you make in subsequent modules. Portfolio artifacts come from M0.2 onward. (4) REFERENCES: - [Coverage matrix — M0.1 row](../../hiring_map/coverage_matrix.json) req-002=2, req-021=1, req-001=1 - [Hiring alignment map §6 — M0.1 talking points](../../hiring_map/hiring_alignment_map.md) - [Kolb 1984](https://citt.it.ufl.edu/resources/course-development/the-learning-process/types-of-learners/kolbs-four-stages-of-learning/) CE seed for next module