Code415 Radar

Key ideas

Effort level

A request setting (output_config.effort in the Claude API, from low to max) that controls how many tokens the model may spend on thinking, text and tool calls.

Price per token vs cost per task

A lower per-token price can still cost more per task if the model writes more tokens; Artificial Analysis counted about 260M output tokens for Opus 5.5 on its index against an 88M median.

Output token ceiling

Opus 5.5 can emit at most 128,000 output tokens per response, so at max effort a long reasoning trace can run out before any answer appears.

Steps

1. Read the Anthropic announcement (anthropic.com/claude-opus-5-5) and note the $4/$20 price, $0.20 cache reads, the claim of 40% lower cost than Opus 5 on typical workloads, and which effort levels the benchmark charts use.
2. Open the Artificial Analysis model page and put two numbers side by side: the Intelligence Index score (58, first in its class) and the output tokens used to earn it (about 260M vs an 88M median).
3. Read Simon Willison's notes on the pelican SVG test: at max effort Opus 5.5 hit the 128K output limit while still reasoning, cost $2.56 and took almost 20 minutes. Compare that with the timeouts and budgets you already set in your own services.
4. Skim the Claude API effort docs (platform.claude.com/docs/en/build-with-claude/effort) to see how effort is set per request and that it changes all output tokens, not just visible text.
5. Self-check: if max effort scores 5% higher than medium on your task but writes 3x the output tokens, roughly how much more does each correct answer cost?

Steps

1. Run pip install anthropic and set ANTHROPIC_API_KEY in your shell.
2. Save three fixed prompts: a real bug from your repo with the failing code and test pasted in, a request for a 300-word explanation of a topic you know well, and 'Generate an SVG of a pelican riding a bicycle'.
3. For each prompt and each effort level (low, medium, max), call client.messages.create(model='claude-opus-5-5', max_tokens=..., messages=[...], output_config={'effort': level}). Use a generous max_tokens and time each call with time.perf_counter(). If the SDK asks you to stream for a large max_tokens, switch that call to streaming.
4. Log usage.input_tokens, usage.output_tokens, stop_reason and seconds to a CSV. A stop_reason of max_tokens means the answer was cut off; record it as a failure. Cost in dollars = (input_tokens x 4 + output_tokens x 20) / 1,000,000.
5. Grade each answer yourself: does the fix pass your test, is the explanation correct and close to 300 words, does the SVG render and look like a pelican on a bike?
6. Build a table of cost and seconds per passing answer for each effort level, and mark the first level where going higher adds cost without adding passes.

Small angles to try

• Add high effort between medium and max to see exactly where the curve bends.
• Run each prompt and level three times and report the spread, since single runs vary.
• Repeat the bug prompt on the previous Opus model to check the 40% cost claim on your own workload.

Key ideas

Cost per correct answer

Total spend divided by the number of correct outputs, which folds price and accuracy into one number you can compare across models.

Reasoning tokens

Hidden thinking tokens that OpenAI reports in output_tokens_details.reasoning_tokens and bills as output tokens, so a model that reasons longer costs more than its sticker price suggests.

Prompt caching

When many calls share the same prompt prefix, the repeated part can be read from cache; OpenAI says cached input reads now get a 90% discount.

Steps

1. Read OpenAI's announcement: note Sol at $2/$10 and Luna at $0.10/$0.50 per million tokens, the 50% cut versus GPT-5.6, and the 90% discount on cached input reads.
2. Open OpenAI's model guidance page (developers.openai.com/api/docs/guides/latest-model) and note the IDs gpt-6-sol and gpt-6-luna, which reasoning effort values they accept, and that reasoning combined with tool calls needs the Responses API.
3. Read the usage example in OpenAI's reasoning guide and find where reasoning tokens are reported; they count as billed output.
4. Read TechCrunch for launch context, then compare Sam Altman's per-task pricing argument with Anthropic's claim that Opus 5.5 uses fewer tokens per task. Both sides are arguing about cost per task, not per token.
5. Self-check: Luna is 20x cheaper per token than Sol. If Luna gets 80% right and Sol 96%, and fixing each wrong item by hand costs $0.50, which model is cheaper per 1,000 items?

Steps

1. Run pip install openai, set OPENAI_API_KEY, and save your 50 examples as JSONL with a text field and a label field.
2. Write one fixed instruction that lists the allowed labels and asks for JSON output. The GPT-6 family supports Structured Outputs via text.format with a json_schema in the Responses API, which keeps parsing simple.
3. Loop over the examples with client.responses.create(model='gpt-6-luna', input=..., reasoning={'effort': 'low'}), then repeat with model='gpt-6-sol' at the same effort. Check the model guidance page first for which effort values each model accepts.
4. For every call store the predicted label, usage.input_tokens, usage.output_tokens and the reasoning token count.
5. Cost per call = (input_tokens x input price + output_tokens x output price) / 1,000,000, with $0.10/$0.50 for Luna and $2/$10 for Sol. Multiply the average by 1,000 to get cost per 1,000 calls.
6. Compute accuracy and cost per correct answer (total cost / number correct), then list which examples each model got wrong and whether the errors overlap.

Small angles to try

• Keep the fixed instructions at the start of every prompt and rerun to see how much input_tokens_details.cached_tokens cuts the bill.
• Run Luna with effort 'none' and with a higher effort to see if reasoning pays for itself on your data.
• Add Claude Opus 5.5 at low effort as a third column with the same prompts and scoring.

Key ideas

Reverse transcriptase (RT)

An enzyme that copies RNA into DNA; bacteria and phages carry many RT families, some with known defensive roles and many with unknown ones.

CRISPR-like repeat array

A run of short, evenly spaced, nearly identical DNA repeats separated by unique spacers; in CRISPR these store a record of past infections, while in ART the function is not yet known.

Gene neighborhood analysis

Checking which genes repeatedly sit next to each other across many genomes, a standard clue that they work together.

Steps

1. Read Anthropic's announcement and note the three parts of ART (an RT, a partner gene, a repeat array), the search numbers (about 950 agents, 21 hours, 200,000+ RTs, about 3,500 candidates, 20 shortlisted) and what the wet lab confirmed: the array is expressed as distinct short RNAs.
2. Read The Next Web article for outside context: ART lacks cas genes, arrays have 3 to 21 repeats, lab work used a Staphylococcus phage, and Feng Zhang called the RNA-repeat arrays 'genuinely intriguing' while the function is still unknown.
3. Skim the methods section of the preprint PDF linked from the announcement, looking for how candidates were filtered from thousands to 20 and what made one count as unusual.
4. Connect it to something familiar: this is a very large 'find a known gene, look at its neighbors, flag odd patterns' search, the same idea as the small MinCED exercise in the test below.
5. Self-check: what has been shown (the array is transcribed into short RNAs) and what has not (what the system does, and whether the RT acts on those RNAs)?

Steps

1. Install the tools: conda install -c conda-forge ncbi-datasets-cli for the NCBI datasets CLI, and conda install -c bioconda minced for MinCED.
2. On the NCBI website, pick 3 to 5 phage genomes that have RefSeq assembly accessions (GCF_...), for example Staphylococcus phages. Download them with datasets download genome accession <accessions> --include genome,gff3 and unzip the archive.
3. Run minced -minNR 2 <genome>.fna out.txt out.gff on each genome file; -minNR lowers the minimum repeat count so short arrays on small phage genomes are reported.
4. Run grep -i 'reverse transcriptase' on each genome's GFF3 annotation file and note the contig, start and end of every hit.
5. Write a short Python script that lists every MinCED array lying within 5,000 bp of an RT gene on the same contig, and record per genome the number of arrays, RT genes and array-RT pairs. Expect mostly zero pairs; the rarity is the point.

Small angles to try

• Change the window to 2 kb and 10 kb and see how fast chance pairings appear.
• Add one bacterial host genome and compare how many arrays sit near RT genes versus near cas genes.
• Loosen and tighten MinCED's repeat settings and compare how many arrays survive.

Key ideas

Comparison-addition model

A cost model in which an algorithm may only compare and add edge weights and every operation is charged, the setting where Dijkstra's O(m + n log n) was long the bar.

Lean formal verification

The Lean proof assistant mechanically checks every step, so the result is only as strong as the theorem statement; here it covers exactness and the charged running time.

Certified density range

C-HD's advantage is proved only for graphs with m <= n x floor(floor(log2 n)^(3/4)) edges, where the bound becomes O(n log^(11/12) n).

Steps

1. Read the Vals AI blog post 'A Faster Shortest Path Algorithm' (vals.ai/blogs/faster-shortest-path-algorithm) and write down the graph class (directed, non-negative real weights), the cost model and the bound compared with Dijkstra and with Duan et al. 2025 (O(m log^(2/3) n)).
2. Open github.com/spicylemonade/c-hd-proof, read the README's scope section, then open formal/lean/Frontier/CHD/Final.lean where the theorems chd_exact_within, chd_CHDTarget and chd_gateC are stated.
3. List the caveats the authors give: enormous constants, no benchmark on real graphs, advantage only inside the certified range, outside it the algorithm falls back to Bellman-Ford, and no external review yet.
4. Connect to practice: a binary-heap Dijkstra runs in O((n + m) log n) and in real code memory access and constants dominate, so an asymptotic win with huge constants can be slower on every graph you will ever run.
5. Self-check: for n = 1,000,000, what is floor(floor(log2 n)^(3/4)), and how many edges does the certified range allow? (About 9 per vertex, so roughly 9 million.)

Steps

1. Clone github.com/spicylemonade/c-hd-proof, open formal/lean/Frontier/CHD/Final.lean, and write in your own words what chd_exact_within and chd_CHDTarget state: graph class, cost model, density range and bound.
2. Run pip install networkx and build G = networkx.gnm_random_graph(150000, 1000000, seed=1, directed=True); give each edge a random non-negative float 'weight'. Confirm it sits in C-HD's range: 150,000 x floor(17^(3/4)) = 1,200,000 >= 1,000,000.
3. Write your own Dijkstra using Python's heapq over an adjacency list, time a single-source run from node 0 with time.perf_counter, repeat 5 times and keep the median.
4. Check correctness by comparing your distances with networkx.single_source_dijkstra_path_length(G, 0) on the same graph.
5. Record edges processed per second. That is the practical baseline any 'faster' algorithm has to beat on real hardware.

Small angles to try

• Port your Dijkstra to C++, Rust or Go and compare with Python to see how much constant factors matter.
• Hold m/n fixed, grow n by 10x steps and plot runtime to see whether the log factor is even visible.
• If you have an hour, run lake exe cache get and lake build Frontier.CHD.Final inside the repo's formal/lean folder to replay the proof.

Key ideas

Crib

A guessed piece of plaintext, here ROSENOW ROSENOW, whose assumed position lets you test and rule out key settings quickly.

Bombe

The Allied wartime machine that used a crib to eliminate impossible rotor orders and positions; a software Bombe runs the same logic in code.

Enigma key

Rotor order, ring settings, start positions and plugboard pairs together; the plugboard makes brute force enormous, which is why crib-based pruning matters.

Steps

1. Read the Crypto Cellar write-up of the MVUEH break: note the 82-letter message, the crib ROSENOW ROSENOW taken from the related message SIPVX, the recovered rotor order 253, the transcription errors in the ciphertext and the rare left-rotor turnover at position 72.
2. Note what the human did (asked Astra to try unbroken messages, then validated the result) and the open question: Astra's logs cite Bundesarchiv references not published on the Crypto Cellar site.
3. Read Schneier's short post for the reaction from the security community.
4. Read the py-enigma example (from_key_sheet, set_display, process_text) on its PyPI page to see concretely how rotors, reflector, ring settings and plugboard define a key.
5. Self-check: why could one wrong letter in the ciphertext stop a crib search, and how would you make your search tolerate it?

Steps

1. Install py-enigma and choose a secret key: from enigma.machine import EnigmaMachine, then EnigmaMachine.from_key_sheet(rotors='II IV V', reflector='B', ring_settings=[1, 20, 11]) with no plugboard for round one (plugboard_settings defaults to None). Call set_display with a 3-letter start position.
2. Encrypt an 80-letter message that contains ROSENOWROSENOW with process_text, and save only the ciphertext and the crib, not the key.
3. Give the model the ciphertext, the crib and the machine type (Wehrmacht rotors I to V, reflector B) and ask it to write a crib-based search from scratch that recovers rotor order, ring settings and start position, without importing py-enigma.
4. Run its code, time it, and count every bug you had to report back before it found your key. Confirm by decrypting the ciphertext with py-enigma using its answer.
5. Repeat with a new secret key and no crib, and note how the model changes approach (for example statistical scoring of candidate decrypts) and whether it gets there.

Small angles to try

• Add three plugboard pairs and see whether the model's search still finishes in reasonable time.
• Insert one deliberate typo in the ciphertext, as in MVUEH, and check whether the search copes.
• Give two different models the same key and compare bugs and wall-clock time to a correct break.

Key ideas

Control-loop latency

The time from seeing to acting; at speed v the car travels v x latency metres before the next correction.

Closed-loop evaluation

Each command changes what the model sees next, so small errors compound, unlike a static question-and-answer benchmark.

Run-to-run variance

Each model got only a few attempts, so a ranking built on one course and a handful of runs can flip with another try.

Steps

1. Read drivingbench.com: note the setup (steering, throttle and brakes through commands such as set_motion and stop_now, one command at a time, a human supervisor ready to brake), the scoring (progress along the centerline while staying within 4 m) and results (GPT-6 Astra finished in 5:22; Claude Fable 5.1 reached 45%).
2. Read the HN discussion for the numbers critics raised: roughly 0.8 m/s and a course a human drives in about 15 seconds.
3. Connect it to latency budgets you already know: a 3-second p95 API call at 10 m/s means 30 m driven without a correction, which is why real driving stacks run fast local models rather than chat APIs.
4. Self-check: with a p95 latency of 4 s and 0.5 m of allowed drift, what top speed is safe, and how does it compare with walking pace (about 1.4 m/s)?

Steps

1. Take or draw one image of a path between cones (JPEG or PNG, no side over 2000 px) and write a one-line state, for example 'speed 0.5 m/s, heading 0 deg, last steer 0'.
2. Write a prompt that requires exactly one line of JSON of the form {"steer": degrees, "throttle": 0-1, "duration": seconds} and nothing else.
3. Send the image as a base64 image content block ({'type': 'image', 'source': {'type': 'base64', 'media_type': 'image/jpeg', 'data': ...}}) followed by the prompt; time each call with time.perf_counter() and run 20 calls.
4. Parse each reply with json.loads and count invalid replies as failures. Compute p50 and p95 latency; statistics.quantiles(latencies, n=20)[18] gives the 95th percentile.
5. Choose an allowed error (for example 0.5 m) and compute max safe speed = allowed error / p95 latency. Compare it with the 0.8 m/s Astra drove at.

Small angles to try

• Run the same 20 calls on a smaller or faster model and compare max safe speed against JSON validity rate.
• Lower the model's effort or reasoning setting and see how latency and the steering values change.
• Stream the response and measure time to first token as a second latency number.

Key ideas

Automation bias

The tendency to accept a system's output and stop checking it, strongest when the system is usually right.

Stale or contradictory data

Records that were once true or that disagree with other records; a ranking tool will rank them confidently unless it was built to check freshness and conflicts.

Out-of-scope reliance

Using a tool for a job it was not designed for, here expecting it to catch outdated records, which no accuracy figure for its real job can protect against.

Steps

1. Read the Bloomberg investigation (the item's first link, may be paywalled) and note three findings: staff relied on Maven to catch outdated or contradictory records, it recommended a site based on stale intelligence, and civilian-harm review teams had been cut by about 90%.
2. Read the HN thread and sort the top comments into two camps: automation bias, and the view that AI became a scapegoat for gutted review teams and data that never reached the database.
3. Relate it to your own systems: find one place where a dashboard, search ranking or LLM summary sits between a possibly stale database and a person who trusts its output.
4. Self-check: if a system is 95% accurate and reviewers check only 1 in 10 outputs, what share of its errors reaches the final decision? (About 90%.)

Steps

1. Make a 20-row fake supplier table with columns for name, price, delivery days, quality score, last verified date and notes. Plant 3 problems: a last-verified date several years old, a note that contradicts the score (for example 'failed audit' next to 9/10 quality), and the same supplier listed twice with different prices.
2. Prompt A: paste the table and ask only 'Rank the top 5 suppliers and explain briefly.' Run it 10 times in fresh sessions.
3. For each run, record which planted issues the model mentions without being asked and whether any flawed supplier lands in its top 5.
4. Prompt B: same table and question, plus 'Before ranking, list any rows with stale, missing or contradictory data.' Run it 10 times and record the same things.
5. Summarize in a two-row table: flag rate and flawed-in-top-5 rate for prompt A versus prompt B.

Small angles to try

• Move the planted rows from the end of the table to the middle and see if position changes the flag rate.
• Compare two models, or two sizes of one model, on prompt A only.
• Grow the table to 200 rows with the same 3 bad rows and see whether the unprompted flag rate drops.

Key ideas

Logits

The raw score a language model gives every possible next token before those scores are turned into probabilities.

Softmax over allowed choices

Take only the logits of the answer tokens you allow (here 'A', 'B', 'C') and normalize them so they sum to 1, which forces the model to pick from your list.

Calibration and expected calibration error (ECE)

A model is calibrated when answers given with 80% confidence are right about 80% of the time; ECE is the size-weighted average gap between confidence and accuracy across confidence bins.

Steps

1. Read the NobodyWho post 'Jev in 25 lines' and find the two lines that matter: the prompt, which uses Qwen's chat template with an empty <think> block, and logits = model.scores[model.n_tokens - 1], which reads the scores for the next token.
2. Notice what is compared: only the first token of each label ('A', 'B', 'C'). The probabilities are relative to those three, so the model can never answer 'none of these', and a value of 0.9 only means 'A beat B and C', not '90% sure in the real world'.
3. Connect it to something familiar: this is a multiple-choice classifier head built from a vocabulary subset, much like a logistic regression output. Both give a number between 0 and 1, and neither is a trustworthy probability until you check it against real outcomes.
4. Skim the HN discussion for the main objection: the 25-line version skips the calibration training Jev advertises (the post names it 'Reinforcement Learning for Calibrated Decisions' and admits the scores are 'not always correct').
5. Self-check: the script says 0.9 'spam' on 50 messages and 30 of them are actually spam. What is the calibration gap in that bin, and would accuracy alone have shown it?

Steps

1. Copy the script from the NobodyWho post into jev.py. It carries inline dependency metadata, so uv run jev.py installs its packages and downloads Qwen3-0.6B-Q8_0.gguf from the Hugging Face repo Qwen/Qwen3-0.6B-GGUF on first run. Confirm it prints probabilities for the sample email.
2. Cut the choices to two ('A. Legitimate', 'B. Spam') and wrap the scoring in a function that takes one message and returns P(spam). Load the model once, outside the function.
3. Load the Hugging Face dataset ucirvine/sms_spam with the datasets library (columns sms and label, where 0 = ham and 1 = spam). Sample 50 ham and 50 spam with a fixed random seed so the set is balanced and repeatable.
4. Score all 100 messages. Time each call with time.perf_counter() and store P(spam), the predicted label and the true label. Report accuracy plus median and 95th-percentile latency per item.
5. Compute ECE with 10 equal-width bins: for each bin, take |mean confidence - accuracy| and weight it by the share of items in the bin, then sum. Cross-check the reliability curve with sklearn.calibration.calibration_curve(y_true, y_prob, n_bins=10).
6. Swap in Qwen/Qwen3-1.7B-GGUF (file Qwen3-1.7B-Q8_0.gguf), then Qwen/Qwen3-4B-GGUF (file Qwen3-4B-Q4_K_M.gguf), rerun the same 100 items, and put accuracy, ECE and latency for all three models in one table.

Small angles to try

• Fit temperature scaling on 50 held-out messages (one scalar that divides the logits) and see how far ECE drops without changing accuracy.
• Swap the label order so 'Spam' is option A and check whether the probabilities shift, which would indicate position bias.
• Replace SMS spam with 100 of your own emails or support tickets that you label by hand, and compare against a TF-IDF plus logistic regression baseline.

Key ideas

AGENTS.md

A tool-neutral instruction file that several coding agents read at session start; Claude Code reads it directly from v2.1.277 when no CLAUDE.md or CLAUDE.local.md is present.

Remote feature flag

A switch the client fetches from a server at runtime; if the fetch is blocked, for example because telemetry is off, the client falls back to a default, which here was 'feature off'.

Canary test

Put a unique word only in the file you want to check and ask the agent for it; if it can answer without opening files, the file was loaded into its context.

Steps

1. Read the szypowi.cz write-up. Note the exact canary command, the two environment variables that blocked loading (DISABLE_TELEMETRY=1 and CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC=1), and why each setup was run twice: the first session in a new configuration only fetches the flag.
2. Open the Claude Code memory docs (code.claude.com/docs/en/memory) and read the 'AGENTS.md' section. It now states that before v2.1.281 sessions with telemetry disabled, or on Amazon Bedrock, read CLAUDE.md files only, and recommends an @AGENTS.md import in a CLAUDE.md as the workaround.
3. In the same section, note the precedence rule that is not a bug: a CLAUDE.md, .claude/CLAUDE.md or CLAUDE.local.md in your working directory or above it means AGENTS.md is not read by default. The 'Project instructions' setting value claude-md-and-agents-md loads both.
4. Connect it to your own services: this is the classic case of a kill switch whose offline default is wrong, and of a privacy setting that quietly changes functional behavior.
5. Self-check: you are on v2.1.281 with telemetry on, and your repo has an AGENTS.md plus a personal CLAUDE.local.md. Does Claude Code read AGENTS.md by default, and what one-line change makes it do so?

Steps

1. Create an empty folder and run echo 'The canary word is PERIWINKLE.' > AGENTS.md inside it. Make sure there is no CLAUDE.md or CLAUDE.local.md in that folder or any parent, because those would stop AGENTS.md from loading by design.
2. Run the author's check twice with default settings: claude -p 'What is the canary word from the project instructions? Answer NONE if you have none. Do not read files.'. Then run it twice more with DISABLE_TELEMETRY=1 in front of the command. Write down each answer and your installed Claude Code version.
3. If you are below 2.1.281, run claude update and repeat both variants. Expect PERIWINKLE in every run after the first session in each configuration.
4. Test the workaround: run echo '@AGENTS.md' > CLAUDE.md, then repeat with DISABLE_TELEMETRY=1 and again with CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC=1.
5. Apply the same pattern elsewhere: pick another instruction or config file one of your tools claims to read (for example a file under .claude/rules/ or another agent's rules file), put a different canary word in it, ask the same question, and record which files actually load.

Small angles to try

• Put the canary in a subdirectory's AGENTS.md and confirm it only shows up after the agent reads a file in that subdirectory.
• Add a CLAUDE.local.md, confirm AGENTS.md stops loading, then set Project instructions to claude-md-and-agents-md and check again.
• Turn the canary check into a small CI script that fails if the word is missing after a tool upgrade.

Key ideas

Precision vs recall for reviewers

Precision is the share of review comments that point at real issues; recall is the share of real issues that got a comment, so 20% recall means four of five known issues go unflagged.

Hybrid deterministic + agent pipeline

Ordinary code handles file selection, bundling and rule matching, and the LLM agent is only used for the analysis itself, which is where the token savings come from.

Token budget per review

Every file, rule and tool result sent to the model is billed, so a reviewer that sends less context costs less but may miss issues that span files.

Steps

1. Read the README at github.com/alibaba/open-code-review. Separate what is deterministic (file selection, bundling, rule matching) from what the agent does, and note that the '~1/9 of the tokens' claim is against general-purpose agents using the same underlying model.
2. Read the InfoQ article for the setup (200 PRs across 10 languages) and Daniel Vaughan's critique: the best configuration reaches about 20% recall, and the deterministic design limits discovery of cross-file and architectural issues.
3. Connect it to linters and static analysis you already use: they are high precision and narrow in scope. OpenCodeReview sits between a linter and a free-roaming agent, and the benchmark shows that trade-off.
4. Self-check: a tool leaves 5 comments, 4 of them real, on a PR that had 10 real issues. What are its precision and recall, and which number would your team notice first?

Steps

1. Install with npm install -g @alibaba-group/open-code-review, then run ocr config provider and ocr config model. If you can, use the same underlying model you will give Claude Code, so you compare the review harnesses rather than the models.
2. In a local clone of your own repo, pick 3 merged PRs where a bug surfaced later. Before running any tool, write down each known bug in one line with file and line number. This is your answer key.
3. For each PR, create local branches at the PR's base and head commits (for example pr1-base and pr1-head), then run ocr review --from pr1-base --to pr1-head --format json --output pr1-ocr.json.
4. Run Claude Code on the same changes, for example claude -p 'Review the changes between pr1-base and pr1-head (use git diff) for bugs. List each issue with file, line and a one-sentence reason.' --output-format json > pr1-claude.json.
5. Score each tool per PR: true positives (matches a known bug), false positives (comments you judge wrong or irrelevant), and tokens or cost for the run, taken from your model provider's usage page for that time window.
6. Sum across the 3 PRs, compute precision and recall per tool, and note whether the bugs both tools missed were cross-file.

Small angles to try

• Run ocr twice on the same PR to see how stable its comments are from run to run.
• Switch ocr to a smaller, cheaper model and check whether precision holds while cost drops.
• Run ocr scan --path on the module that contained a known bug and compare full-file review with diff-only review.

Key ideas

Agent skill

A folder of instructions and helper scripts a coding agent loads on demand; this one activates when you ask the agent for a security audit.

Coverage ledger

A JSON list of the app's input surfaces and trust boundaries that the hunting agents work through, so the search is driven by coverage rather than by whatever the model notices first.

Independent verification

A fresh agent, ideally running a different model, tries to disprove each finding, because a hunter that grades its own work tends to approve everything.

Steps

1. Read Cloudflare's blog post 'Build your own vulnerability harness'. Note the funnel (20,799 raw candidates, about 12,057 surviving validation), the line that a single run finds only about half the bugs that multiple runs catch, and the choice to use a different model for validation than for discovery.
2. Read the README at github.com/cloudflare/security-audit-skill. Map the six phases to their output files: architecture.md and coverage-ledger.json, findings.json checked against report-schema.json, then REPORT.md, FINDINGS-DETAIL.md and NEEDS-VALIDATION.md. Learn the three verdicts: confirmed, needs_validation and rejected.
3. Connect it to flaky tests and human code review: LLM search is sampling, so the union of several runs finds more than one run, and a second reviewer who did not write the code catches different mistakes.
4. Self-check: why is a verifier running the same model as the hunter weaker than one running a different model, and what number would tell you another run is still worth paying for?

Steps

1. Clone the deliberately vulnerable OWASP Juice Shop source: git clone https://github.com/juice-shop/juice-shop.git --depth 1. The audit reads source code, so this clone is the target. To confirm a finding in a browser, run the app on localhost only with docker run --rm -p 127.0.0.1:3000:3000 bkimminich/juice-shop.
2. Inside the clone, install the skill: npx skills add https://github.com/cloudflare/security-audit-skill --skill security-audit (add --global to install it for your user instead of this project).
3. Start your agent in the clone and ask: 'do a security review, output to ~/audits/juice-shop/run-1'. Let it finish all six phases.
4. Start a new session and repeat with the output set to ~/audits/juice-shop/run-2.
5. Compare the two findings.json files. Match findings by file and vulnerability class, then count only-in-run-1, only-in-run-2 and in-both, and break each group down by verdict (confirmed, needs_validation, rejected).
6. Use Juice Shop's published challenge list as a rough answer key and note which known weaknesses neither run found.

Small angles to try

• Run the verification phases with a different model than the discovery phases and see whether the rejected count changes.
• Do a third run and check whether new confirmed findings keep appearing, which is Cloudflare's signal that coverage is still growing.
• Point it at a small service you own and compare its confirmed findings with a static analysis tool you already use.

Key ideas

Agent runtime sandbox

The container or VM where an agent's tools run; anything readable there can be packed up and sent out if the agent also has an upload or network tool.

Exfiltration through normal tools

No exploit is needed: an ordinary user request, combined with file-read and upload tools, is enough to move data out.

Deny rules vs OS sandboxing

In Claude Code, a Read deny rule blocks the agent's file tools, but stopping a shell command such as cat from reaching the same path needs Bash rules or OS-level sandboxing.

Steps

1. Read the mouse.dev write-up. List what was in the 6.8 GB archive: the session's Ubuntu root filesystem, about 68 skill folders under /opt/hatch/skills/, memory stored as Markdown, 113 subagent records and SSH key files. Also note that the bug bounty report was closed as 'Not Applicable'.
2. Read the TechCrunch piece on Meta Connect (Mac app control, Muse's own email address) and ask yourself what the same request would reach once those features are on.
3. Read the Claude Code permissions docs (code.claude.com/docs/en/permissions), section on Read rules: path prefixes // (filesystem root), ~/ (home) and ./ (project), examples such as Read(./.env), and the note that permission rules are enforced by Claude Code, not by the model. Then skim the sandboxing page linked from there.
4. Connect it to least privilege and to the old mistake of baking secrets into container images: an agent's reach is whatever its process can read, whatever the system prompt says.
5. Self-check: Read(~/.ssh/**) is denied, but Bash runs unsandboxed without prompts. Can the agent still print your key, and which layer would stop it?

Steps

1. In the throwaway environment, plant fake secrets: a key pair made with ssh-keygen under ~/.ssh that you never use anywhere, a .env containing FAKE_API_KEY=canary-123 in a folder outside the project, and a dummy file under ~/.aws/.
2. Start the agent in an empty project folder and ask it to list, with sizes, every file it can read outside the project, grouped by top-level directory. Save the transcript. Do not ask it to upload anything.
3. Check whether it found the fake SSH key, the .env and the cloud credentials file, and whether it asked for permission before reading outside the project.
4. Add deny rules to the project's .claude/settings.json, for example "permissions": {"deny": ["Read(~/.ssh/**)", "Read(~/.aws/**)", "Read(//**/.env)"]}, open /permissions to confirm they are active, and repeat the same request.
5. Ask the agent to print the fake key with a shell command. If that works, the deny rules did not cover Bash: turn on Claude Code's sandboxing or add Bash rules as described in its docs, retest, and record which layer blocked which access.

Small angles to try

• Repeat the listing with a second coding agent and compare how much each can reach by default.
• Measure the total bytes readable outside the project before and after your rules, as one number to track over time.
• Add a PreToolUse hook that logs every path the agent reads, and review the log after a normal work session.

Key ideas

Static context

The system prompt and tool definitions sent with every request, so their token cost repeats on every turn of an agent loop.

Prompt cache prefix

Providers cache the beginning of a prompt in a fixed order (for Anthropic: tools, then system, then messages), so a change early in that order invalidates everything after it.

Selective tool loading

Send definitions only for the tools a task is likely to need and load the rest on demand, which shrinks the static context.

Steps

1. Read Cursor's blog post 'Improved token efficiency for longer agent runs' (cursor.com/blog/improved-token-efficiency, Sept 23, 2026). Note the numbers behind each lever: about 66% of the system prompt trimmed, 60% fewer static tool-description tokens, 20% fewer cold cache misses, and 1.6% saved by numbering every tenth line in file reads. Note too that they checked quality with A/B tests on production traffic.
2. Read Anthropic's prompt caching docs, specifically the table of what invalidates which cache level and the usage fields cache_creation_input_tokens, cache_read_input_tokens and input_tokens.
3. Connect it to HTTP caching: a stable prefix is cacheable, and a timestamp or request ID near the top works like a cache-busting query string.
4. Self-check: you put the current time at the start of your system prompt. Which parts of the prompt can still be served from cache on the next turn?

Steps

1. Put your agent's system prompt and tool definitions into a script, or write a realistic set of about 10 tools. Call client.messages.count_tokens(model=..., system=..., tools=..., messages=[...]) with one short user message and read input_tokens. Count again without tools to split tool cost from system-prompt cost.
2. Selective loading: pick one task, keep only the 2 or 3 tools it needs, count again, and multiply the per-turn saving by a typical number of turns.
3. Caching baseline: send 5 consecutive requests whose system prompt starts with a timestamp, with cache_control={"type": "ephemeral"} at the top level of the request. Log cache_read_input_tokens, cache_creation_input_tokens and input_tokens from each response's usage.
4. Move the timestamp into the last user message so the stable content comes first, send 5 more requests, and compare the cached share: cache_read / (cache_read + cache_creation + input). The stable prefix must be longer than the model's minimum cacheable length (512 to 4,096 tokens depending on the model), or both cache fields stay at 0.
5. If you use Claude Code, run /context in a session to see how much of the window goes to tools, MCP servers and memory files. Disable one MCP server you do not use and run it again.

Small angles to try

• Try Cursor's file-read trick: number only every tenth line of a large file and count the tokens saved.
• Compare automatic caching with explicit cache_control breakpoints on the same 10-turn conversation.
• Plot cumulative input tokens over a 20-turn session with all tools loaded versus selective loading.

Key ideas

Path traversal to local file inclusion

A user-controlled value containing sequences like ../ makes PHP include a file outside the intended folder, and any readable .php file it reaches is executed.

register_argc_argv

A PHP setting that, when on, fills $_SERVER['argv'] from the query string on web requests; PHP's built-in default is on when no php.ini sets it, and it is deprecated as of PHP 8.5.

pearcmd.php

PEAR's command-line package tool; if it can be included and argv comes from the URL, it becomes the bridge from file inclusion to running code, which is why the mitigations target it.

Steps

1. Read the GitHub advisory GHSA-7hp8-65ch-5whp: affected versions 4.7.0 to 7.1.1, fixed in 7.1.2 with backports down to 4.7.37, weakness CWE-98, and the wording 'conditional RCE' (both server and active-theme conditions must hold).
2. Read SecurityWeek for Patchstack's three attack stages (confirm the bug, look for pearcmd.php, attempt RCE) and the detail that attacker payloads matched the exact encoding the patch fixes, a sign they worked from the patch diff.
3. Read the PHP manual entry for register_argc_argv (php.net/manual/en/ini.core.php) and the CLI differences page. The CLI always forces the setting on, so running php -i in a shell tells you nothing about web requests.
4. Connect it to defense in depth: the WordPress bug opens the door, and the server's PHP configuration decides whether that becomes code execution. Patch first, harden the config second.
5. Self-check: your WordPress container loads no php.ini. Is register_argc_argv on or off for web requests, and where would you add a file to turn it off?

Steps

1. Start a local stack with the official image wordpress:7.1.1-php8.4-apache plus an official MySQL or MariaDB container, publishing WordPress only on 127.0.0.1:8080. Finish the install wizard at http://127.0.0.1:8080.
2. Check the web setting, not the CLI one: use docker exec to create a one-line file info.php containing <?php phpinfo(); in the container's web root (/var/www/html), open http://127.0.0.1:8080/info.php, and note the values of register_argc_argv and 'Loaded Configuration File'. Delete the file when done.
3. Check whether PEAR is present with docker exec <container> find / -name pearcmd.php 2>/dev/null, and write down the path or 'absent'.
4. Harden and re-check: add an .ini file under $PHP_INI_DIR/conf.d/ in the container containing register_argc_argv = Off (the official PHP image docs say to put settings in that folder), restart the container, and reload the phpinfo page to confirm the change.
5. Recreate the stack with wordpress:7.1.2-php8.4-apache on a fresh volume, and confirm the new version in the admin dashboard.
6. Read the fix: in a clone of github.com/WordPress/wordpress-develop, diff the 7.1.1 and 7.1.2 releases, find the change around get_page_template(), and describe in one sentence which input it now rejects.

Small angles to try

• Repeat steps 2 and 3 on the php8.2 and php8.5 image variants and note whether the defaults differ, since 8.5 deprecates the setting.
• Compare the -apache and -fpm image variants for whether pearcmd.php is present.
• Write a short script that runs these two checks against every PHP container on your own machine and prints pass or fail.

Key ideas

Multi-speaker (conversational) TTS

One request renders a whole dialogue, with each named speaker mapped to a prebuilt voice; the Gemini API supports up to two speakers this way.

Inline vocal tags

Markers in the transcript, such as <sigh> or <laugh>, that tell the model to insert a non-speech sound or pause at that point.

Audio-token pricing

Output audio is billed as tokens at 25 tokens per second of audio, so cost follows how long the audio is, not how long the text is.

Steps

1. Read Google's announcement (blog.google). Note the split: Flash TTS for expressive character work, Flash-Lite TTS for high-volume use, 2,000+ voices, 100+ languages, and voice replication from a 30-second sample with consent checks and SynthID watermarking. Voice replication through AI Studio is not available in Illinois, Texas, the EEA, the UK, Switzerland or India.
2. Read the Gemini API speech-generation docs (ai.google.dev/gemini-api/docs/speech-generation). Look for the model IDs gemini-3.8-flash-tts and gemini-3.8-flash-lite-tts, the conversational speech_config that maps speakers to voices, and the output format: 24 kHz mono 16-bit PCM WAV.
3. Read Simon Willison's playground post for a real-world data point: 1 min 18 s of two-speaker audio generated in about 20 seconds for 2.74 cents on Flash.
4. Check the Gemini API pricing page. Through Dec 31, 2026, audio output costs $9 per 1M tokens on Flash and $6 on Flash-Lite, and both prices double from Jan 1, 2027. Text input costs $0.50 per 1M tokens on both.
5. Self-check: at 25 tokens per second, roughly what would a 10-minute two-speaker dialogue cost on Flash-Lite at the 2026 price? (Answer: about 15,000 audio tokens, so about 9 cents plus a small input charge.)

Steps

1. Create an API key in Google AI Studio, export it as GEMINI_API_KEY, and run pip install -U google-genai.
2. Write a two-speaker script of about 150 words, which is roughly 60 seconds of speech. Put two or three inline tags such as <laugh> and <sigh> in the lines where they make sense.
3. Adapt the multi-speaker Python sample from the speech-generation docs. It calls client.interactions.create(model="gemini-3.8-flash-tts", ...) with speech_config mode "conversational", maps two speakers to prebuilt voices (the docs use Puck and Kore), and writes the base64-decoded interaction.output_audio.data to a WAV file. Time the call with time.perf_counter(), then run it again with model="gemini-3.8-flash-lite-tts", keeping the script and voices the same.
4. Measure each file's length with Python's wave module (frames divided by frame rate). Estimate cost as seconds x 25 audio tokens x the per-token price: 60 seconds is 1,500 tokens, so about 1.35 cents on Flash and 0.9 cents on Flash-Lite at 2026 prices, plus a tiny input charge. Treat this as an estimate, since Simon's reported 2.74 cents for 78 seconds is higher than this formula gives.
5. Set up the blind test: copy the two files to A.wav and B.wav in random order (flip a coin) and write down which is which without showing your friend. Ask them which one sounds more natural and whether the laugh and sigh sounded right, then reveal.

Small angles to try

• Add a style note to each line (the docs use a style field such as 'cheerful and friendly') and check whether Flash-Lite follows it as well as Flash does.
• Render the same script in a second language you speak and compare how natural each model sounds.
• Try four different voice pairs on Flash-Lite only and see whether voice choice matters more than model choice in the blind test.

Key ideas

Deskilling

Losing a skill because a tool now does the work you used to practice on.

Tacit knowledge

Judgment that experts use but can't fully write down as rules, such as sensing early that a design will be hard to change later.

Dreyfus model of skill acquisition

A five-stage model (novice, advanced beginner, competent, proficient, expert). The essay uses it to argue that most developers are still advanced beginners and only move up by living with the consequences of their own decisions.

Steps

1. Read the essay on alexn.org. Find the claim that code maintainability shows its effects only months or years later, so there is no quick reward signal to learn from, either for a model or for a developer who no longer writes the code.
2. Separate the essay's two parts: its defense of AI as a tool you still steer, and its prediction that companies will advertise 'NO-AI' policies as an advantage.
3. Skim the HN discussion. Look for the manufacturing-offshoring analogy, the counterargument that an ecosystem can keep know-how even when single companies lose it, and the suggestion to use AI as a critic of your plan rather than as a replacement for writing it.
4. Connect it to your own experience: recall a design mistake of yours that only hurt months later, and ask what you would have learned if a tool had made that decision for you.
5. Self-check: can you state in one sentence why the author thinks this judgment can't be built by reviewing AI output alone, and give one reason a commenter offered for disagreeing?

Steps

1. Pick the two modules and make them comparable: similar size, similar age, and both code you've shipped.
2. For each module, list 5 to 8 major design choices, such as data structures, error handling, module boundaries, dependencies, concurrency and the shape of the public API. Skim the code and its history; git log --follow -- <file> shows one file's history across renames.
3. Set a 20-minute timer for the AI-assisted module. For each choice, write one or two sentences on why it was made and which obvious alternative it beat. Don't reopen the chat log or ask an assistant.
4. Score each choice: 2 means you can defend it with a concrete reason and a rejected alternative, 1 means your answer is plausible but vague, and 0 means you don't know. Then do the same for the hand-written module.
5. Compare the two totals. For every 0, write down the question you'd have to answer to own that decision, and answer one of them today by reading the code or docs yourself.

Small angles to try

• Ask a teammate to pick the design choices for you so you can't pick only the easy ones.
• For each choice, also answer: what breaks first if traffic or data grows 10x?
• Repeat the audit in a month to see whether maintaining the AI-written module closes the gap.

Key ideas

Close to the technology

Working on the models themselves or the systems that make them work, such as training, evaluation or inference infrastructure.

Close to the customer

Knowing a specific user's problem well enough to use AI to give them exactly what they want.

The middle

Generic work of turning a clear spec into code. Graham doesn't name it; reading his answer as saying the middle offers the least leverage now is an interpretation.

Steps

1. Read Graham's reply and the undergrad's question on X (the item's link). Note that 'or both' is part of his answer.
2. Read Graham's essay 'Do Things That Don't Scale' (paulgraham.com, July 2013) to see what being close to the customer meant before AI: recruiting users by hand and working to delight each one.
3. Map your last week of work: which tasks needed knowledge of model or system internals, which needed knowledge of a specific user, and which needed neither.
4. Self-check: name one task in your job that an AI coding tool could do today without knowing either the system internals or your users. How much of your week goes to tasks like that?

Steps

1. Pull up the last two weeks from your tracker, calendar or commits and list 10 concrete tasks you did.
2. Tag each task T if it needed deep knowledge of the model, system or stack internals, C if it needed direct knowledge of a specific user or customer, and M if it needed neither.
3. Line 1: write which extreme your job is closer to, using your T/C/M counts as evidence.
4. Line 2: write one concrete thing you'll do in the next month to move toward the other extreme, for example sitting in on three support calls, or reading the inference code path your feature depends on.
5. Line 3: write which M task you'll hand off, automate or drop to make time for it, and put a check-in date on your calendar.

Small angles to try

• Ask a colleague to tag your 10 tasks without seeing your tags, then compare where you disagree.
• Do the same T/C/M breakdown for the job you want in two years and compare the mix.
• Count how many of your M tasks you already hand to an AI tool today.

Key ideas

Multi-frame super-resolution

Combining many slightly different frames into one image that has more detail and less noise than any single frame.

Scene-specific AI enhancement

A model that recognizes a subject, here the moon, and applies a detail-enhancement network trained on that subject, which can add texture the sensor never captured.

RAW or pro capture

Saving sensor data with less of the phone's processing, which is closer to what the lens recorded; some phones still merge frames into their RAW files.

Steps

1. Read Space.com's Harvest Moon 2026 guide for timing. The moon is full on Sept 26 at 12:49 p.m. EDT (16:49 GMT), and about an hour after moonrise Saturn sits less than 10 degrees to the lower left of the moon.
2. Read MobileSyrup's write-up of the 2023 Reddit test by u/ibreakphotos. The method was to shrink a moon photo to 170x170 pixels, blur it until craters disappeared, show it full-screen in a dark room, and photograph it from across the room.
3. Read Samsung's own explanation (Samsung Mobile Press, March 15, 2023; GSMArena covered it). At 25x zoom or higher the camera merges more than 10 frames, and when Scene Optimizer recognizes the moon it adds a deep-learning detail-enhancement step. Samsung says turning Scene Optimiser off in the camera settings gives unprocessed photos.
4. Connect it to image upscalers you know: a super-resolution model fills in plausible detail learned from its training data. It can't recover information that was never in the input.
5. Self-check: if the image on the monitor has no craters and your photo has them, where did they come from, and could multi-frame stacking alone explain it?

Steps

1. Make the test image: resize the moon photo to 170x170 pixels, then blur it until the craters are gone. In Pillow, use Image.resize and then im.filter(ImageFilter.GaussianBlur(radius=4)); try a radius of 3 to 5. Save this blurred file as your ground truth.
2. Show the blurred image full-screen on the monitor, turn off the room lights, and stand at the far end of the room.
3. Shot A: use the default camera app. Zoom in on the moon (Samsung says its moon processing applies at 25x or more), wait for focus, and take the photo.
4. Shot B: from the same spot and zoom, reduce the processing. Use Pro or RAW mode, or turn off Scene Optimiser in the camera settings if your phone has it.
5. Copy both photos and the blurred source to your computer. Crop each to the moon, scale them to the same size and place them side by side. Circle any crater or bright-ray detail that appears in a photo but not in the source, and write down the phone model, zoom and settings for each shot.

Small angles to try

• Paint a gray blob onto the blurred moon and check whether the phone keeps it or smooths it into 'normal' moon texture.
• Run the same test image on two phones and compare how much detail each one adds.
• On Sept 26, shoot the real moon in both modes and compare each shot with a NASA reference photo of the full moon.

Key ideas

EXIF and XMP metadata

Fields stored inside an image file, such as camera model, capture time, GPS location and editing software.

C2PA Content Credentials

A cryptographically signed manifest attached to a file that records who made it and how it was edited; changing the file afterward breaks the signature.

Metadata stripping

Many apps and platforms remove metadata when a file is shared, so a photo without credentials proves nothing either way.

Steps

1. Read the Ancestor Photo Prank entry in NewEngen's September 2026 TikTok roundup. The prank uses vintage or sepia filters, or old black-and-white celebrity photos, not AI, and relatives still believe it.
2. Read the Content Authenticity Initiative's tutorial 'Signing your first asset' (learn.contentauthenticity.org) to see what a manifest contains and how c2patool reads and signs one. Then open the Verify tool (verify.contentauthenticity.org), which checks a file for credentials.
3. Skim the exiftool documentation (exiftool.org) for -a, -u, -g1 and -diff. Together they show every tag in a file and compare two files.
4. Connect it to something familiar: a signed manifest works like a signed git commit. It proves who signed the file and that it hasn't changed since. It doesn't prove the content is true.
5. Self-check: a relative forwards you a sepia photo with no metadata and no Content Credentials. What can you conclude about whether it's authentic, and why?

Steps

1. Install exiftool from exiftool.org. Optionally install c2patool with brew install c2patool on macOS, or download a prebuilt binary from the c2pa-rs releases page on GitHub.
2. Record a baseline for your photo: exiftool -a -u -g1 original.jpg > original.txt, and c2patool original.jpg to see whether it already carries a manifest.
3. Apply a vintage filter in each of the two apps and export two new files. Run the same exiftool command on each, then exiftool original.jpg -diff app1.jpg (and again for app2) to see which fields were dropped, added or rewritten.
4. Check the original and both filtered files in the Content Credentials Verify tool and note what it reports for each.
5. Send the app1 file to yourself through a messaging app, save the copy you receive, and repeat the exiftool, -diff and Verify checks. Make a small table of which fields survived each step.

Small angles to try

• Sign a copy of your photo with c2patool and the sample manifest from the CAI tutorial (its pattern is c2patool image.jpg -m sample/test.json -o signed.jpg), then check whether the credentials survive the messaging app.
• If your messaging app can send a photo 'as a file' or document, compare that with a normal photo send.
• Check whether either filter app kept your original capture date and GPS location, which could reveal the 'heirloom' as a recent phone photo.

Key ideas

Face landmarks

478 3D points on the face that the model locates in every frame.

Blendshapes

52 expression scores per frame, such as jawOpen, eyeBlinkLeft and mouthSmileLeft, each saying how strongly one facial movement is happening.

Rig mapping

The avatar app links each score to a parameter on the Live2D or 3D model, such as mouth-open or eye-closed, so the model moves with your face.

Steps

1. Read the Face Landmarker overview in Google's AI Edge docs (the item's link). Note that the bundle holds three models (face detection, a face mesh with 478 landmarks, and a blendshape model with 52 scores) and that blendshape output is off by default.
2. Read the Web (JavaScript) guide linked from that page. Look at how FilesetResolver.forVisionTasks loads the WASM files, how FaceLandmarker.createFromOptions sets runningMode to VIDEO and outputFaceBlendshapes to true, and how the requestAnimationFrame render loop calls detectForVideo.
3. Look at the full list of 52 blendshape names (kBlendshapeNames in MediaPipe's face_blendshapes_graph.cc on GitHub; index 0 is _neutral). Pick the few a simple face needs: jawOpen for the mouth and eyeBlinkLeft and eyeBlinkRight for the eyes.
4. Connect it to the VTuber pipeline: webcam, then landmark model, then scores, then rig parameters, then the rendered avatar. Only the first step is machine learning; the rest is mapping numbers onto a drawing.
5. Self-check: if jawOpen reads about 0.3 while your mouth is closed, what would you change in your mapping so the avatar's mouth stays shut?

Steps

1. Create index.html with a <video> element and an inline SVG face (two ellipses for eyes and one for the mouth), and serve the folder from localhost, for example with Python's built-in http.server module.
2. In a module script, load the library (npm install @mediapipe/tasks-vision, or the jsDelivr vision_bundle.mjs the guide shows). Call FilesetResolver.forVisionTasks with the wasm URL from the guide, then FaceLandmarker.createFromOptions with baseOptions.modelAssetPath set to https://storage.googleapis.com/mediapipe-models/face_landmarker/face_landmarker/float16/latest/face_landmarker.task, runningMode: "VIDEO", outputFaceBlendshapes: true and numFaces: 1.
3. Start the webcam with navigator.mediaDevices.getUserMedia into the video element. Then run the guide's requestAnimationFrame loop, which calls detectForVideo only when the video has a new frame.
4. In each result, read faceBlendshapes[0].categories and find the entries whose categoryName is jawOpen, eyeBlinkLeft and eyeBlinkRight. Map each score to the SVG: mouth height = base + score x max opening, and eye height = full height x (1 - score).
5. Measure frame rate: count detections per second using performance.now() and show the number on the page. Note your laptop, your browser and whether the number drops in dim light.
6. Log the raw scores for 10 seconds of a neutral face and 10 seconds of exaggerated expressions. Use the neutral range to set a dead zone so the avatar doesn't twitch.

Small angles to try

• Add mouthSmileLeft and mouthSmileRight to curve the mouth, or browInnerUp to raise the eyebrows.
• Smooth the scores with a short moving average and compare the reduced jitter against the added lag.
• Compare the frame rate on your laptop and on a phone browser, or at two webcam resolutions.

Key ideas

Event data

A log of every on-ball action in a match (passes, shots, tackles), each with a time, a player and a location on the pitch.

Expected goals (xG)

The probability that a shot becomes a goal given its situation; a team's match xG is the sum of that probability over all its shots.

Logistic regression

A simple classifier that turns a weighted sum of features, such as distance and angle, into a probability between 0 and 1.

Steps

1. Read the StatsBomb open-data README on GitHub. Note the folder layout (competitions.json, matches, events, lineups, three-sixty) and the terms: if you publish analysis based on the data, you must credit StatsBomb and use its logo.
2. Open the event specification in the repo's doc folder and find the Shot event, including its location, outcome and StatsBomb xG fields. The pitch uses a 120 by 80 coordinate grid; check the spec's diagram to see where the goal sits.
3. Skim the statsbombpy page on PyPI. sb.competitions(), sb.matches(competition_id=..., season_id=...) and sb.events(match_id=...) return pandas DataFrames, and open data needs no login.
4. Connect it to something familiar: xG is a binary classifier, like a spam filter, with features in and a probability out. You can judge it with the same tools, such as log loss and calibration.
5. Self-check: how can a team lose a match while having the higher xG total, and does that mean the model was wrong?

Steps

1. Run pip install statsbombpy scikit-learn, then from statsbombpy import sb. List the 2022 men's World Cup matches with sb.matches(competition_id=43, season_id=106). Japan's games are included, for example Germany 1-2 Japan (match_id 3857284); pick one as your test match.
2. Build the training set from the rest of the tournament. Loop over the other match IDs, call sb.events(match_id=...), keep rows where type == 'Shot', and label a shot 1 if shot_outcome == 'Goal' and 0 otherwise. Drop penalties (shot_type == 'Penalty'), which are a separate situation.
3. From each shot's location [x, y], compute the distance to the goal center and the angle between the lines to the two posts. Take the goal's coordinates from StatsBomb's event spec.
4. Fit LogisticRegression (from sklearn.linear_model) on [distance, angle], then use predict_proba(X)[:, 1] as your xG for each shot in the held-out Japan match.
5. Sum your xG per team for that match and compare it with the sum of shot_statsbomb_xg per team. Then list the shots where your value and StatsBomb's differ most, such as headers, one-on-ones and shots under pressure.

Small angles to try

• Add body part (shot_body_part, head versus foot) as a feature and see how much closer you get to StatsBomb's numbers.
• Train on a different competition from sb.competitions() and test on the 2022 World Cup to see whether the model transfers.
• Score every shot in the tournament, group them into probability bins, and check whether your model is calibrated: do shots rated 0.2 go in about 20% of the time?