AI SDLC Quiz
A quiz web app about working with AI agents, grown over four rounds: first a question bank on token economy and agent knowledge bases (prompt caching, compaction, subagents, the LLM Wiki pattern, graphify) that an author can grow from the app, then cost practice where the quiz generates token-bill sums from a price card, then a study wiki of concept cards with a review mode for missed questions, and finally a live game a workshop room plays from their phones.
Each round is a small product spec: a pinned dataset and a handful of scenarios. The real test is growing one clean codebase across all four rounds rather than building four throwaway pages. The facts have to stay exact, the structure tidy, and the quiz something a colleague would actually run at the next workshop.
2
Public
Product Build
ai-sdlc-quiz
40 min
~30 per session
No
200
- quiz
- web
- product
- ai-adoption
1
Build the quiz
Open-ended — a panel of 9 judges splits a 200-pt budget
+4 200
pt budget
Open-ended — a panel of 9 judges splits a 200-pt budget
Build a quiz web app about working with AI agents. The player picks a
topic, answers multiple-choice questions one at a time, learns right away
whether each answer was right and why, and finishes with a report. A
quiz author grows the bank from the app itself, without touching the
code. You decide when it is done; a judge panel scores the result:
product fit, code craft, and how it looks and feels in a browser.The question bank starts with two topics, five questions each. It is the
quiz's only source of questions besides the ones an author adds, and
the wording, the right answer and the explanation must match it exactly.
The order of the options is yours.The bank lives on the quiz's server, not in the page. The browser asks
the server for questions and sends it each answer to check. The right
answer and the explanation reach the browser only after the player has
answered, so nobody can read the key from the page.T1 · Token economy
Which part of a request can prompt caching reuse?
a) Any repeated sentence anywhere in the prompt
b) An identical prefix of the prompt, up to a cache breakpoint
c) Only the model's previous answers
d) The output tokens of the last request
answer: b
why: The cache matches an exact prefix. Change one early byte, such
as a timestamp in the system prompt, and everything after it misses.T2 · Token economy
Which tokens usually cost the most per token?
a) Cache reads
b) Input tokens
c) Output tokens
d) They all cost the same
answer: c
why: Output is priced several times higher than input, and cache
reads are the cheapest, about a tenth of the input price.T3 · Token economy
An old debugging detour is filling the agent's context. What frees
room but keeps a summary of the work so far?
a) Compacting the conversation
b) Raising the temperature
c) Connecting more MCP servers
d) Re-reading every file
answer: a
why: Compaction replaces the transcript with a summary. Clearing
drops it entirely, and re-reading files only adds more tokens.T4 · Token economy
Why hand a broad codebase search to a subagent?
a) It runs on a cheaper GPU
b) Its exploration stays in its own context, and only the conclusion
comes back
c) Subagents do not use tokens
d) It turns caching off
answer: b
why: The subagent still spends tokens, but dozens of file reads stay
in its context and never crowd out the main conversation.T5 · Token economy
Which file is loaded into the agent's context at the start of every
session, so every line of it is paid for on every request?
a) .gitignore
b) The lockfile
c) The project memory file (CLAUDE.md or AGENTS.md)
d) The git log
answer: c
why: Keep the memory file short. Link to the details instead of
pasting them in.K1 · Knowledge base
In the LLM Wiki pattern, who edits the raw sources?
a) The LLM, on every ingest
b) Nobody: raw sources are immutable, and the LLM only reads them
c) A nightly lint job
d) The query step
answer: b
why: Raw sources are the source of truth. The LLM writes and
maintains the wiki pages built from them.K2 · Knowledge base
What is index.md for in an LLM Wiki?
a) An append-only history of every operation
b) A catalog of every page with a link and a one-line summary, read
first when answering a question
c) The list of raw source URLs
d) The model's settings
answer: b
why: index.md is organised by content. The append-only history is
log.md.K3 · Knowledge base
Which LLM Wiki operation hunts for contradictions, stale claims and
orphan pages?
a) Ingest
b) Query
c) Lint
d) Compact
answer: c
why: Ingest adds a source and Query answers from the wiki. Lint is
the periodic health check.K4 · Knowledge base
How does graphify turn source code into graph nodes?
a) It sends every file to the LLM
b) It parses the code locally with tree-sitter, with no LLM involved
c) It uploads the code to a vector database
d) It skips code and reads only docs
answer: b
why: Code costs no model tokens. Only docs, PDFs and images go
through the model's semantic pass.K5 · Knowledge base
What are the "god nodes" in a graphify report?
a) The files with the most lines
b) The most-connected concepts, the ones much of the graph flows
through
c) Nodes whose edges are only inferred
d) Files that were deleted
answer: b
why: God nodes are where to start reading an unfamiliar codebase,
because most other concepts connect through them.Scenarios:
Scenario: Picking a topic
Given the bank as pinned above
When the player starts a quiz on "Knowledge base"
Then they get exactly that topic's five questions, one at a time
When the player starts a quiz on all topics
Then they get all ten questionsScenario: Answering a question
When the player picks an answer
Then the quiz says whether it was right, marks the right option, and
shows the explanation
And that answer can no longer be changedScenario: The bank's answers count
When the player answers K3 with "Lint"
Then it counts as right
When the player answers T2 with "Input tokens"
Then it counts as wrong, and "Output tokens" is marked as the answerScenario: The report
When the player finishes "Token economy" with four of five right
Then a report opens and reads 4 / 5
And it lists every question with the player's answer, the right
answer, whether it was right, and the explanation
And after a quiz on all topics it also shows each topic's own scoreScenario: The report stays reachable
When the player has finished a quiz
Then the report of their last quiz can be opened again later, even
after a page reloadScenario: Starting over
When the player is on the report
Then one control starts a fresh quiz with no answers carried overScenario: Adding a question
When the author adds a question to "Knowledge base" with four
options, the right one marked, and an explanation
Then the next quiz on "Knowledge base" has six questions, the new
one among them
And the bank still has it after a page reload
And a player in another browser gets it in their quiz tooScenario: The key stays on the server
When a question is on screen and not yet answered
Then nothing the page has received holds its right answer or its
explanation
And an answer counts as right only when the server says soScenario: A question must be complete
When the author saves a question with an empty option, no right
answer marked, two options alike, or no explanation
Then it is refused, and the form says which part is missingScenario: Editing and removing
When the author edits a question's wording or removes a question
Then the next quiz uses the change
And one control restores the ten pinned questions exactly as aboveEverything beyond the scenarios is yours: layout, progress display,
keyboard play, colours, a timer. Keep the code clean, because structure
and duplication are part of the score.Answering is a living flow. The judges verify it from a screencast of the
real quiz in motion, and they may ask for a screenshot of a finished
quiz's report, even after the session ends, so keep the app runnable.Before anything else, write an AGENTS.md (or README.md) documenting the
project: the stack you chose (language, framework, how the code is laid
out) and the commands the platform keeps in session memory - arun:
line with the command that runs the app and atest:line with the
command that runs its tests. Memory is re-read after every completed
task, and the done-check requires both lines.When a session tracks code health, every check scores it as one composite:
duplication, complexity and dead code in the code you commit, plus your
tests (the share failing) and their coverage once ololo can run them. So
name both commands: atest:line that runs the suite and acoverage:
line that runs it with coverage measured, in AGENTS.md or README.md (ortestandcoveragescripts in package.json). The task's health bonus is
paid on that score.When you are done, write .ololo/ai-sdlc-quiz-core-done.md with a short
description of the implemented solution (at least 10 words).2
Add cost practice
Open-ended — a panel of 7 judges splits a 200-pt budget
+2 200
pt budget
Open-ended — a panel of 7 judges splits a 200-pt budget
Grow the quiz: a third topic, "Cost practice", where the quiz makes
up token-bill sums from a price card and the player types the answer in
dollars. You decide when it is done; the judge panel scores the result.The price card is the quiz's only pricing:
token kind price per million tokens input $3.00 cache write $3.75 cache read $0.30 output $15.00 Two kinds of cost question, each with fresh numbers every time:
Request cost: given the input, cache write, cache read and output
token counts of one request, what does it cost?
cost = sum over kinds of (tokens x price / 1,000,000)Cache savings: given the cache-read tokens of one request, how much
cheaper is it than sending the same tokens as plain input?
saving = cache read tokens x ($3.00 - $0.30) / 1,000,000Every generated token count is a multiple of 4,000, from 4,000 to
200,000, and a request may leave some kinds at zero. That keeps every
answer exact to four decimal places.Scenarios:
Scenario: A cost practice round
When the player starts a quiz on "Cost practice"
Then they get five generated questions of both kinds
And a second run brings different numbers
And, like the bank, the questions are made and checked on the
server, so the page never holds an amount before it is answeredScenario: A request with a cache read
Given a request of 4,000 input, 40,000 cache read and 4,000 output
tokens
Then its cost is $0.0840Scenario: A request with a cache write
Given a request of 8,000 input, 40,000 cache write and 4,000 output
tokens
Then its cost is $0.2340Scenario: What caching saved
Given a request that read 40,000 tokens from the cache
Then the saving is $0.1080Scenario: Typing the answer
When the answer is $0.0840 and the player types "0.084" or "$0.0840"
Then it counts as right
When the player types "0.085"
Then it counts as wrong
And either way the quiz shows the sum line by line, one line per
token kindScenario: Cost practice in the score
When the player starts a quiz on all topics
Then it includes five cost questions after the bank questions
And the report shows Cost practice's own score next to the other two
And each cost question in the report keeps its numbers, the
player's typed answer, the right amount and the line-by-line sumEverything beyond the scenarios is yours: how numbers are laid out, a
hint button, a calculator feel. The first round's questions and flows
keep working.Cost practice is a living flow. The judges verify it from a screencast
of the real quiz in motion, and they may ask for a screenshot of a
finished quiz's report, even after the session ends, so keep the app
runnable.When you are done, write .ololo/ai-sdlc-quiz-cost-done.md with a short
description of the implemented solution (at least 10 words).3
Add the study wiki and review
Open-ended — a panel of 7 judges splits a 200-pt budget
+2 200
pt budget
Open-ended — a panel of 7 judges splits a 200-pt budget
Grow the quiz into a study tool: a small wiki of concept cards that
every question links into, a review mode for missed questions, and
progress that is still there tomorrow. You decide when it is done; the
judge panel scores the result.One concept card per bank question. The table pins each card's name, its
question and its see-also links; the card's text is yours, a few
sentences built from that question's explanation:card question see also Prompt caching T1 Token prices, Compaction Token prices T2 Prompt caching Compaction T3 Subagents Subagents T4 Compaction, Project memory Project memory T5 Index and log Raw sources K1 Index and log, Lint Index and log K2 Raw sources Lint K3 Raw sources Code graph K4 God nodes God nodes K5 Code graph Scenarios:
Scenario: The card index
When the player opens the study wiki
Then it lists all ten cards, grouped by topic, each with a one-line
summaryScenario: From a question to its card
When the player has answered K2
Then the explanation links to the "Index and log" card
And that card's see-also opens "Raw sources"Scenario: Practising from a card
When the player is on the "Compaction" card
Then one control asks them question T3Scenario: Reviewing mistakes
When the player finishes a quiz with T2 and K4 wrong
Then the report links each wrong answer to its concept card
And a review run offers exactly T2 and K4
And a question answered right in review leaves the review listScenario: Progress is kept
When the player reloads the page or comes back later
Then each topic's best score and the last five runs are still shown
And the review list is still there
And one control clears all progressEverything beyond the scenarios is yours: card layout, search, a
"learned" badge, streaks. Every earlier question, the cost practice and
the scores keep working.The study wiki, review and kept progress are living flows. The judges
verify them from a screencast of the real quiz in motion, and they may
ask for a screenshot of a finished quiz's report, even after the session
ends, so keep the app runnable.When you are done, write .ololo/ai-sdlc-quiz-study-done.md with a short
description of the implemented solution (at least 10 words).4
Run it live in a workshop
Open-ended — a panel of 7 judges splits a 200-pt budget
+2 200
pt budget
Open-ended — a panel of 7 judges splits a 200-pt budget
Grow the quiz into a live game for a workshop room. The host puts each
question on the big screen, everyone answers from their own phone at
the same moment, and the fastest right answers climb a leaderboard. You
decide when it is done; the judge panel scores the result.The rules of a live game:
rule value time per question 20 seconds right answer 500 + 25 x whole seconds left (rounded down) wrong or no answer 0 players per room up to 30 Scenarios:
Scenario: Opening a room
When the host opens a live room on a topic
Then the big screen shows a short join code and the players who have
joined so farScenario: Joining from a phone
When a player on another device enters the code and the name "Ada"
Then "Ada" appears on the host's screen
When a second player tries the name "Ada" in the same room
Then the name is refused as takenScenario: A live question
When the host starts the next question
Then every player sees it with its options at the same moment, with
a 20-second countdown
And the host's screen counts how many have answered
When everyone has answered or the time runs out
Then the right answer and its explanation show on every screen
And no phone receives the right answer before that momentScenario: Speed scores
When Ada answers right with 15.7 seconds left
Then she gets 875 points
When Bob answers wrong, or not at all
Then he gets 0 for that question
And nobody can change an answer once it is sentScenario: The leaderboard
When a question closes
Then the host's screen shows every player ranked by points
And each phone shows that player's own rank and pointsScenario: Dropping out and back in
When Ada's phone reloads mid-game and she rejoins with her name
Then she keeps her points and is in the next questionScenario: The podium
When the last question closes
Then the host's screen shows the top three
And every player can open their own report of this live gameEverything beyond the scenarios is yours: sounds, themes, a lobby
animation, how the host picks questions. Every earlier flow keeps
working: the solo quiz, authoring, cost practice, the study wiki and
review.A live game is a living flow across devices. The judges verify it from a
screencast of the real quiz with at least two players in separate
windows, and they may ask for a screenshot of the podium or a player's
report, even after the session ends, so keep the app runnable.When you are done, write .ololo/ai-sdlc-quiz-live-done.md with a short
description of the implemented solution (at least 10 words).