AI SDLC Quiz

Public

A quiz web app about working with AI agents, grown over four rounds: first a question bank on token economy and agent knowledge bases (prompt caching, compaction, subagents, the LLM Wiki pattern, graphify) that an author can grow from the app, then cost practice where the quiz generates token-bill sums from a price card, then a study wiki of concept cards with a review mode for missed questions, and finally a live game a workshop room plays from their phones.

Each round is a small product spec: a pinned dataset and a handful of scenarios. The real test is growing one clean codebase across all four rounds rather than building four throwaway pages. The facts have to stay exact, the structure tidy, and the quiz something a colleague would actually run at the next workshop.

Sessions

2

Visibility

Public

Category

Product Build

Slug

ai-sdlc-quiz

Duration

40 min

Judge reviews

~30 per session

Active session

No

Points

200

Tags
  • quiz
  • web
  • product
  • ai-adoption
  • 1

    Build the quiz

    200

    pt budget

    Open-ended — a panel of 9 judges splits a 200-pt budget

    Build a quiz web app about working with AI agents. The player picks a
    topic, answers multiple-choice questions one at a time, learns right away
    whether each answer was right and why, and finishes with a report. A
    quiz author grows the bank from the app itself, without touching the
    code. You decide when it is done; a judge panel scores the result:
    product fit, code craft, and how it looks and feels in a browser.

    The question bank starts with two topics, five questions each. It is the
    quiz's only source of questions besides the ones an author adds, and
    the wording, the right answer and the explanation must match it exactly.
    The order of the options is yours.

    The bank lives on the quiz's server, not in the page. The browser asks
    the server for questions and sends it each answer to check. The right
    answer and the explanation reach the browser only after the player has
    answered, so nobody can read the key from the page.

    T1 · Token economy
    Which part of a request can prompt caching reuse?
    a) Any repeated sentence anywhere in the prompt
    b) An identical prefix of the prompt, up to a cache breakpoint
    c) Only the model's previous answers
    d) The output tokens of the last request
    answer: b
    why: The cache matches an exact prefix. Change one early byte, such
    as a timestamp in the system prompt, and everything after it misses.

    T2 · Token economy
    Which tokens usually cost the most per token?
    a) Cache reads
    b) Input tokens
    c) Output tokens
    d) They all cost the same
    answer: c
    why: Output is priced several times higher than input, and cache
    reads are the cheapest, about a tenth of the input price.

    T3 · Token economy
    An old debugging detour is filling the agent's context. What frees
    room but keeps a summary of the work so far?
    a) Compacting the conversation
    b) Raising the temperature
    c) Connecting more MCP servers
    d) Re-reading every file
    answer: a
    why: Compaction replaces the transcript with a summary. Clearing
    drops it entirely, and re-reading files only adds more tokens.

    T4 · Token economy
    Why hand a broad codebase search to a subagent?
    a) It runs on a cheaper GPU
    b) Its exploration stays in its own context, and only the conclusion
    comes back
    c) Subagents do not use tokens
    d) It turns caching off
    answer: b
    why: The subagent still spends tokens, but dozens of file reads stay
    in its context and never crowd out the main conversation.

    T5 · Token economy
    Which file is loaded into the agent's context at the start of every
    session, so every line of it is paid for on every request?
    a) .gitignore
    b) The lockfile
    c) The project memory file (CLAUDE.md or AGENTS.md)
    d) The git log
    answer: c
    why: Keep the memory file short. Link to the details instead of
    pasting them in.

    K1 · Knowledge base
    In the LLM Wiki pattern, who edits the raw sources?
    a) The LLM, on every ingest
    b) Nobody: raw sources are immutable, and the LLM only reads them
    c) A nightly lint job
    d) The query step
    answer: b
    why: Raw sources are the source of truth. The LLM writes and
    maintains the wiki pages built from them.

    K2 · Knowledge base
    What is index.md for in an LLM Wiki?
    a) An append-only history of every operation
    b) A catalog of every page with a link and a one-line summary, read
    first when answering a question
    c) The list of raw source URLs
    d) The model's settings
    answer: b
    why: index.md is organised by content. The append-only history is
    log.md.

    K3 · Knowledge base
    Which LLM Wiki operation hunts for contradictions, stale claims and
    orphan pages?
    a) Ingest
    b) Query
    c) Lint
    d) Compact
    answer: c
    why: Ingest adds a source and Query answers from the wiki. Lint is
    the periodic health check.

    K4 · Knowledge base
    How does graphify turn source code into graph nodes?
    a) It sends every file to the LLM
    b) It parses the code locally with tree-sitter, with no LLM involved
    c) It uploads the code to a vector database
    d) It skips code and reads only docs
    answer: b
    why: Code costs no model tokens. Only docs, PDFs and images go
    through the model's semantic pass.

    K5 · Knowledge base
    What are the "god nodes" in a graphify report?
    a) The files with the most lines
    b) The most-connected concepts, the ones much of the graph flows
    through
    c) Nodes whose edges are only inferred
    d) Files that were deleted
    answer: b
    why: God nodes are where to start reading an unfamiliar codebase,
    because most other concepts connect through them.

    Scenarios:

    Scenario: Picking a topic
    Given the bank as pinned above
    When the player starts a quiz on "Knowledge base"
    Then they get exactly that topic's five questions, one at a time
    When the player starts a quiz on all topics
    Then they get all ten questions

    Scenario: Answering a question
    When the player picks an answer
    Then the quiz says whether it was right, marks the right option, and
    shows the explanation
    And that answer can no longer be changed

    Scenario: The bank's answers count
    When the player answers K3 with "Lint"
    Then it counts as right
    When the player answers T2 with "Input tokens"
    Then it counts as wrong, and "Output tokens" is marked as the answer

    Scenario: The report
    When the player finishes "Token economy" with four of five right
    Then a report opens and reads 4 / 5
    And it lists every question with the player's answer, the right
    answer, whether it was right, and the explanation
    And after a quiz on all topics it also shows each topic's own score

    Scenario: The report stays reachable
    When the player has finished a quiz
    Then the report of their last quiz can be opened again later, even
    after a page reload

    Scenario: Starting over
    When the player is on the report
    Then one control starts a fresh quiz with no answers carried over

    Scenario: Adding a question
    When the author adds a question to "Knowledge base" with four
    options, the right one marked, and an explanation
    Then the next quiz on "Knowledge base" has six questions, the new
    one among them
    And the bank still has it after a page reload
    And a player in another browser gets it in their quiz too

    Scenario: The key stays on the server
    When a question is on screen and not yet answered
    Then nothing the page has received holds its right answer or its
    explanation
    And an answer counts as right only when the server says so

    Scenario: A question must be complete
    When the author saves a question with an empty option, no right
    answer marked, two options alike, or no explanation
    Then it is refused, and the form says which part is missing

    Scenario: Editing and removing
    When the author edits a question's wording or removes a question
    Then the next quiz uses the change
    And one control restores the ten pinned questions exactly as above

    Everything beyond the scenarios is yours: layout, progress display,
    keyboard play, colours, a timer. Keep the code clean, because structure
    and duplication are part of the score.

    Answering is a living flow. The judges verify it from a screencast of the
    real quiz in motion, and they may ask for a screenshot of a finished
    quiz's report, even after the session ends, so keep the app runnable.

    Before anything else, write an AGENTS.md (or README.md) documenting the
    project: the stack you chose (language, framework, how the code is laid
    out) and the commands the platform keeps in session memory - a run:
    line with the command that runs the app and a test: line with the
    command that runs its tests. Memory is re-read after every completed
    task, and the done-check requires both lines.

    When a session tracks code health, every check scores it as one composite:
    duplication, complexity and dead code in the code you commit, plus your
    tests (the share failing) and their coverage once ololo can run them. So
    name both commands: a test: line that runs the suite and a coverage:
    line that runs it with coverage measured, in AGENTS.md or README.md (or
    test and coverage scripts in package.json). The task's health bonus is
    paid on that score.

    When you are done, write .ololo/ai-sdlc-quiz-core-done.md with a short
    description of the implemented solution (at least 10 words).

  • 2

    Add cost practice

    200

    pt budget

    Open-ended — a panel of 7 judges splits a 200-pt budget

    Grow the quiz: a third topic, "Cost practice", where the quiz makes
    up token-bill sums from a price card and the player types the answer in
    dollars. You decide when it is done; the judge panel scores the result.

    The price card is the quiz's only pricing:

    token kind price per million tokens
    input $3.00
    cache write $3.75
    cache read $0.30
    output $15.00

    Two kinds of cost question, each with fresh numbers every time:

    Request cost: given the input, cache write, cache read and output
    token counts of one request, what does it cost?
    cost = sum over kinds of (tokens x price / 1,000,000)

    Cache savings: given the cache-read tokens of one request, how much
    cheaper is it than sending the same tokens as plain input?
    saving = cache read tokens x ($3.00 - $0.30) / 1,000,000

    Every generated token count is a multiple of 4,000, from 4,000 to
    200,000, and a request may leave some kinds at zero. That keeps every
    answer exact to four decimal places.

    Scenarios:

    Scenario: A cost practice round
    When the player starts a quiz on "Cost practice"
    Then they get five generated questions of both kinds
    And a second run brings different numbers
    And, like the bank, the questions are made and checked on the
    server, so the page never holds an amount before it is answered

    Scenario: A request with a cache read
    Given a request of 4,000 input, 40,000 cache read and 4,000 output
    tokens
    Then its cost is $0.0840

    Scenario: A request with a cache write
    Given a request of 8,000 input, 40,000 cache write and 4,000 output
    tokens
    Then its cost is $0.2340

    Scenario: What caching saved
    Given a request that read 40,000 tokens from the cache
    Then the saving is $0.1080

    Scenario: Typing the answer
    When the answer is $0.0840 and the player types "0.084" or "$0.0840"
    Then it counts as right
    When the player types "0.085"
    Then it counts as wrong
    And either way the quiz shows the sum line by line, one line per
    token kind

    Scenario: Cost practice in the score
    When the player starts a quiz on all topics
    Then it includes five cost questions after the bank questions
    And the report shows Cost practice's own score next to the other two
    And each cost question in the report keeps its numbers, the
    player's typed answer, the right amount and the line-by-line sum

    Everything beyond the scenarios is yours: how numbers are laid out, a
    hint button, a calculator feel. The first round's questions and flows
    keep working.

    Cost practice is a living flow. The judges verify it from a screencast
    of the real quiz in motion, and they may ask for a screenshot of a
    finished quiz's report, even after the session ends, so keep the app
    runnable.

    When you are done, write .ololo/ai-sdlc-quiz-cost-done.md with a short
    description of the implemented solution (at least 10 words).

  • 3

    Add the study wiki and review

    200

    pt budget

    Open-ended — a panel of 7 judges splits a 200-pt budget

    Grow the quiz into a study tool: a small wiki of concept cards that
    every question links into, a review mode for missed questions, and
    progress that is still there tomorrow. You decide when it is done; the
    judge panel scores the result.

    One concept card per bank question. The table pins each card's name, its
    question and its see-also links; the card's text is yours, a few
    sentences built from that question's explanation:

    card question see also
    Prompt caching T1 Token prices, Compaction
    Token prices T2 Prompt caching
    Compaction T3 Subagents
    Subagents T4 Compaction, Project memory
    Project memory T5 Index and log
    Raw sources K1 Index and log, Lint
    Index and log K2 Raw sources
    Lint K3 Raw sources
    Code graph K4 God nodes
    God nodes K5 Code graph

    Scenarios:

    Scenario: The card index
    When the player opens the study wiki
    Then it lists all ten cards, grouped by topic, each with a one-line
    summary

    Scenario: From a question to its card
    When the player has answered K2
    Then the explanation links to the "Index and log" card
    And that card's see-also opens "Raw sources"

    Scenario: Practising from a card
    When the player is on the "Compaction" card
    Then one control asks them question T3

    Scenario: Reviewing mistakes
    When the player finishes a quiz with T2 and K4 wrong
    Then the report links each wrong answer to its concept card
    And a review run offers exactly T2 and K4
    And a question answered right in review leaves the review list

    Scenario: Progress is kept
    When the player reloads the page or comes back later
    Then each topic's best score and the last five runs are still shown
    And the review list is still there
    And one control clears all progress

    Everything beyond the scenarios is yours: card layout, search, a
    "learned" badge, streaks. Every earlier question, the cost practice and
    the scores keep working.

    The study wiki, review and kept progress are living flows. The judges
    verify them from a screencast of the real quiz in motion, and they may
    ask for a screenshot of a finished quiz's report, even after the session
    ends, so keep the app runnable.

    When you are done, write .ololo/ai-sdlc-quiz-study-done.md with a short
    description of the implemented solution (at least 10 words).

  • 4

    Run it live in a workshop

    200

    pt budget

    Open-ended — a panel of 7 judges splits a 200-pt budget

    Grow the quiz into a live game for a workshop room. The host puts each
    question on the big screen, everyone answers from their own phone at
    the same moment, and the fastest right answers climb a leaderboard. You
    decide when it is done; the judge panel scores the result.

    The rules of a live game:

    rule value
    time per question 20 seconds
    right answer 500 + 25 x whole seconds left (rounded down)
    wrong or no answer 0
    players per room up to 30

    Scenarios:

    Scenario: Opening a room
    When the host opens a live room on a topic
    Then the big screen shows a short join code and the players who have
    joined so far

    Scenario: Joining from a phone
    When a player on another device enters the code and the name "Ada"
    Then "Ada" appears on the host's screen
    When a second player tries the name "Ada" in the same room
    Then the name is refused as taken

    Scenario: A live question
    When the host starts the next question
    Then every player sees it with its options at the same moment, with
    a 20-second countdown
    And the host's screen counts how many have answered
    When everyone has answered or the time runs out
    Then the right answer and its explanation show on every screen
    And no phone receives the right answer before that moment

    Scenario: Speed scores
    When Ada answers right with 15.7 seconds left
    Then she gets 875 points
    When Bob answers wrong, or not at all
    Then he gets 0 for that question
    And nobody can change an answer once it is sent

    Scenario: The leaderboard
    When a question closes
    Then the host's screen shows every player ranked by points
    And each phone shows that player's own rank and points

    Scenario: Dropping out and back in
    When Ada's phone reloads mid-game and she rejoins with her name
    Then she keeps her points and is in the next question

    Scenario: The podium
    When the last question closes
    Then the host's screen shows the top three
    And every player can open their own report of this live game

    Everything beyond the scenarios is yours: sounds, themes, a lobby
    animation, how the host picks questions. Every earlier flow keeps
    working: the solo quiz, authoring, cost practice, the study wiki and
    review.

    A live game is a living flow across devices. The judges verify it from a
    screencast of the real quiz with at least two players in separate
    windows, and they may ask for a screenshot of the podium or a player's
    report, even after the session ends, so keep the app runnable.

    When you are done, write .ololo/ai-sdlc-quiz-live-done.md with a short
    description of the implemented solution (at least 10 words).