iPAS Adaptive Mock Exam

The problem

In mid-2026 I was preparing for the iPAS AI Application Planner exam, a certification run by Taiwan’s Ministry of Economic Affairs. The only official practice material is a few past papers. Redoing them teaches the answers, not the concepts, and hides where you are weak.

What I built

An adaptive mock-exam system, built solo with Claude Code using a spec → plan → implement → review loop. A Python pipeline turns the past-paper PDFs into a structured question bank. A Next.js app runs full or random sittings, autosaves each answer, grades on the server and shows a dashboard. Behind it, models served from an internal gateway write an explanation for every option and generate verified variants of each official question, so practice keeps changing and can aim at weak spots.

How it works

1. The ingest pipeline parses 6 papers into 300 questions and 1,200 options. A mandatory gate (50 questions × 4 options × 1 answer per paper) must pass before anything is loaded.
2. For each official question, one model generates variants at three difficulty levels: same concept, new scenario or numbers.
3. A different model answers each variant blind. A mismatch, low confidence, more than one correct answer or out-of-scope content discards it. Embeddings drop near-duplicates within the same seed.
4. Every question, official or generated, gets an explanation of why the right option is right and why each other option is wrong.
5. Submitting calls one write RPC that scores against the answer key on the server and records the result once.
6. Weak-spot practice groups wrong answers by seed question and serves unseen variants from those families.

Decisions

• No AI in the first two phases. Parsing and a working exam app came first, so the product was useful before any model was involved.
• The answer key never reaches the browser. Questions are readable only server-side, options and the correct label live in separate columns, and grading happens inside the database.
• Two independent models, no human reviewer. The answer key is the part that must not be wrong, so the generator never verifies its own output: a different model answers each variant blind, and a mismatch discards it.
• Match weak spots by seed family, not tags. Across 2,094 questions there were 3,584 distinct free-text concept tags and 71% appeared only once, so tags became display-only.

Outcome

The bank holds 2,094 questions (300 official, 1,794 generated variants), all with per-option explanations. The app is containerised, and a CI workflow is defined for lint, type checks, unit tests and end-to-end tests. I used it to prepare and passed both papers on 2026-08-15, scoring 70 and 76. One known gap is written down and still open: the data layer’s ownership rules let a user edit their own stored score directly.