How I work with AI
- I build AI systems that cite their sources and keep sensitive data on self-hosted models: a contract-review agent, an AI pipeline for a weekly publication, a question-answering assistant for a print report.
- I build them with Claude Code on a written loop: spec, plan, implement test by test, then review from more than one angle.
- Certified: MOEA iPAS AI Application Planner, 2026, both papers passed (70 / 76). See certifications
The loop
How I build, written down. The specs and plans are committed next to the code.
-
Spec
Write down what it must do, and what it must not.
-
Plan
Break it into tasks that can each be tested on their own.
-
Implement
Test by test, with Claude Code. Every task ends tested and committed.
-
Review
From more than one angle: design, executability, security, and a second model where it matters.
next piece of work
What AI is not allowed to do in my systems
Five rules. Each one is built into a system you can read about.
-
The generator never verifies its own output.
A second model answers every generated exam question blind. Mismatches and low-confidence answers are thrown away.
Built into iPAS Adaptive Mock Exam
-
The vision model never re-types text. It only decides order and structure.
The words come from the InDesign file itself. The vision model only puts the text blocks of a page in reading order.
Built into Sixty Impact Report
-
Code decides where images go, not an LLM.
Image placement comes from each image’s heading path in the source document. An LLM only tidies text inside one record.
Built into Tzu Chi Annual Report
-
The agent drafts; the person makes the commit.
After each turn the agent proposes a commit. Nothing reaches git until a person confirms, edits or discards it.
Built into WebForge
-
Judgement in the agent, evidence in code.
Tools pass the evidence and store the results. The report format and the routing stay deterministic.
Built into legit
Areas of practice
Seven areas, each rated with the same level rules as my skills page. Open one for its techniques, principles and evidence.
-
Retrieval
Proven (level 4 of 5)I build retrieval that has to cite its sources: law clauses for a contract agent, page numbers for a print report, records for an archive. I also decide when retrieval is the wrong tool.
Matching skill Retrieval with citations (RAG)
Techniques, principles and evidence
Techniques
- Hybrid recall: pgvector and pg_trgm run in parallel, each route fault-tolerant, then a Qwen3 reranker with a score threshold
- Clause alignment: split a contract into clauses and match each one to law articles, quoting law text verbatim from the source
- Multimodal ingest: a vision model orders the text blocks of an InDesign page, the words come from the IDML itself
- Answers cite (page N); strong hits expand to the whole page; broad questions use page summaries made at ingest
- Structured Markdown files with wiki-links instead of RAG, with a RAG hook kept for later
- Embedding near-duplicate filtering for generated questions
Principles
-
RAG is for finding fragments in a sea of unstructured text. Structured, bounded data that the model can read directly does not need it; add a RAG layer only when the text outgrows the context window.
A novel written with structured files -
The vision model never re-types text. It only decides order and structure.
Sixty Impact Report -
When the report quotes a law, read the article back verbatim from the source.
legit
Evidence
- legit Vector and trigram recall over a 20,491-article law database, reranked, clause-by-clause alignment; hit@1 0.821 on 56 clauses
- Sixty Impact Report Local multimodal RAG over an InDesign report; the answer model is instructed to cite every claim by page
- Digital Archive 1.5 Text, semantic and image search over an archive, plus an assistant that checks its own citations
- A novel written with structured files Written spec arguing why a long-form fiction system uses structured files instead of RAG
-
Agents & tools
Proven (level 4 of 5)I have built agents on the Claude Agent SDK and a native MCP server for my own backend. My rule is that the model judges and code holds the evidence and the side effects.
Matching skill AI agents & tool use
Techniques, principles and evidence
Techniques
- Claude Agent SDK sessions locked to one working directory, resumed by session id
- Native MCP endpoint per tenant, opening with an intent-to-tool map
- Agent SDK + Playwright MCP + project Skills for browser automation
- Skill self-repair: on a broken selector, try fallbacks, snapshot the DOM, promote a new candidate, persist it to learned.json
- Named SKILL.md files as system prompts for each pipeline step
- Guardrails: per-turn tool budgets, submit guards, a confirm step before git
Principles
-
Judgement in the agent, evidence in code. Tools pass evidence and store results; report format and routing stay deterministic.
legit -
The agent drafts, the person makes the commit.
WebForge -
Build the MCP surface for the model: every error carries a suggested fix, destructive tools accept dry_run, duplicate tools get cut.
drust
Evidence
- drust Native MCP per tenant with intent-to-tool instructions; duplicate tools cut in v1.37 so a model picks the right one first
- WebForge Chat-to-commit website editor on the Claude Agent SDK, built in 3 days
- Invoice Agent Agent SDK + Playwright MCP collects vendor invoices; each vendor skill repairs its own selectors
- legit Contract-review agent that shows each tool call and ends with a structured assessment
- Tzu Chi Weekly AI pipeline Claude Agent SDK parses and rewrites each weekly issue, with tools disabled and two timeout budgets
-
Evaluation
In use (level 3 of 5)I measure AI output against fixed sets before I trust it, and I keep the numbers that look bad. The sets are still small.
Matching skill LLM evaluation
Techniques, principles and evidence
Techniques
- Golden set from real cases, 3 runs per case, run-to-run agreement and failure counts
- Escape rate measured per run, not only by majority vote
- hit@1 floors that every retrieval change must re-pass
- ASR round-trip with length-aware CER as a quality gate for generated speech
- Two-model answer verification gate for generated exam questions
- Small eval harness that drives Claude through fixed MCP tasks
Principles
-
Every retrieval change re-runs its eval; hit@1 may not fall below the recorded floor.
legit -
A majority-vote table hides a single run that escapes, so escapes are counted per run.
legit -
The generator never verifies its own output.
iPAS Adaptive Mock Exam
Evidence
- legit Golden set of 10 real cases (9 scored, 27 runs): run-to-run agreement 56% to 81%, failed runs 10 to 2 of 27; case-type accuracy still 44%
- iPAS Adaptive Mock Exam 1,794 generated variants passed a blind second-model check; mismatches and low confidence are discarded
- Voice Collection QA Loop Every generated sentence is transcribed back and scored by CER; failures go to a re-record queue
- drust MCP onboarding eval harness: drives Claude through 5 fixed tasks and scores the tool-call traces; the live before/after run is still pending
-
Local models
In use (level 3 of 5)The AI I run on sensitive text, contracts and archive data, uses self-hosted open models, so that data never leaves the organisation; the weekly-issue pipeline, which handles published content, uses Claude. I tune models for latency and keep keys on the server.
Matching skill Self-hosted models
Techniques, principles and evidence
Techniques
- Qwen3 embedding (2,560 dimensions), Qwen3 reranker and Qwen3 vision behind an OpenAI-compatible gateway
- Reasoning models with thinking switched off where latency matters
- gpt-oss-120b and Qwen3.6 as separate generator and verifier
- Gemma 4 GGUF on llama.cpp with Open WebUI, tools exposed over MCP
- Qwen3.6 on Apple Silicon with MLX
Principles
-
The API key stays server-side and never reaches the browser.
Live Subtitles -
No masking: models run on-premise, so the full contract can be read and logs carry no personal data.
legit -
Convert to the model's script only at the model boundary, character by character; word-level conversion changes what the speech model says.
Voice Collection QA Loop
Evidence
- Sixty Impact Report Chat, vision, embedding and rerank all on self-hosted Qwen3; no external AI API
- legit All AI calls go through the foundation's own model gateway
- Live Subtitles Translation dropped from about 24 s per sentence by disabling thinking, with Gemma as an alternative
- Experiment Gemma 4 on llama.cpp driving Gmail, Calendar and Drive through MCP; Qwen3.6 chat agent on MLX (experiments)
-
Voice & speech
Built (level 2 of 5)I have built speech pipelines end to end. Live captions and voice collection are prototypes; the text-to-speech step of the weekly issue runs inside an internal production pipeline.
Matching skill Speech (ASR & TTS)
Techniques, principles and evidence
Techniques
- Browser audio, VAD, ASR, translation, broadcast to rooms over WebSocket
- Interim and final captions as one upsert per sentence id
- Conservative cross-sentence correction with a pinyin guard
- OpenCC conversion for Traditional Chinese, done in code
- TTS narration with subtitle timing aligned to the voice, assembled with ffmpeg
Principles
-
Correct conservatively: only homophones, guarded by pinyin; a clean sentence comes back unchanged.
Live Subtitles -
Traditional Chinese is enforced deterministically by OpenCC, not left to the model.
Live Subtitles -
Recording QC and generation checks share one CER function.
Voice Collection QA Loop
Evidence
- Live Subtitles Live Chinese-to-English subtitles, working in 2 days; latency design lowered the silence cut from 600 to 500 ms and added 1 s interim captions
- Voice Collection QA Loop Voice collection with a closed CER loop; voice cloning is not switched on yet
- Tzu Chi Weekly AI pipeline Text-to-speech audio for every article of the weekly issue
-
AI-native development
Foundation (level 5 of 5)I build with Claude Code on a written loop: spec, plan, implement test by test, then review from more than one angle. It is how I ship, and I say so.
Matching skill AI-native development
Techniques, principles and evidence
Techniques
- Spec → plan → implement → review, with specs and plans committed in docs/superpowers
- Repo guides (CLAUDE.md) with facts and red lines for any agent that enters
- Subagents per task: Sonnet for mechanical work, Opus for judgement
- Cross-model work: Codex writes the formal migrations; Claude plans each batch and checks every migration against the live database
- Append-only decision logs and self-written audit reports
Principles
-
Every task ends independently testable and committed.
Live Subtitles -
Subagents get an explicit model every time: Sonnet for mechanical work, Opus for judgement.
legit -
Change judgement logic in order: skill file, then code, then tests, then read it back.
legit
Evidence
- drust v1.66 in about 4 months; open source (AGPL-3.0) and in use on 6 Tzu Chi sites and tools
- WebForge Working prototype in 3 days, 5 plans and 3 specs across milestones M1 to M4
- legit Built in about 5 weeks, 73 spec and plan documents, an append-only decision log
- Digital Archive 1.5 Built on a spec → plan → implement → review loop; Codex wrote the formal migrations, Claude planned each batch and checked every migration against the live database
- Tzu Chi Weekly AI pipeline 22 specs and plans in the repo
-
Teaching & adoption
Built (level 2 of 5)I help non-engineers start using AI. I have written guides at work and built a full course; its delivery is not documented here.
Matching skill Teaching AI to non-engineers
Techniques, principles and evidence
Techniques
- Course content as data: one manifest is the single content source for the videos and the instructor handbook
- 8 chapters and 60 scenes for a 2-hour session
- Markdown taught as structure, not formatting
- User guides and best-practice notes for colleagues
Principles
-
Structure is AI's navigation map.
Claude for non-engineers -
Claude drafts; you judge and finalise. Delegate, describe, judge, take responsibility.
Claude for non-engineers
Evidence
- Claude for non-engineers Hands-on Claude course for administrative and planning staff; videos and handbook both draw on one content manifest (the handbook skeleton is script-generated, then completed by hand)
- About User guides and best-practice documents to lower colleagues' learning curve with AI tools (résumé)
Built in days
Elapsed time from first spec to the stated result.
- 3 days WebForge working prototype
- 2 days Live Subtitles working live captions, Chinese to English
- 19 days Identity Platform proof of concept
- about 6 weeks iPAS Adaptive Mock Exam adaptive mock-exam system, then used to pass the exam
- about 4 months drust from start to v1.66