The first release works without an AI provider. Later releases add capabilities only after there is something concrete to test. Keep a short change log and a few sample questions for every stage.
Make the smallest working service
Learn Python functions, JSON, HTTP requests, and status codes by running a read-only study-notes endpoint.
- Download and run the downloadable Python starter.
- Ask it a question, inspect its JSON answer and source, then try an empty question and a missing route.
- Explain what the server returns and what a client does with it.
Deliverable: a working local API and three saved example requests.
Ask a model through an API
Keep the same request and response shape. Add a server-side model call for a learning question; compare it with the deterministic answer from the previous stage.
- Try three prompts, including an ambiguous question and one outside your notes.
- Log the prompt version and inspect inaccurate or unsupported claims.
- Keep API keys on the server, never in browser code or a public repository.
Deliverable: one model-backed endpoint plus three documented results and a failure case.
Answer from your own notes
Split 5–10 documents into small passages, index them, retrieve useful passages, and make the model cite what it used.
- Start with keyword search as a baseline, then compare embedding search on paraphrases.
- Save the passage text and its source URL with each vector.
- Require “I don't know” when retrieved passages cannot support an answer.
Deliverable: cited answers, an index of your notes, and a side-by-side retrieval comparison.
Measure answer quality
Write tests before adding autonomy. Include straightforward, paraphrased, conflicting, and unanswerable questions.
- Record whether the right passages appeared, whether each answer is supported, and whether abstention worked.
- Change only one prompt or retrieval setting at a time and compare with the baseline.
- Keep a small regression set for every later release.
Deliverable: ten test questions, a pass/fail rubric, and a short error report.
Add one controlled tool action
Let the assistant choose a read-only note lookup and use the result in its answer. Add workflow state or MCP only when you can explain the need.
- Give the tool a narrow schema and a clear stop condition.
- Trace one successful call and test a missing result, a bad tool choice, and a repeated call.
- Require approval before any future write action.
Deliverable: an inspectable tool workflow with three failure tests.
Serve, deploy & observe
Expose a stable API, package it for the hosting environment, and watch real failures. FastAPI and Docker are useful here; choose deployment that supports your runtime.
- Add input limits, useful errors, health checks, and request logging without storing private questions unnecessarily.
- Track latency, cost, retrieval misses, and unsupported answers.
- Run the stage-four evaluation set before every release, then verify a live request.
Deliverable: a deployed demo, a deployment guide, and a small quality dashboard or log review.