Cruxly

Cruxly › AI engineering › Portfolio Projects

Build Projects That Get You Hired: The Process, the Shared Test Seam and How to Show It

seven stages from idea to interview, one tiny offline package (llmkit) that every project imports, and the README, pitch and resume bullet

Open the interactive lesson

Project: "I want a portfolio that makes a hiring manager stop scrolling. I do not want to pay for an API key just to run my own tests, and I do not want ten repos that each reinvent the same fake model client. Show me the process, the shared foundation, and how to present the result."

Time-box: about 5 to 6 hours. You will have a running, tested Python package called llmkit (four modules, 15 passing tests, standard library only), a one-page project process you can reuse, a README skeleton with a results table, a 90-second pitch script and a resume-bullet formula. Every later project lesson in this topic starts with pip install -e llmkit and imports from it.

What a portfolio project is for, and what the hiring side actually says

A project proves one thing: that you can take an LLM feature from a vague idea to something measured, shipped and explained. A demo that works once on the happy path proves that you can follow a tutorial. A project with a golden set, a stated cost per request, a failure you found and fixed, and a one-command run proves judgement, which is the scarce part. The ten lessons that follow are built around that difference.

What the sources say, and how much to trust it. In the research pass for this topic (24 October 2026) the author summarised nine hiring-side sources (the list is not reproduced here and I could not independently re-check each one): an interview-prep guide, a training company, a practitioner blog, a staffing firm, a course vendor, an Indian training provider, an aggregator of self-reported interview loops, one job listing, and a practitioner's notes on evals. They agree more than I expected:

:::warn Treat this as consensus, not data Almost every source is a recruiter, a course seller or a blogger. None is a survey of hiring managers, several have a placement or course interest, and the claims that stand out (a live link beating a repo for Indian recruiters, wrappers being discounted) are repeated but unsourced beyond the authors' experience. I quote no salary band and no 'percent of job posts mention RAG' figure, because those sources do not support them. Use the list as a checklist of what to make easy to see, not as proof of what any particular company wants. :::

Two sources are worth singling out because they are a method, not an opinion. Hamel Husain's evals notes say to start with error analysis (read real traces before writing rubrics), to prefer binary pass or fail over arbitrary scores, to use code assertions where you can, and to keep the broken cases in the CI set. The loop in the next section is built on that, and Evaluation Foundations: Golden Sets, Error Analysis and Reading Benchmarks explains the reasoning. This topic does not re-teach RAG, structured outputs, evals or serving; it tells you how to build and present them, and each lesson names the earlier lessons by italic title it stands on.

What this means for scope. One substantial project with real measurement beats four weekend toys. If you can only finish two of the ten, finish them to the standard here: seam, golden set, report, gate, README. The first two project lessons (extraction, then RAG) are chosen because both have an answer you can check mechanically, which makes honest numbers cheap.

The seven stages: idea, scope, build, measure, ship, write-up, talk about it

Run every project as the same seven stages, and treat the metric as the first deliverable, not the last. The time boxes below are for one project of the size in this topic (14 to 30 hours); they are budgets to push against, not promises.

@@ 1 | Idea (1 to 2 hours) Pick a user with a real, painful task: a person you can name, or a public dataset with a real task. Write four lines: user, task, how they do it today, what better looks like. Reject any idea where a regex or a SQL query would do the job; that is the 'AI where a regex would do' red flag. Artifact: docs/problem.md.

@@ 2 | Scope (2 hours) Write the success metric as a number before any code, for example 'answers correct on 100 questions at or above 0.75, p95 under 6 seconds, under half a cent per answer'. List three non-goals. Cut to one vertical slice you can demo in a weekend. Artifact: a 'Scope and metrics' section in docs/design.md.

@@ 3 | Build (60 to 75 percent of the time) Seam first (the next section), then the dumbest end-to-end pipeline that produces an output, then milestones in order. Commit at each milestone with its test or eval green, so the git history reads as a story.

@@ 4 | Measure (15 to 20 percent) Build the golden set early, before you tune anything. Read 30 failures by hand and sort them into classes. Fix the biggest class, rerun, and record before and after with the sample size and an interval. Artifacts: evals/reports/ and docs/failures.md.

@@ 5 | Ship (3 to 5 hours) A Dockerfile, a free-tier deploy, a rate limit and a spend cap (so a public demo cannot cost you money), a /health endpoint, secrets in environment variables, and a check of cold-start behaviour.

@@ 6 | Write-up (3 to 4 hours) The README from the template below, plus a 600 to 900 word post titled around what broke and what you changed. Failures are the interesting part.

@@ 7 | Talk about it (2 hours) Rehearse the 90-second pitch and the five interview questions out loud, and know every number in the README and the command that regenerates it.

Why the number comes first. If you decide 'good enough' after seeing results, you will pick the threshold your system already meets. Writing 'at or above 0.75 on 100 questions' before the first line of code, and committing it, is what makes the later report credible. It also protects you from the most common scoping mistake, which is a project whose success cannot be measured at all.

Why measure sits between build and ship, with a loop back. Measurement is not a final report. The useful work is the loop: the first golden-set run on your dumbest pipeline gives the 'before' row, error analysis tells you which class of failure is largest, and the fix gives the 'after' row. A project with only an 'after' number cannot show that you did anything.

The offline test contract: what fake-client tests prove, and what they do not

Every project in this topic runs its tests on a laptop with no network, no GPU and no API key, and the same code then runs against a real model by changing three environment variables. The mechanism is a seam: one small interface that everything calls, with three implementations behind it.

What a fake-client test proves. Plumbing and invariants: a schema rejects float money, a retry fires exactly once, a budget stops spending, an approval gate blocks, a cache hits, a failover switches, and the metric code agrees with a number computed by hand. These are the things that break when you refactor, and they are deterministic, so they belong in CI.

What a fake-client test does not prove. Anything about model quality. A scripted fake that returns the right JSON proves your parser handles the right JSON. Every lesson in this topic states that in one sentence, and every quality number that depends on a real model is labelled 'illustrative, measure your own'. Numbers computed by running the lesson's own offline code (the arithmetic of a metric, the effect of a retrieval change on a bundled fixture) are real and quotable, because nothing in them depended on a model.

The three implementations, and the one decision each encodes.

Why standard library only. Installing a vendor SDK per provider couples your tests to one company's release cadence. A forty-line HTTP client does not, and a reader on a locked-down Windows laptop can run everything with python -m unittest.

:::key Verified 24 October 2026 Ollama documents an OpenAI-compatible endpoint at http://localhost:11434/v1/ and supports /v1/chat/completions; for local use the API key is 'required but ignored' (the docs use the placeholder ollama). Source: docs.ollama.com/api/openai-compatibility, fetched 24 October 2026. OpenAI's own API reference was not fetchable by my tooling on that date; a search summary says it marks max_tokens deprecated in favour of max_completion_tokens (reasoning models accept only the latter), while other servers (Ollama, vLLM) still read max_tokens. That is why OpenAICompatClient takes a max_tokens_param argument. Check the current API reference before you point it at a hosted API. :::

Build the client seam: llmkit.client

Define the interface before the first project, and make it small enough to read in one sitting. The folder layout is:

llmkit/
  llmkit/__init__.py   client.py   embed.py   cost.py   evalkit.py
  tests/test_llmkit.py
  demo.py

Create it, add a minimal pyproject.toml with name = "llmkit" and requires-python = ">=3.9", and install it into your virtual environment with pip install -e llmkit. Everything below is Python 3.9-compatible (I ran it on Python 3.9.13 on Windows), so it uses typing.Optional rather than X | None.

Completion carries the text, token usage in and out, latency in milliseconds and the model name. LLMClient is an abstract base class with one method. FakeLLM, ReplayLLM and OpenAICompatClient implement it. Read the code before the notes under it.

# -*- coding: utf-8 -*-
"""llmkit.client: one interface, three ways to satisfy it offline or online."""
import hashlib
import json
import os
import re
import time
import urllib.request
from abc import ABC, abstractmethod
from dataclasses import dataclass
from typing import Callable, Dict, List, Optional, Tuple, Union


def approx_tokens(text: str) -> int:
    """Rough token count (about 4 characters per token). A stated approximation."""
    return max(1, (len(text) + 3) // 4)


def messages_tokens(messages: List[dict]) -> int:
    return sum(approx_tokens(m["content"]) + 4 for m in messages)


@dataclass(frozen=True)
class Completion:
    text: str
    input_tokens: int
    output_tokens: int
    latency_ms: float
    model: str = ""


class LLMClient(ABC):
    model = "unknown"

    @abstractmethod
    def complete(self, messages: List[dict], **opts) -> Completion:
        """messages: [{"role": "system|user|assistant", "content": str}, ...]"""


Reply = Union[str, Callable[[List[dict]], str]]


class FakeLLM(LLMClient):
    """Deterministic. Queue entries are used first, in order; then rules in order.
    A rule is (needle, reply): needle is a substring or compiled regex searched in the
    last user message. A reply is a string or a function of the messages."""

    def __init__(self, rules: Optional[List[Tuple[Union[str, "re.Pattern"], Reply]]] = None,
                 queue: Optional[List[Reply]] = None, default: Optional[Reply] = None,
                 model: str = "fake-1"):
        self.rules, self.queue, self.default = list(rules or []), list(queue or []), default
        self.model, self.calls = model, []

    def complete(self, messages, **opts):
        self.calls.append({"messages": messages, "opts": opts})
        reply = None
        if self.queue:
            reply = self.queue.pop(0)
        else:
            last = next((m["content"] for m in reversed(messages) if m["role"] == "user"), "")
            for needle, r in self.rules:
                hit = needle.search(last) if hasattr(needle, "search") else needle in last
                if hit:
                    reply = r
                    break
            if reply is None:
                reply = self.default
        if reply is None:
            raise LookupError("FakeLLM: no queue entry or rule matched")
        text = reply(messages) if callable(reply) else reply
        tin, tout = messages_tokens(messages), approx_tokens(text)
        return Completion(text, tin, tout, 5.0 + 0.2 * tout, self.model)  # simulated latency


def request_key(messages: List[dict], opts: dict) -> str:
    blob = json.dumps({"m": messages, "o": opts}, sort_keys=True, ensure_ascii=False)
    return hashlib.sha256(blob.encode("utf-8")).hexdigest()


class CassetteMiss(KeyError):
    pass


class ReplayLLM(LLMClient):
    """Replays recorded responses keyed by a hash of the request. With record_with=<real
    client>, a miss calls it and saves the answer; without it, a miss raises."""

    def __init__(self, path: str, record_with: Optional[LLMClient] = None):
        self.path, self.record_with = path, record_with
        self.model = "replay"
        self.tape: Dict[str, dict] = {}
        if os.path.exists(path):
            with open(path, encoding="utf-8") as f:
                self.tape = json.load(f)

    def complete(self, messages, **opts):
        key = request_key(messages, opts)
        if key not in self.tape:
            if self.record_with is None:
                raise CassetteMiss(key[:12])
            c = self.record_with.complete(messages, **opts)
            self.tape[key] = {"text": c.text, "in": c.input_tokens, "out": c.output_tokens,
                              "ms": c.latency_ms, "model": c.model}
            with open(self.path, "w", encoding="utf-8") as f:
                json.dump(self.tape, f, indent=1, sort_keys=True, ensure_ascii=False)
        r = self.tape[key]
        return Completion(r["text"], r["in"], r["out"], r["ms"], r["model"])


class OpenAICompatClient(LLMClient):
    """POST {base_url}/chat/completions. Works with hosted APIs, vLLM and Ollama.
    Settings come from the environment (LLM_BASE_URL, LLM_API_KEY, LLM_MODEL), never code."""

    def __init__(self, base_url: Optional[str] = None, api_key: Optional[str] = None,
                 model: Optional[str] = None, timeout: float = 60.0,
                 max_tokens_param: str = "max_tokens"):
        self.base_url = (base_url or os.environ.get("LLM_BASE_URL", "")).rstrip("/")
        self.api_key = api_key or os.environ.get("LLM_API_KEY", "")
        self.model = model or os.environ.get("LLM_MODEL", "")
        if not self.base_url or not self.model:
            raise RuntimeError("set LLM_BASE_URL and LLM_MODEL (and LLM_API_KEY if the host needs one)")
        self.timeout, self.max_tokens_param = timeout, max_tokens_param

    def complete(self, messages, **opts):
        body = {"model": self.model, "messages": messages,
                "temperature": opts.get("temperature", 0)}
        if "max_tokens" in opts:
            body[self.max_tokens_param] = opts["max_tokens"]
        headers = {"Content-Type": "application/json"}
        if self.api_key:
            headers["Authorization"] = "Bearer " + self.api_key
        req = urllib.request.Request(self.base_url + "/chat/completions",
                                     json.dumps(body).encode("utf-8"), headers)
        t0 = time.perf_counter()
        with urllib.request.urlopen(req, timeout=self.timeout) as resp:
            data = json.loads(resp.read().decode("utf-8"))
        ms = (time.perf_counter() - t0) * 1000.0
        usage = data.get("usage") or {}
        text = data["choices"][0]["message"]["content"] or ""
        return Completion(text, int(usage.get("prompt_tokens", messages_tokens(messages))),
                          int(usage.get("completion_tokens", approx_tokens(text))), ms,
                          data.get("model", self.model))

Notes on decisions in that file.

The test that proves the HTTP client is real. The suite starts a tiny http.server on a free local port that returns a canned chat-completions response, points the client at it, and asserts the path (/v1/chat/completions), the Authorization: Bearer header, the max_tokens field and the parsed usage. That proves the request is well formed and the response is parsed, with no network and no key. It does not prove anything about any specific vendor, which is why the dated box above tells you to check the vendor's reference.

To run against a real model later, set LLM_BASE_URL, LLM_MODEL and, if the host needs one, LLM_API_KEY in your shell or a .env file that is in .gitignore. Commit an .env.example with empty values.

The rest of the seam: embeddings, cost, and the statistics a claim needs

Three small modules complete the foundation: a deterministic embedder, a cost meter, and just enough statistics to say how sure you are. Each is short because each does one job.

HashEmbedder turns text into a fixed-size vector with no model download: lower-case tokens plus adjacent word pairs are hashed (with BLAKE2, not Python's built-in hash, which changes between processes) into 256 slots with a random sign, then the vector is length-normalised. Two texts that share words get a high cosine; unrelated texts get about zero. It is a stand-in for a real embedding model in tests, so retrieval code runs for real and tests are deterministic, but real-embedder quality is the reader's own measurement.

# -*- coding: utf-8 -*-
"""llmkit.embed: a deterministic embedder for tests. No model download, no numpy."""
import hashlib
import math
import re
from typing import List

_TOKEN = re.compile(r"[a-z0-9_]+")


def tokenize(text: str) -> List[str]:
    return _TOKEN.findall(text.lower())


class HashEmbedder:
    """Feature-hashed bag of words plus bigrams, signed, L2-normalised."""

    def __init__(self, dim: int = 256, bigrams: bool = True):
        self.dim, self.bigrams = dim, bigrams

    def _slot(self, feature: str):
        h = hashlib.blake2b(feature.encode("utf-8"), digest_size=8).digest()
        n = int.from_bytes(h, "big")
        return n % self.dim, (1.0 if (n >> 63) & 1 else -1.0)

    def embed_one(self, text: str) -> List[float]:
        toks = tokenize(text)
        feats = toks + ([a + " " + b for a, b in zip(toks, toks[1:])] if self.bigrams else [])
        v = [0.0] * self.dim
        for f in feats:
            i, s = self._slot(f)
            v[i] += s
        norm = math.sqrt(sum(x * x for x in v))
        return [x / norm for x in v] if norm else v

    def embed(self, texts: List[str]) -> List[List[float]]:
        return [self.embed_one(t) for t in texts]


def cosine(a: List[float], b: List[float]) -> float:
    dot = sum(x * y for x, y in zip(a, b))
    na = math.sqrt(sum(x * x for x in a))
    nb = math.sqrt(sum(y * y for y in b))
    return dot / (na * nb) if na and nb else 0.0

CostMeter records each completion and adds up tokens and dollars from a price table. The table in the file is assumed, not a vendor quote: replace it with current prices from the provider's pricing page on the day you measure, and date the measurement in your README. It also supports a hard limit that raises BudgetExceeded, which is how a later lesson stops a public demo from spending your money. MeteredLLM wraps any client so every call is metered.

# -*- coding: utf-8 -*-
"""llmkit.cost: count tokens and dollars. Prices are ASSUMED, not a quote: replace them."""
from typing import Dict, List, Optional, Tuple

from .client import Completion, LLMClient

# dollars per million tokens (input, output). Assumed round numbers for teaching.
ASSUMED_PRICES: Dict[str, Tuple[float, float]] = {
    "small": (0.15, 0.60),
    "large": (2.50, 10.00),
    "fake-1": (0.15, 0.60),
    "replay": (0.15, 0.60),
}


class BudgetExceeded(RuntimeError):
    pass


class CostMeter:
    def __init__(self, prices: Optional[Dict[str, Tuple[float, float]]] = None,
                 limit_usd: Optional[float] = None):
        self.prices = dict(prices or ASSUMED_PRICES)
        self.limit_usd = limit_usd
        self.calls = 0
        self.input_tokens = 0
        self.output_tokens = 0
        self.dollars = 0.0
        self.latencies: List[float] = []

    def record(self, c: Completion) -> float:
        pin, pout = self.prices.get(c.model, self.prices["small"])
        d = (c.input_tokens * pin + c.output_tokens * pout) / 1e6
        self.calls += 1
        self.input_tokens += c.input_tokens
        self.output_tokens += c.output_tokens
        self.dollars += d
        self.latencies.append(c.latency_ms)
        if self.limit_usd is not None and self.dollars > self.limit_usd:
            raise BudgetExceeded("spent $%.6f, limit $%.6f" % (self.dollars, self.limit_usd))
        return d

    def percentile(self, p: float) -> float:
        xs = sorted(self.latencies)
        return xs[min(len(xs) - 1, int(p * len(xs)))] if xs else 0.0

    def summary(self) -> dict:
        return {"calls": self.calls, "input_tokens": self.input_tokens,
                "output_tokens": self.output_tokens, "dollars": round(self.dollars, 6)}


class MeteredLLM(LLMClient):
    """Wrap any client so every call is metered. Stacks with Fake, Replay or real."""

    def __init__(self, inner: LLMClient, meter: CostMeter):
        self.inner, self.meter, self.model = inner, meter, inner.model

    def complete(self, messages, **opts):
        c = self.inner.complete(messages, **opts)
        self.meter.record(c)
        return c

evalkit has the statistics you need to avoid overclaiming. bootstrap_ci resamples your per-item scores with a fixed seed and returns the point estimate and a 95 percent percentile interval. wilson is the standard interval for a proportion such as 'correct out of n'. paired_compare takes the same items scored under system A and system B, bootstraps the mean of the differences, counts wins and losses and runs an exact sign test on the disagreements. Paired is the important word: it uses the fact that both systems saw the same items, which is far more sensitive than comparing two separate intervals.

# -*- coding: utf-8 -*-
"""llmkit.evalkit: the least statistics a quality claim needs."""
import math
import random
from typing import Callable, Dict, List, Sequence, Tuple


def mean(xs: Sequence[float]) -> float:
    return sum(xs) / len(xs) if xs else 0.0


def bootstrap_ci(values: Sequence[float], stat: Callable = mean, n_boot: int = 2000,
                 alpha: float = 0.05, seed: int = 0) -> Tuple[float, float, float]:
    """Percentile bootstrap. Returns (point, low, high). Seeded, so reruns match."""
    rng = random.Random(seed)
    n = len(values)
    if n == 0:
        return 0.0, 0.0, 0.0
    stats = sorted(stat([values[rng.randrange(n)] for _ in range(n)]) for _ in range(n_boot))
    lo = stats[int((alpha / 2) * n_boot)]
    hi = stats[min(n_boot - 1, int((1 - alpha / 2) * n_boot))]
    return stat(values), lo, hi


def wilson(k: int, n: int, z: float = 1.96) -> Tuple[float, float, float]:
    """Wilson score interval for a proportion k/n."""
    if n == 0:
        return 0.0, 0.0, 0.0
    p = k / n
    d = 1 + z * z / n
    centre = (p + z * z / (2 * n)) / d
    half = z * math.sqrt(p * (1 - p) / n + z * z / (4 * n * n)) / d
    return p, max(0.0, centre - half), min(1.0, centre + half)


def sign_test_p(wins: int, losses: int) -> float:
    """Two-sided exact sign test on the disagreements (ties are dropped)."""
    n = wins + losses
    if n == 0:
        return 1.0
    k = min(wins, losses)
    tail = sum(math.comb(n, i) for i in range(k + 1)) / 2 ** n
    return min(1.0, 2 * tail)


def paired_compare(a: Sequence[float], b: Sequence[float], n_boot: int = 2000,
                   seed: int = 0) -> Dict[str, float]:
    """Same items scored under system A and system B. Bootstrap the mean of (b - a)."""
    assert len(a) == len(b), "paired scores must align item by item"
    diffs = [y - x for x, y in zip(a, b)]
    point, lo, hi = bootstrap_ci(diffs, mean, n_boot, 0.05, seed)
    wins, losses = sum(d > 0 for d in diffs), sum(d < 0 for d in diffs)
    return {"mean_diff": point, "ci_low": lo, "ci_high": hi, "b_wins": wins,
            "a_wins": losses, "ties": len(diffs) - wins - losses,
            "p_sign": sign_test_p(wins, losses),
            "significant": not (lo <= 0.0 <= hi)}

Wire the names together. Later lessons write from llmkit import LLMClient, HashEmbedder, wilson, so the package __init__.py must re-export exactly these names:

# -*- coding: utf-8 -*-
from .client import (Completion, LLMClient, FakeLLM, ReplayLLM, OpenAICompatClient,  # noqa: F401
                     CassetteMiss, approx_tokens)
from .embed import HashEmbedder, cosine  # noqa: F401
from .cost import CostMeter, MeteredLLM, BudgetExceeded  # noqa: F401
from .evalkit import bootstrap_ci, paired_compare, wilson  # noqa: F401

And demo.py, which produces the output below:

# -*- coding: utf-8 -*-
import os
import tempfile

from llmkit import (CostMeter, FakeLLM, HashEmbedder, MeteredLLM, ReplayLLM, bootstrap_ci,
                    cosine, paired_compare)

q = [{"role": "user", "content": "Capital of Karnataka?"}]
meter = CostMeter()
llm = MeteredLLM(FakeLLM(rules=[("Karnataka", "Bengaluru")]), meter)
print(llm.complete(q).text, meter.summary())

tape = os.path.join(tempfile.mkdtemp(), "tape.json")
ReplayLLM(tape, record_with=FakeLLM(default="recorded once")).complete(q, temperature=0)   # "live" run
print(ReplayLLM(tape).complete(q, temperature=0).text, "(replayed, no client)")             # CI run

e = HashEmbedder()
a, b, c = (e.embed_one(t) for t in ["sort a python list", "how do I sort a list in python", "monsoon in Kerala"])
print("cosine near %.3f far %.3f" % (cosine(a, b), cosine(a, c)))

old = [1] * 60 + [0] * 40          # 60/100 correct
new = [1] * 72 + [0] * 28          # 72/100 correct
print("old CI %.2f (%.2f-%.2f)" % bootstrap_ci(old, seed=1))
print({k: round(v, 4) if isinstance(v, float) else v for k, v in paired_compare(old, new, seed=1).items()})

Run it. The 15 unit tests (not printed, to save space) cover the fake's queue and rules, the cassette hit and miss, the local HTTP stub, the embedder, the cost arithmetic and budget stop, and hand-computed values for Wilson and the sign test; write your own for each behaviour listed in the notes above. The output below is exactly what this code printed (the demo run from the llmkit folder, with the package installed):

Bengaluru {'calls': 1, 'input_tokens': 10, 'output_tokens': 3, 'dollars': 3e-06}
recorded once (replayed, no client)
cosine near 0.488 far 0.000
old CI 0.60 (0.51-0.70)
{'mean_diff': 0.12, 'ci_low': 0.06, 'ci_high': 0.19, 'b_wins': 12, 'a_wins': 0, 'ties': 88, 'p_sign': 0.0005, 'significant': True}

and the tests:

----------------------------------------------------------------------
Ran 15 tests in 0.842s

OK

Read the last line of the demo. Going from 60 to 72 correct out of 100 looks like a clear win. The old system's own interval (0.51 to 0.70) barely excludes 0.72, and for a smaller gain it would not exclude it at all; the paired comparison, using the fact that the same 100 items were scored twice, says the mean improvement is 0.12 with a 95 percent interval of 0.06 to 0.19, and that the new system won 12 items and lost none. Here the improvement is solid. With 100 items and a smaller gain, it often would not be, and the right sentence in the README would be 'not distinguishable from the baseline at n=100'.

The names later lessons rely on. Do not rename them: llmkit.client has LLMClient, Completion (text, input_tokens, output_tokens, latency_ms, model), FakeLLM, ReplayLLM, OpenAICompatClient; llmkit.embed has HashEmbedder (embed, embed_one) and cosine; llmkit.cost has CostMeter, MeteredLLM, BudgetExceeded; llmkit.evalkit has bootstrap_ci, paired_compare, wilson.

How to show it: the repo, the README, the live demo

A reviewer decides in about the time it takes to scroll one screen, so the first screen must contain the result. The repo checklist: one project per repository and pinned on your profile; the README with the results table above the fold; a passing CI badge; an MIT or Apache-2.0 licence; no committed secrets or large files; a commit history that reads as milestones; and a make demo (or a documented python -m ... line) that works offline on the fake client.

The README template, in order. Every project lesson fills this in with its own numbers.

@@ 1 | One sentence: the problem and who it is for Not 'a chatbot'. 'Turns scanned supplier invoices into validated JSON and routes doubtful ones to a person.'

@@ 2 | A 30-second GIF or a live URL, and the one-command run make demo should work offline.

@@ 3 | The results table Golden-set size and how it was built; the headline metric with a confidence interval or at least n; baseline versus final; p50 and p95 latency; cost per request and per 1,000 requests with model name and price date; test count and what the CI gate blocks; the date and model version of the measurement, because numbers rot.

@@ 4 | Architecture and three decisions One diagram and the three design decisions with what each gave up.

@@ 5 | How the evals work Set size and provenance, metrics, the gate threshold and why, how to rerun.

@@ 6 | Failure analysis Three concrete failures, root cause, fix, and the metric before and after.

@@ 7 | Limits and next steps Honest. Include what the offline tests do and do not prove.

@@ 8 | Repo map, how to run tests, how to reproduce the eval, licence.

Hosting a live demo: verified facts, October 2026. A live link is repeatedly recommended, but the free-tier landscape moved, and two of the options people still recommend no longer work the way older tutorials say.

:::key Verified 24 October 2026 (check current limits before you rely on any of this) Hugging Face Spaces. The Spaces overview page says 'CPU Basic' hardware (2 vCPU, 16 GB RAM) has no hourly cost, but 'creating a Space that runs on compute (Gradio or Docker) requires a paid plan' (PRO for personal accounts); Static Spaces are free for everyone, and free accounts can host up to 2 Gradio Spaces on ZeroGPU. Free hardware goes to sleep after a period of inactivity. So a free Docker Space is no longer available to a new free account; a static page or a ZeroGPU Gradio Space are the free routes. Render. The free-instances page says a Free web service spins down after 15 minutes without inbound traffic, takes 'about one minute' to wake, has no persistent disk, and each workspace gets 750 free instance hours a month. Fly.io. Pricing docs: 'New organizations don't have a free tier or a monthly free usage allowance'; a new account gets a trial of up to 2 hours of Machine runtime or 7 days, whichever comes first. Google Cloud Run. A search summary (not the primary page, which my tooling could not render) reports a free tier of 2 million requests, 180,000 vCPU-seconds and 360,000 GiB-seconds a month under request-based billing in Tier 1 regions; verify on cloud.google.com/run/pricing before you use it. :::

Whatever you choose, build for the sleeping instance: the README says the first request may take a minute, and a 60 to 90 second screen recording sits next to the link for when the free instance is down. Show one failure on purpose in that recording. For the public demo, put a per-IP rate limit, a maximum question length and a daily spend cap (CostMeter(limit_usd=...)) in front of any real model, and fall back to the fake client when the cap is hit. Scan the repo for secrets before you make it public (a pre-commit hook or the host's secret scanner), because a leaked key is the most expensive portfolio mistake there is.

The 90-second pitch, the resume bullet, and an honest time estimate

Prepare the pitch as a five-beat script, because interviews open with 'tell me about a project' and most people ramble.

@@ 1 | 0 to 15 s: the problem and the user, in plain words 'Finance teams at small distributors key in supplier invoices by hand. I built a pipeline that extracts them into validated JSON.'

@@ 2 | 15 to 35 s: the one design decision that mattered, and why 'Hybrid retrieval, because dense-only search missed exact error codes.' Name the alternative you rejected.

@@ 3 | 35 to 60 s: how you measured it Golden set size and origin, the metric, the gate. 'A 100-document set, 20 held out, with an interval on every number.'

@@ 4 | 60 to 80 s: the failure you found and fixed The thing hiring-side sources say reads as seniority. Name it, give before and after.

@@ 5 | 80 to 90 s: cost and latency, and what you would do next One number each, with the date, and one sentence on the next step.

The resume-bullet formula. Built <system> (<stack>) that <does X for Y>; moved <metric> from <A> to <B> on a <n>-example golden set, p95 <t> s at <$c> per 1,000 requests; CI eval gate blocks regressions; deployed at <short URL>.

:::warn Examples below are ILLUSTRATIVE The numbers in these examples are placeholders showing the shape. Replace each with a figure your repo regenerates with one command. Never put a number on a resume that you cannot reproduce, because interviewers drill into exactly the claims you call hard. :::

The five questions to rehearse, for any project. What was the hardest part? What failed? What would you change? Why this model or approach and not a framework? How do you know it works? If your answer to the last one starts with a number and its interval, you are ahead of most candidates. Each project lesson adds five project-specific questions with answer outlines.

Pitfalls across all projects. Tuning on the held-out set; a golden set written by looking at the answer text (lexical leakage that flatters keyword search); reporting an average where a per-class breakdown shows the real weakness; no baseline row; one run of a non-deterministic metric with no interval; secrets in git history; a README that describes the architecture and shows no results; and a 'live demo' that sleeps for a minute with no recording next to it.

Stretch goals. Publish llmkit as a template repository other projects clone; add a STEmbedder behind the same embed interface for real runs; add a telemetry module that writes one JSONL line per call; add a GitHub Actions workflow running python -m unittest and your eval gate offline.

Honest time estimate. About 2 hours to type, run and understand llmkit, 1 hour to set up the repo skeleton and CI file, 1 hour to draft the README skeleton and the pitch, and 1 to 2 hours of reading the sources and the process section. Five to six hours, and it saves that much on every project after the first.

How to say it in the interview

"Every project I build goes through the same seven stages, and the number comes before the code: I write the success metric, build the dumbest end-to-end version, build a golden set early, read thirty failures by hand, fix the biggest class, and report before and after with an interval. All of it runs offline through one small seam, a client interface with a scripted fake, a replay cassette and an HTTP client, so my tests need no key and my CI is free. The README leads with the results table, and I only claim numbers that one command regenerates."

Checklist

Practice this interactively
Share this lessonWhatsAppLinkedInX

Related lessons