Cruxly › AI engineering interview preparation

AI engineering interview preparation

AI engineering interviews test whether you can build and operate LLM-backed systems, not just call an API. These lessons run from foundations and prompting through evaluation, retrieval, agents, multimodal, serving and cost, training and governance, then case studies and portfolio projects, each with what to say and what to check.

LLM Foundations

Transformers, tokens, embeddings, sampling and the pre-train to chat-model pipeline: the mental model every later lesson (and every interviewer) assumes you have.

• Transformers and Attention, From the Whiteboardone attention row by hand, then the parameter and KV-cache arithmetic
• Tokenization: BPE, Vocabulary Size, and Why Tokens Are the Unit of Everythingbuild a BPE merge table by hand, then price a vocabulary choice
• Embeddings and Similarity: Cosine, Dot Product, and What a Vector Encodesthe contrastive loss, cosine versus dot versus L2, and 50 million vectors in gigabytes
• Sampling and Decoding: Temperature, Top-p, Beam Search, and Determinismlogits to a chosen token, and why temperature 0 still is not repeatable
• From Pretraining to Chat Model: The Pipeline and the Scaling Arithmeticwhat each training stage changes, and training cost from 6ND

Prompting, Structured Output & Tool Calling

Prompts as versioned code, schemas that hold, the function-calling loop, reasoning models, and how to improve a prompt without guessing.

• Prompt Engineering as Engineering: Structure, Examples and What Actually Moves Qualityfive parts, one rule for documents, diverse examples, and a paired test that tells you whether an edit was real
• Structured Outputs: Schemas, Validation Loops and What 'Guaranteed' Meanswhat schema-constrained decoding guarantees, the five ways a valid-looking answer still breaks, and a validation loop with a cap
• Function Calling Basics: The Loop, Parallel Calls, Errors and Forced Choicethe model asks, your code runs, the history grows: the loop, parallel results, iserror, toolchoice and the token bill
• Reasoning Models and Test-Time Compute: When to Think and How to Prompt Themthinking tokens are billed output, effort is the dial, and the right level comes from a paired sweep, not a default
• The Prompt Lifecycle: Versioning, Regression Tests and Automatic Optimisationprompts as versioned artifacts, a regression suite, a canary you can actually read, and what an optimiser needs before you trust it

Evaluation

Golden sets, error analysis, metrics that mean something, offline versus online evaluation, and how to catch a hallucination before a user does.

• Evaluation Foundations: Golden Sets, Error Analysis and Reading Benchmarksweek one on a feature with no evals, a golden set you can defend, 1,090 versus 625 items, and why 92 percent on MMLU says little
• Metrics That Mean Something: Task Metrics, Ranking Metrics, pass@k and SignificancenDCG@3 by hand, MRR, an unbiased pass@k, a 14-to-5 split that three tests grade differently, and cost per success
• LLM-as-a-Judge: Evaluation That Scales Without Lying to Youa model grading a model is a measuring instrument, and instruments have biases
• Hallucination: Taxonomy, Detection and Mitigation in Production37 unsupported claims in 21 answers, a verbatim-quote check, abstentions from 4 to 13, and a threshold set by rupees
• Offline Versus Online Evaluation: Shadow Mode, A/B Tests and Feedback Loopsz = 2.9 and still do not ship: the rollout ladder, biased thumbs, peeking at 14 percent false positives, and the weekly flywheel

Retrieval and RAG

Build RAG from raw SDKs, then make it survive real data — multi-tenant, multilingual, and evaluated on every deploy.

• RAG From Scratch, No Frameworkchunk, embed, retrieve, generate — with only the SDKs
• Chunking and Document Parsing: Structure-Aware Splits, Parent-Child, Late Chunking and Contextual Retrievalparse first, count the overlap tax, and test five strategies on one span-labelled set
• Choosing, Evaluating and Fine-Tuning an Embedding Modela 300-query eval beats the leaderboard, plus storage, re-embed cost and the migration plan
• ANN Indexes in Depth: HNSW, IVF-PQ, DiskANN and the Recall/Latency/Memory Triangle307 GB of vectors, a 25.6 GB graph, a 9.6 GB code table, and why a filter returns 3 results instead of 10
• Hybrid Search, Rerankers and Late InteractionBM25 by hand, RRF by hand, the cost of a reranker, and a 200 ms budget you can defend
• Query Understanding and Agentic Retrieval: Rewriting, Routing, Multi-Hop and GraphRAGfix the query before the index, then pay for a loop only when the question needs three documents
• Multi-Tenant, Multilingual RAG at Scalerow-level security, language routing, and graph-aware hops
• Ingestion Pipelines: Sync, Freshness, Permissions and Deletionhashes instead of re-embeds, a leak suite instead of hope, and a delete that reaches five stores
• Evaluation Gates That Actually Block a Bad DeployRAGAS, LLM-as-judge, and CI that fails the build on a fluent wrong answer

Agentic Systems

Stateful agents, tool contracts, approval gates, and an audit trail you can actually defend.

• The Agent Loop in Depth: ReAct, Plan-and-Execute, Reflection and Stop Conditionsper-step reliability compounds, context cost grows quadratically, and the stop decision never belongs to the model alone
• Designing Tools an Agent Can Actually Usea tool is an interface whose user can't ask you what you meant
• Agent Memory: Working, Episodic, Semantic and Procedural, Without the Rotwhat to write, what to read back, what 8,000 tokens a turn costs, and how memory gets poisoned or goes stale
• Frameworks, Protocols and Durable Execution: MCP, A2A and Long-Running Agentsbuild or adopt, what the 2026-07-28 MCP revision changes for your server, and how a 2-hour run survives a crash
• MCP Servers, Prompt Injection, and Tamper-Evident Auditexposing a system as an agent tool, safely, with proof of what happened
• Guardrails for an Agent That Takes Real Actionstool contracts, approval gates, and the line between automated and autonomous
• Coding Agents and Computer-Use Agents: Harness, Sandbox and Verificationa 40-step session costs $9.84 uncached and about $1.07 cached; the sandbox, the oracle and the screenshot loop that decide whether it ships

Multimodal & Document Understanding

How images become tokens, what they cost, and when to use OCR, a vision-language model, or both.

• Vision-Language Models: How Images Become Tokens, What They Cost, and Where They Fail28-pixel patches, a token formula you can compute by hand, 4.7 million tokens for an hour of video, and the failures to test for
• Document Understanding: OCR, Layout Models, VLMs and the Hybrid That Usually Winsroute by page, extract with citations and abstention, and price 100,000 pages a month at $1,120, $56 or about $109

LLM Backend Infrastructure

Multi-provider failover, cost attribution, and async pipelines — the backend that makes an LLM call survive production.

• Multi-Provider Failover That Actually Fails Overcircuit breakers, fallback chains, and the flapping trap
• Cost Attribution and Runaway Preventiontag before you send, cap before you regret it
• Prompt Caching: Paying for the Same Tokens Onceprompt layout is the cache key, and one timestamp can cost you all of it

AI-Native Interfaces

Streaming responses, confidence and severity indicators, model selection, and accessibility for an interface that thinks out loud.

• Streaming Interfaces That Show Their WorkSSE plumbing, confidence surfacing, and a stream that fails halfway
• Model Selection: Cost, Latency, and Capability Trade-offsthe frontend inherits every consequence of a choice it doesn't make
• Multi-Turn Conversation State and Context-Window Limitsthe conversation the user experiences is not the one the model sees

Reliability, Security & Governance

Observability, semantic caching, adversarial red-teaming, and data-protection compliance as code, not an afterthought.

• Safety and Alignment Basics for Engineerswhere trained behaviour ends and product policy begins, two filters that (at assumed rates) leave 0.32 percent and cost 3.4 percent of…
• Observability and Cost Governance for LLM SystemsLangfuse-style tracing, semantic caching, and the alert that catches drift
• Red-Teaming and Compliance as Codethe OWASP LLM Top 10, adversarial testing, and DPDP erasure that actually erases

NLP and Text Classification

Encoder-vs-LLM break-even arithmetic, label quality, per-class evaluation tied to business cost, and multilingual tokenizer economics.

• Text Classification: When a Fine-Tuned Encoder Beats an LLMthe break-even arithmetic, and the hybrid everyone lands on
• Building the Labeled Datasetlabel quality caps model quality — no architecture recovers from bad labels
• Evaluation Beyond Accuracyper-class metrics, calibration, cost-optimal thresholds, and drift that actually matters
• Multilingual and Domain-Specific Texttoken inflation, code-switching, uneven degradation, and per-language evaluation

Fine-Tuning and Model Adaptation

Fine-tune vs RAG vs prompt as a real decision, LoRA/QLoRA mechanics with the 2026 corrections, and training data hygiene that catches a fine-tune that didn't actually work.

• Fine-Tune, RAG, or Prompt: The Decision Nobody Wants to Make Twicethe data-changes test, the citation test, and the default-to-RAG pattern
• LoRA and QLoRA: What Actually Changes During Fine-Tuningall-linear targeting, the 10x learning rate, and serving many adapters on one base model
• Training Data: Dedup, Contamination, and Knowing It Actually Workednear-duplicate detection, benchmark contamination, catastrophic forgetting, and the before/after eval that proves it
• RLHF, DPO and RL with Verifiable Rewards: How Post-Training Actually Worksa 0.664 DPO loss, a +1.29 GRPO advantage, four models in memory, and the reward you must not let the model game
• Distillation and Synthetic Data: Teachers, Students and the Quality Loopa $400 data bill, a 25-day payback, 10 TB of logits you will not store, and the golden set that decides whether it worked

Inference Serving

GPU memory math, continuous batching, latency budgets and fleet economics — what happens when the model runs on hardware you pay for.

• GPU Memory Math: Weights, KV Cache, and What Actually Fitswhy KV cache, not weights, caps your concurrency
• Quantization Methods Compared: INT8, GPTQ, AWQ, FP8, FP4 and KV-Cache Quantizationone outlier that turns a sub-1-percent error into 31 percent, a 4-bit hand example, and why INT4 gave no speedup at batch 128
• FlashAttention and Why Attention Is a Memory-Traffic Problema 134 MB score matrix per head, an online softmax you can do on paper, and which of FlashAttention, PagedAttention and GQA fixes which…
• Continuous Batching and the Scheduleriteration-level scheduling, and why prefill and decode are two different jobs
• TTFT, TPOT, and the Latency Budgetwhy an average latency number tells you nothing about a streaming endpoint
• Mixture-of-Experts and Long-Context Serving671 GB of weights for a 37B-active model, 11 requests at 128K tokens, and what a 1M-token prompt costs before the first output token
• Autoscaling a GPU Fleet Without Burning the Budgetcold starts, queue-depth scaling, spot draining, and the break-even math

AI System Design Case Studies

Six end-to-end designs (support copilot, enterprise RAG, code assistant, document extraction, voice agent, LLM recommendations) that pull the whole track together under interview conditions.

• Case Study: A Customer-Support Copilot for 2 Million Tickets a Monthdraft-and-suggest, a policy layer the model cannot talk its way past, and a cost line that halves twice
• Case Study: Enterprise Search and Q&A Over 20 Million Documents With Permissions160 million vectors, an ACL on every chunk, and the proof that nobody reads what they should not
• Case Study: An IDE Code Assistant (Completion, Chat and Agent Mode)decode is the cost, a 160 ms completion budget, and why acceptance rate alone lies
• Case Study: Invoice and Contract Extraction at 500,000 Pages a Montha funnel of 88, 8 and 4 percent, the field-versus-page accuracy trap, and two reviewers instead of twenty
• Case Study: A Voice Agent for Inbound Calls850 milliseconds to first audio, an endpointing dial with no free setting, and a cost model that is all audio
• Case Study: Adding LLMs to a Recommendation System Without Melting the Budget200 million item views a day, a $60,000 bill you do not pay, and the three places an LLM actually earns its tokens

Portfolio Projects

Build it yourself: ten tested, measurable AI projects (extraction, RAG, tracing, evals, text-to-SQL, agents, a gateway, voice, fine-tuning and a capstone), plus how to ship, write up and talk about them.

• Build Projects That Get You Hired: The Process, the Shared Test Seam and How to Show Itseven stages from idea to interview, one tiny offline package (llmkit) that every project imports, and the README, pitch and resume bullet
• Project: A Document Extraction Pipeline With Validation and a Field-Level Scorecardschema, repair loop, arithmetic and evidence checks, a second read, confidence routing and a scorecard that reports error among…
• Project: A RAG Assistant Over Real Documents With an Eval Gate in CIhybrid retrieval, cited answers, abstention, a golden set with intervals and a CI gate, over the Python standard-library docs
• Project: A Tracing and Cost Dashboard for LLM Apps (Build Your Own Mini-Langfuse)span trees, token and cost accounting you can verify to the cent, percentiles, budget stops, redaction and sampling, in a library with a…
• Project: An Eval Harness and LLM-as-Judge Toolkit You Can Trustdeterministic graders first, a judge that must pass a calibration bar, paired comparisons with intervals, and a regression gate whose…
• Project: A Text-to-SQL Analyst With Read-Only Guardrails and Execution-Based Evalsa read-only connection the model cannot widen, a validator that refuses what it cannot explain, schema linking that decides what the model…
• Project: A Tool-Using Agent With Guardrails, an Approval Gate and an Audit Logtyped tools, budgets that stop before a call, a policy engine the model cannot argue with, a durable pause for large refunds, a…
• Project: A Cost-Aware LLM Gateway With Routing, Caching, Failover and Budget CapsAn OpenAI-compatible proxy that routes by difficulty, caches safely, fails over, and enforces budgets under concurrency
• Project: A Hindi and Hinglish Voice Assistant With Endpointing, Barge-In and a Latency BudgetSpeech in, grounded speech out, with a per-turn latency budget, barge-in and a document-photo path
• Project: Fine-Tune or Distill a Small Model, With a Proper Before/After EvaluationDistil a scripted teacher into a small student, then prove with intervals, leakage checks and a break-even gate whether to ship it
• Capstone Project: A Support Copilot That Combines RAG, a Guarded Agent and a GatewayDraft-and-suggest copilot: cited answers, approval-gated actions, per-customer gateway keys, an eval gate and a rupee cost model