Data Structures for AI

Topic

In one sentence: A practical course on the data structures that power modern AI systems — tensors, embeddings, sparse matrices, attention/KV caches, ANN indexes (HNSW, IVF, PQ), graph stores, and tries — covering both their internal mechanics and how to choose and tune them in production.

The full topic: “Data Structures for AI” sits at the intersection of classical CS data structures and modern AI workloads. Most AI courses skip the data-structure layer entirely (jumping from math directly to model APIs); most data-structure courses skip AI (stopping at trees, hashmaps, and graphs in the abstract). This course fills the gap: how the data structures actually used in AI systems work, why they’re shaped the way they are, what their failure modes look like in production, and how to pick the right one for a workload.

This course is NOT about ML model architectures (transformers as models), training algorithms (SGD, Adam), or theoretical complexity proofs. It IS about: how a transformer’s KV cache is physically laid out and why that matters for latency; why HNSW beat tree-based ANN; what a Bloom filter does inside a vector database’s filter step; how sparse-dense hybrid representations work in retrieval.

Target Student

Primary audience: Software engineers and ML engineers with 1–4 years of experience who are building AI-powered systems (RAG, agents, model-serving infrastructure, recommendation systems, AI features in apps). They already know Python and basic data structures (arrays, hashmaps, trees) but want to confidently reason about the data-structure choices inside vector DBs, model serving, and retrieval systems.

Who this is NOT for:

• Absolute beginners to programming or Python
• ML researchers looking for theoretical depth on training algorithms
• Pure data-structures students (use CLRS or similar)
• Anyone wanting “another LangChain tutorial”

Assumed prior knowledge:

• Comfortable writing Python: classes, list/dict comprehensions, basic NumPy
• Knows core data structures: arrays, hash maps, balanced trees, graphs (BFS/DFS)
• Knows what Big-O is and can compute it for simple loops
• Has used (or at least heard of) an embedding model and a vector DB at a surface level
• Comfortable reading code in one of: PyTorch, NumPy, or equivalent

Depth and Breadth

Choice: broad-with-depth-at-key-points

Justification: The course breadth covers the full data-structure stack of an AI system, but goes deep on the structures students most commonly mis-tune: tensors/strides, HNSW, KV caches, sparse-dense retrieval, and graph-of-thought structures. Surface-level structures (hashmaps for tokenizers, simple Bloom filters) get a single lesson; load-bearing ones (HNSW, attention layouts) get worked examples + interactive labs.

Length and Pacing

• Target total hours: 12–15 hours of student effort
• Number of modules: 6 modules (firm estimate; curriculum architect can adjust ±1)
• Pacing assumption: evenings-and-weekends, self-paced over 3–6 weeks

Delivery Format

Primary format: Markdown folder (file-system-based course package with .md lessons, quizzes, assignments, and HTML labs)

Secondary outputs wanted: None for v1.

If instructor-led: N/A for v1 (self-paced).

Constraints

Technology constraints:

• Must cover: NumPy (foundational), PyTorch tensors (mention CUDA/memory layout briefly), FAISS or similar ANN library, at least one vector DB (e.g., LanceDB, Qdrant, or pgvector — picked by SME pass)
• Must avoid: deep dives into specific cloud APIs (course should be portable); proprietary-only tools without an open-source equivalent
• Version pins: Python 3.11+, NumPy 1.26+, PyTorch 2.x — confirmed by validation pass

Scope constraints:

• Hard exclusions: transformer model architecture deep-dives, training math, RLHF/post-training topics, GPU kernel programming (CUDA C++)
• Budget constraint: 6 modules, 12–15 hours student effort, single delivery session

Content constraints:

• Voice/tone: senior engineer explaining to a smart junior teammate — precise, opinionated, plain prose; no marketing speak; honest about trade-offs
• Examples from specific domains: prefer retrieval/RAG, recommendation systems, and LLM serving examples since those are where students will most likely apply the knowledge
• Avoid: medical/finance regulatory examples (too domain-heavy)

Success Criteria

What does a successful course look like?

• A grad can read a vector DB’s documentation and explain why it chose the index type it did (instead of treating it as a black box)
• A grad can be handed a slow retrieval system and form a hypothesis about which data structure is the bottleneck
• A grad can implement a from-scratch HNSW (or k-NN with simple heuristics), an ANN-vs-brute-force benchmark, and a transformer-style KV cache buffer — and explain the engineering choices they made
• A grad can defend the choice of one ANN index over another in a design interview

Pipeline Configuration

• Compressed 6-stage pipeline (per skill’s “Scaling Down” guidance for an ~12 hour course): collapsed Discovery (1+2+3 → one combined Discovery subagent), full Architecture (4), inline Pedagogy (folded into Writing per skill guidance), Validation pre+post (6), Writing+Assessment+Lab (7+8+9 per module), QA (10), Packaging (11). Stage 12 condensed to a single watchlist appendix.
• User checkpoints enabled: After scope is set (informal — brief sign-off below); after module map is produced (hard checkpoint); after first module is written (voice check)
• Parallelism target: 2–3 modules concurrent in build phase