01
LLM From Scratch
Taking the modern LLM stack apart, then building one back up.
A self-directed attempt to understand large language models from the inside. It runs from tokens, embeddings and retrieval through agents and serving, then on to the main event: building a GPT from the ground up and fine-tuning it with LoRA. Every experiment gets logged, and the local work runs on a laptop with no GPU, which keeps the ambition honest. The repo stays private until the loss is presentable.
10M
parameter GPT
4-bit
quantization target
0
contributions to the GPU Apocalypse
- How LLMs work
- Embeddings & RAG
- Agents & MCP (not started yet)
- Tiny GPT from scratch (not started yet)
- LoRA fine-tuning (not started yet)
2 of 5 stages complete.
- Python
- PyTorch
- Ollama
- Embeddings
- RAG