TinyTorch lets developers build PyTorch from scratch

Developed by researchers at Harvard University and ETH Zurich, TinyTorch is a 20-module educational curriculum designed to teach machine learning systems from the ground up. Instead of just importing PyTorch, students write their own functional machine learning framework in pure Python, mirroring PyTorch's API. The entire course is designed to run locally on highly accessible hardware, requiring only a dual-core 2 GHz CPU, 4 GB of RAM, and no GPU.

The curriculum is delivered via Jupyter notebooks and managed through a command-line interface called tito. It guides learners through four tiers of development, culminating in six historical milestones. These milestones include replicating Rosenblatt's 1958 Perceptron, resolving the 1969 XOR crisis, implementing the 1986 backpropagation revival, building a 1998 convolutional neural network that clears a 75 percent accuracy threshold on CIFAR-10, constructing a 2017 transformer, and running MLPerf-style benchmarks. To keep the hardware requirements low, the course ships with two offline datasets under 50 megabytes: 1,000 grayscale digits and 350 conversational question-answer pairs.

For practitioners, building a framework from scratch demystifies complex abstractions like autograd, memory allocation, and attention mechanisms. Although the resulting pure Python code runs 100 to 10,000 times slower than production PyTorch, this performance gap serves as a pedagogical tool to illustrate the necessity of vectorization. The project deliberately excludes complex production features like C++ or CUDA layers, dispatchers, and distributed training to keep the codebase readable.

TinyTorch originated from a 2020 Harvard graduate seminar on TinyML and has since grown into a massive open-source project, with its GitHub repository climbing from 2,000 stars in August 2025 to over 27,000. Currently in preview, the creators are targeting full classroom readiness for Fall 2026, with a maintenance commitment extending through 2027.