Deep learning teaching

October 2026. I design these practicals to keep iteration times short, so students see results and feel rewarded quickly. In our undergraduate and postgraduate practicals, each student gets an isolated Multi-Instance GPU (MIG) slice of an 80 GB NVIDIA A100: roughly 1/7 of its compute resources and 10 GB of dedicated GPU memory. In practicals 5–9, we train small models from scratch in minutes: a GPT on Shakespeare following Karpathy’s nanoGPT, image diffusion on CIFAR-10 horses, audio flow matching on a few AudioMNIST voices, and implicit video networks.

GPT from scratch

Building a GPT in plain PyTorch connects byte tokens and causal attention to next-token prediction. An excerpt generated by the teaching model after a couple of minutes of training on a student’s A100 share:

ROMEO:
 that he had served up his world sons and ourselves!

ROMEO:
Why, now what she did not wear them but spent?

Image diffusion

A small diffusion model learns to turn noise into images using only the horse class of CIFAR-10. Training the teaching model and generating these samples also takes a couple of minutes on a student’s A100 share.

Audio flow matching

Generate speech from noise, then interpolate between female and male voices. Holding the initial noise fixed lets students hear the effect of changing the conditioning. These are five versions of “seven”.