The spelled-out intro to language modeling: building makemore

Name: The spelled-out intro to language modeling: building makemore
Description: Andrej Karpathy builds a bigram character-level language model from scratch, introducing torch.Tensor and the framework of training, sampling and loss evaluation. It is the first step toward a GPT-style Transformer in his makemore series.

Andrej Karpathy

Andrej Karpathy builds a bigram character-level language model from scratch, introducing torch.Tensor and the framework of training, sampling and loss evaluation. It is the first step toward a GPT-style Transformer in his makemore series.

HN recommends it as a long but worthwhile way to demystify how language models work technically.

4 Hacker News mentions · 2024–2024

Topics

Machine learning & deep learning
LLMs & generative AI

Watch on YouTube

Explore more videos