Resources

Basics of deep learning and LLMs

• Introduction to Deep Learning by Sebastian Raschka
• NanoGPT by Andrey Karpathy - simplest transformer built from scratch
• Vardan Papyan’s course on analysis of deep learning

Topics on interpretability and AI safety

• Overview of mechanistic interpretability
• Coefficient giving (formerly Open Philanthropy) - A call for AI safety research
• My research blogs
• Wisconsin AI Safety Initiative

Interviews and perspectives

• Ilya Sutskever – We’re moving from the age of scaling to the age of research
• Andrej Karpathy - AGI is still a decade away
• Richard Sutton – Father of RL thinks LLMs are a dead end