Reading

Books & papers

• Role of Bias Terms in Dot-Product Attention

  Mahdi Namazifar, Devamanyu Hazarika, Dilek Hakkani-Tur

• Build a Large Language Model (from Scratch)

  Sebastian Raschka

• AI Engineering

  Chip Huyen

• Designing Machine Learning Systems

  Chip Huyen

• Attention Is All You Need

  Ashish Vaswani et al.

• An Image Is Worth 16x16 Words: Transformers for Image Recognition at Scale

  Alexey Dosovitskiy et al.

• GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints

  Joshua Ainslie et al.

• Hands-On Machine Learning with Scikit-Learn and PyTorch

  Aurelien Geron

• Efficient and Robust Approximate Nearest Neighbor Search Using Hierarchical Navigable Small World Graphs

  Yu. A. Malkov, D. A. Yashunin

• Cosine Similarity

  Antonio Lupetti

• Post-training 101 | Tokens for Thoughts

  Han Fang

• Understanding the 4 Main Approaches to LLM Evaluation (From Scratch)

  Sebastian Raschka PhD

• A Gentle Introduction to Graph Neural Networks

  Benjamin Sanchez-Lengeling, Emily Reif, Adam Pearce, Alexander B. Wiltschko

• A Visual Guide to Gemma 4

  Maarten Grootendorst

• Triton: an intermediate language and compiler for tiled neural network computations

  Philippe Tillet, H. T. Kung, David Cox

• Why do CPUs have multiple cache levels?

  Unknown author

• Quantization from the ground up

  Sam Rose

• Tim Rocktäschel

  Tim Rocktaschel

• Inside NVIDIA GPUs: Anatomy of high performance matmul kernels - Aleksa Gordić

  Aleksa Gordic

• Heterogeneous Graph Transformer

  Ziniu Hu, Yuxiao Dong, Kuansan Wang, Yizhou Sun

• Robust LLM Training Infrastructure at ByteDance

  Borui Wan, Gaohong Liu, Zuquan Song, Jun Wang, Yun Zhang, Guangming Sheng, Shuguang Wang, Houmin Wei, Chenyuan Wang, Weiqiang Lou, Xi Yang, Mofan Zhang, Kaihua Jiang, Cheng Ren, Xiaoyun Zhi, Menghan Yu, Zhe Nan, Zhuolin Zheng, Baoquan Zhong, Qinlong Wang, Huan Yu, Jinxin Chi, Wang Zhang, Yuhan Li, Zixian Du, Sida Zhao, Yongqiang Zhang, Jingzhe Tang, Zherui Liu, Chuan Wu, Yanghua Peng, Haibin Lin, Wencong Xiao, Xin Liu, Liang Xiang

• DeepSeek's open-source week and why it's a big deal

  PySpur-AI Agent Builder

• Reducing Activation Recomputation in Large Transformer Models

  Vijay Korthikanti, Jared Casper, Sangkug Lym, Lawrence McAfee, Michael Andersch, Mohammad Shoeybi, Bryan Catanzaro

• The Ultra-Scale Playbook - a Hugging Face Space by nanotron

  Unknown author

• Role of Bias Terms in Dot-Product Attention

  Mahdi Namazifar, Devamanyu Hazarika, Dilek Hakkani-Tur

• GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints

  Joshua Ainslie, James Lee-Thorp, Michiel de Jong, Yury Zemlyanskiy, Federico Lebrón, Sumit Sanghai

• Gemma 3 Technical Report

  Gemma Team, Aishwarya Kamath, Johan Ferret, Shreya Pathak, Nino Vieillard, Ramona Merhej, Sarah Perrin, Tatiana Matejovicova, Alexandre Ramé, Morgane Rivière, Louis Rouillard, Thomas Mesnard, Geoffrey Cideron, Jean-bastien Grill, Sabela Ramos, Edouard Yvinec, Michelle Casbon, Etienne Pot, Ivo Penchev, Gaël Liu, Francesco Visin, Kathleen Kenealy, Lucas Beyer, Xiaohai Zhai, Anton Tsitsulin, Robert Busa-Fekete, Alex Feng, Noveen Sachdeva, Benjamin Coleman, Yi Gao, Basil Mustafa, Iain Barr, Emilio Parisotto, David Tian, Matan Eyal, Colin Cherry, Jan-Thorsten Peter, Danila Sinopalnikov, Surya Bhupatiraju, Rishabh Agarwal, Mehran Kazemi, Dan Malkin, Ravin Kumar, David Vilar, Idan Brusilovsky, Jiaming Luo, Andreas Steiner, Abe Friesen, Abhanshu Sharma, Abheesht Sharma, Adi Mayrav Gilady, Adrian Goedeckemeyer, Alaa Saade, Alex Feng, Alexander Kolesnikov, Alexei Bendebury, Alvin Abdagic, Amit Vadi, András György, André Susano Pinto, Anil Das, Ankur Bapna, Antoine Miech, Antoine Yang, Antonia Paterson, Ashish Shenoy, Ayan Chakrabarti, Bilal Piot, Bo Wu, Bobak Shahriari, Bryce Petrini, Charlie Chen, Charline Le Lan, Christopher A. Choquette-Choo, C. J. Carey, Cormac Brick, Daniel Deutsch, Danielle Eisenbud, Dee Cattle, Derek Cheng, Dimitris Paparas, Divyashree Shivakumar Sreepathihalli, Doug Reid, Dustin Tran, Dustin Zelle, Eric Noland, Erwin Huizenga, Eugene Kharitonov, Frederick Liu, Gagik Amirkhanyan, Glenn Cameron, Hadi Hashemi, Hanna Klimczak-Plucińska, Harman Singh, Harsh Mehta, Harshal Tushar Lehri, Hussein Hazimeh, Ian Ballantyne, Idan Szpektor, Ivan Nardini, Jean Pouget-Abadie, Jetha Chan, Joe Stanton, John Wieting, Jonathan Lai, Jordi Orbay, Joseph Fernandez, Josh Newlan, Ju-yeong Ji, Jyotinder Singh, Kat Black, Kathy Yu, Kevin Hui, Kiran Vodrahalli, Klaus Greff, Linhai Qiu, Marcella Valentine, Marina Coelho, Marvin Ritter, Matt Hoffman, Matthew Watson, Mayank Chaturvedi, Michael Moynihan, Min Ma, Nabila Babar, Natasha Noy, Nathan Byrd, Nick Roy, Nikola Momchev, Nilay Chauhan, Noveen Sachdeva, Oskar Bunyan, Pankil Botarda, Paul Caron, Paul Kishan Rubenstein, Phil Culliton, Philipp Schmid, Pier Giuseppe Sessa, Pingmei Xu, Piotr Stanczyk, Pouya Tafti, Rakesh Shivanna, Renjie Wu, Renke Pan, Reza Rokni, Rob Willoughby, Rohith Vallu, Ryan Mullins, Sammy Jerome, Sara Smoot, Sertan Girgin, Shariq Iqbal, Shashir Reddy, Shruti Sheth, Siim Põder, Sijal Bhatnagar, Sindhu Raghuram Panyam, Sivan Eiger, Susan Zhang, Tianqi Liu, Trevor Yacovone, Tyler Liechty, Uday Kalra, Utku Evci, Vedant Misra, Vincent Roseberry, Vlad Feinberg, Vlad Kolesnikov, Woohyun Han, Woosuk Kwon, Xi Chen, Yinlam Chow, Yuvein Zhu, Zichuan Wei, Zoltan Egyed, Victor Cotruta, Minh Giang, Phoebe Kirk, Anand Rao, Kat Black, Nabila Babar, Jessica Lo, Erica Moreira, Luiz Gustavo Martins, Omar Sanseviero, Lucas Gonzalez, Zach Gleicher, Tris Warkentin, Vahab Mirrokni, Evan Senter, Eli Collins, Joelle Barral, Zoubin Ghahramani, Raia Hadsell, Yossi Matias, D. Sculley, Slav Petrov, Noah Fiedel, Noam Shazeer, Oriol Vinyals, Jeff Dean, Demis Hassabis, Koray Kavukcuoglu, Clement Farabet, Elena Buchatskaya, Jean-Baptiste Alayrac, Rohan Anil, Dmitry, Lepikhin, Sebastian Borgeaud, Olivier Bachem, Armand Joulin, Alek Andreev, Cassidy Hardin, Robert Dadashi, Léonard Hussenot

• Scaling Rectified Flow Transformers for High-Resolution Image Synthesis

  Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, Dustin Podell, Tim Dockhorn, Zion English, Kyle Lacey, Alex Goodwin, Yannik Marek, Robin Rombach

• Denoising Diffusion Probabilistic Models

  Jonathan Ho, Ajay Jain, Pieter Abbeel

• V-VAE: A Variational Auto Encoding Framework Towards Fine-Grained Control over Human-Like Chat

  Qi Lin, Weikai Xu, Lisi Chen, Bin Dai

• Generative Adversarial Networks

  Ian J. Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, Yoshua Bengio

• Auto-Encoding Variational Bayes

  Diederik P. Kingma, Max Welling

• LoRA: Low-Rank Adaptation of Large Language Models

  Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen

• Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks

  Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, Sebastian Riedel, Douwe Kiela

• forejune.co/cuda/

  Unknown author

• Understanding Positional Encoding in Transformers - Blog by Kemal Erdem

  Unknown author

• Model Evaluation, Model Selection, and Algorithm Selection in Machine Learning

  Sebastian Raschka

• An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale

  Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, Neil Houlsby

• Papers with Code - Vision Transformer Explained

  Unknown author

• The State of LLM Reasoning Models

  Sebastian Raschka PhD

• In Search of an Understandable Consensus Algorithm

  Diego Ongaro, John Ousterhout

• Efficient and robust approximate nearest neighbor search using Hierarchical Navigable Small World graphs

  Yu A. Malkov, D. A. Yashunin