Glossary Index

• Scaled Dot Product Attention
• Multi Head Attention
• Linear
• Rotary Positional Encoding
• Grouped Query Attention
• RMSNorm
• SiLU
• SwiGLU
• MLP

Your feedback is greatly appreciated. Welcome to join our Discord Community.
Found an issue? Create an issue / pull request on github.com/skyzh/tiny-llm.
tiny-llm-book © 2025 by Alex Chi Z is licensed under CC BY-NC-SA 4.0.