This project follows the book "Large Language Models from Scratch" by Sebastian Raschka, which provides a comprehensive guide to understanding and implementing large language models from the ground up.
"Large Language Models from Scratch" is a practical guide that helps readers understand the inner workings of large language models (LLMs) by building them from scratch. The book covers essential concepts, architectures, and implementation details of modern LLMs.
This project uses uv for dependency management, which is a fast Python package installer and resolver.
- Python 3.8 or higher
uvpackage installer
- First, install
uvif you haven't already:
curl -LsSf https://astral.sh/uv/install.sh | sh- Create and activate a virtual environment:
uv venv
source .venv/bin/activate # On Unix/macOS
# or
.venv\Scripts\activate # On Windows- Install dependencies:
uv pip install .The project follows the structure and implementations as described in the book, with each chapter focusing on different aspects of LLM development.
This project is for educational purposes and follows the book's content. Please refer to the book's license for more information.