Overview

Relevant source files

• .gitignore
• CHANGELOG.md
• COPYING
• LICENSE
• MANIFEST.in
• README.md
• _changelog_build.py
• cli.py
• pyproject.toml
• studio/backend/cloudflare_tunnel.py
• studio/backend/main.py
• studio/backend/run.py
• studio/backend/tests/test_cloudflare_tunnel.py
• studio/backend/tests/test_export_log_cursor.py
• studio/backend/tests/test_process_lifetime.py
• studio/backend/tests/test_remote_access_settings.py
• studio/backend/tests/test_secure_tunnel_gate.py
• studio/backend/tests/test_training_preflight.py
• studio/backend/utils/changelog.py
• studio/backend/utils/process_lifetime.py
• studio/frontend/src/features/studio/history-card-grid.tsx
• studio/frontend/src/features/studio/live-training-view.tsx
• tests/python/test_construct_chat_template_validation.py
• tests/test_cli_export_unpacking.py
• unsloth/__init__.py
• unsloth/chat_templates.py
• unsloth/models/__init__.py
• unsloth/models/_utils.py
• unsloth/models/dpo.py
• unsloth/models/llama.py
• unsloth/models/loader.py
• unsloth/models/mapper.py
• unsloth/models/mistral.py
• unsloth/models/rl.py
• unsloth/models/rl_replacements.py
• unsloth/models/vision.py
• unsloth/save.py
• unsloth/tokenizer_utils.py
• unsloth_cli/commands/__init__.py
• unsloth_cli/commands/export.py
• unsloth_cli/commands/studio.py
• unsloth_cli/commands/train.py
• unsloth_cli/tests/conftest.py
• unsloth_cli/tests/test_studio_cloudflare_flag.py
• unsloth_cli/tests/test_studio_run_parallel_flag.py
• unsloth_cli/tests/test_studio_secure_flag.py

This document provides a high-level introduction to Unsloth, explaining its purpose, architectural components, licensing model, and how the different parts of the system interact. Unsloth is a library for accelerated fine-tuning and inference of large language models (LLMs), providing 2-5x faster training and 70% lower VRAM usage compared to standard approaches README.md8-56

For detailed information about specific subsystems:

• For installation instructions, see Getting Started (Section 2)
• For model loading internals, see Model Loading Pipeline (Section 3.2)
• For training workflows, see Standard Fine-tuning (Section 6.1)
• For Studio backend details, see Backend Architecture Overview (Section 9.1)

---

System Purpose and Scope

Unsloth accelerates LLM training and inference through custom Triton kernels, model patching, and optimized attention implementations README.md50-53 The system consists of:

1. Core Library (Apache 2.0): The ML optimization engine providing model patching, custom kernels, and training utilities unsloth/models/_utils.py1-13
2. Studio Backend (AGPLv3): A FastAPI-based web service with subprocess orchestration for training, inference, and export README.md9-20
3. Studio Frontend & CLI (AGPLv3): User-facing interfaces for interacting with the system pyproject.toml34-35

The core library can be used as a standalone Python package, while Studio and CLI provide production-ready tools for LLM workflows including Reinforcement Learning (RL), vision fine-tuning, and GGUF export README.md38-56

Sources: pyproject.toml5-12 README.md22-41 unsloth/models/_utils.py1-13

---

Dual Licensing Model

Unsloth employs a dual licensing strategy that separates the ML optimization core from user-facing tools:

Component	License	Location	Purpose
Core Library	Apache 2.0	unsloth/	Embeddable optimization engine for models, kernels, and LoRA unsloth/models/_utils.py1-13
Studio Backend	AGPLv3	studio/	Web service for training/inference/export with subprocess isolation pyproject.toml43-59
Studio Frontend	AGPLv3	studio/frontend/	React/TypeScript UI for model configuration, data recipes, and chat pyproject.toml48-54
CLI	AGPLv3	unsloth_cli/	Command-line interface for training, inference, and launching the studio pyproject.toml34-35

The Apache 2.0 core allows commercial integration without source disclosure requirements, while the AGPLv3 Studio components ensure that network services built on top must share source code modifications.

Sources: pyproject.toml11 unsloth/models/_utils.py1-13 unsloth/models/llama.py1-13 unsloth/models/vision.py1-13

---

Core Architecture Components

Component Hierarchy

The following diagram illustrates the relationship between the AGPLv3 user interfaces and the Apache 2.0 core optimization library.

Diagram: High-Level Component Architecture

Sources: unsloth/__init__.py95-121 unsloth/models/loader.py29-35 unsloth/models/vision.py107-109 pyproject.toml34-35

---

Core Library (Apache 2.0)

The core library provides model optimization through three primary mechanisms:

1. Model Loaders

Loaders serve as the primary entry points for users. They handle model detection, quantization routing, and the application of patches.

Class	File	Purpose
FastLanguageModel	unsloth/models/loader.py30	Entry point for text LLMs (Llama, Mistral, Qwen, etc.).
FastVisionModel	unsloth/models/vision.py84	Entry point for vision-language models (VLMs).
FastBaseModel	unsloth/models/vision.py108	Base class providing logic shared across vision models.

Sources: unsloth/models/loader.py29-35 unsloth/models/vision.py107-109

2. Patching System

Unsloth modifies transformers, peft, and bitsandbytes at load time to inject optimizations. It also handles hardware-specific initialization, such as detecting Apple Silicon (MLX) via _is_mlx_available unsloth/__init__.py62-81

Diagram: Model Loading and Patching Flow

The system performs critical checks to ensure proper precision. For example, it uses _NEEDS_ROPE_FIX to repair Rotary Embedding corruption caused by transformers v5 meta-device loading unsloth/models/loader.py91

Sources: unsloth/__init__.py83-107 unsloth/models/_utils.py132-138 unsloth/models/loader.py91 unsloth/models/llama.py87-92

3. Custom Kernels

Triton-based kernels replace standard PyTorch operations for critical bottlenecks.

• Cross Entropy: unsloth_fused_ce_loss replaces standard loss for better memory efficiency unsloth/models/_utils.py155
• Gradient Checkpointing: patch_unsloth_gradient_checkpointing provides memory-efficient backpropagation unsloth/models/_utils.py142
• Dequantization: fast_dequantize allows for faster processing of 4-bit weights during merging and export unsloth/save.py49

Sources: unsloth/models/_utils.py142-155 unsloth/save.py49

---

Studio Backend (AGPLv3)

Studio isolates heavy ML operations (training, inference, export) in dedicated subprocesses. This ensures proper environment isolation, specifically for models requiring specific transformers versions or complex dependencies such as transformers 4.x vs 5.x unsloth/models/loader.py76-88

Subprocess Orchestration Pattern

Each major task (Training, Inference, Export) is handled by an Orchestrator that spawns a worker process.

Key Backend Components:

• Training Orchestration: Manages UnslothTrainer and streams progress events to the UI.
• Inference Backends: Supports both Transformers-based and GGUF-based inference via llama.cpp integration managed in unsloth/save.py unsloth/save.py18-25
• Export Backend: Handles conversion to GGUF using save_to_gguf unsloth/save.py75

Sources: unsloth/models/loader.py76-88 unsloth/save.py18-25 unsloth/save.py75

---

Model Support and Quantization Routing

Unsloth maintains registries to redirect model names to optimized variants and handle hardware-specific requirements.

Mechanism	Entity	Purpose
Name Mapping	INT_TO_FLOAT_MAPPER	Redirects 4-bit quantized model names to their canonical parents unsloth/models/mapper.py23-24
Precision Control	FORCE_FLOAT32	Forces FP32 for models like gemma3 or qwen3_5 to prevent numerical instability unsloth/models/loader.py120-137
Attention Selection	DISABLE_SDPA_MODEL_NAMES	Disables Scaled Dot Product Attention for specific modules that are incompatible unsloth/models/loader.py27
Version Routing	SUPPORTS_LLAMA31	Conditional logic to handle version-specific features in transformers unsloth/models/loader.py80

Sources: unsloth/models/mapper.py23-24 unsloth/models/loader.py120-137 unsloth/models/loader.py27 unsloth/models/loader.py80

---

Summary of Workflows

Direct Core Library Usage

Users interact with FastLanguageModel or FastVisionModel to load and prepare models for training or inference.

1. Load: FastLanguageModel.from_pretrained patches the architecture at load time unsloth/models/loader.py30
2. Configure PEFT: get_peft_model adds LoRA adapters with optimized target modules unsloth/models/llama.py113
3. Inference: Models are placed in inference mode via FastLanguageModel.for_inference(model) which disables gradients and optimizes the KV cache unsloth/models/rl.py203
4. Reinforcement Learning: PatchRL modifies trl trainers to use Unsloth's optimized generation and memory management unsloth/models/rl.py149

Model Persistence

The library provides specialized functions for saving and exporting models:

• unsloth_save_model: Handles saving LoRA adapters or merged models unsloth/save.py74
• save_to_gguf: Integrates with llama.cpp for GGUF conversion and quantization unsloth/save.py75
• patch_saving_functions: Modifies the base model's save_pretrained to include Unsloth-specific metadata unsloth/save.py76

Sources: unsloth/models/loader.py30 unsloth/models/llama.py113 unsloth/models/rl.py149-203 unsloth/save.py72-78