This page provides an overview of the development infrastructure that supports building, testing, and maintaining scikit-learn. It covers the build system architecture, dependency management with lock files, the CI/CD pipeline (GitHub Actions and Azure Pipelines), and the development workflow tools that ensure code quality and reproducibility.
For specific development setup instructions, see the Contributing guide doc/developers/contributing.rst For details about the build system configuration, see pyproject.toml and meson.build files throughout the codebase.
The development infrastructure consists of several interconnected systems that work together to support development, testing, and release processes.
Sources: doc/developers/contributing.rst1-150 pyproject.toml100-120 sklearn/_min_dependencies.py1-62
The build system uses meson-python as the build backend, replacing legacy setups. Cython is used to compile performance-critical code to C extensions.
Key Build Files:
meson-python, Cython, numpy, scipy).For details, see Build Environments.
Sources: pyproject.toml100-110 sklearn/_min_dependencies.py1-62
Scikit-learn maintains precise dependency configurations using lock files to ensure reproducible builds across platforms (linux-64, win-64, osx-64, linux-aarch64) and numerical backends (MKL, OpenBLAS).
The build_tools/update_environments_and_lock_files.py script generates lock files based on constraints defined in sklearn/_min_dependencies.py and default_package_constraints build_tools/update_environments_and_lock_files.py88-98
Lock File Examples:
build_tools/github/pylatest_conda_forge_mkl_linux-64_conda.lock: Latest dependencies with Intel MKL.build_tools/github/pymin_conda_forge_openblas_min_dependencies_linux-64_conda.lock: Testing against minimum supported versions.For details, see Installation and Dependency Management.
Sources: build_tools/update_environments_and_lock_files.py1-100 sklearn/_min_dependencies.py22-62
The CI/CD infrastructure is split between GitHub Actions (for unit tests, linting, and wheels) and Azure Pipelines.
The unit-tests.yml workflow manages the primary test matrix across Linux, macOS, and Windows .github/workflows/unit-tests.yml113-190
ruff, mypy, and cython-lint via build_tools/linting.sh .github/workflows/unit-tests.yml26-42[all random seeds] to run tests with exhaustive seeding .github/workflows/unit-tests.yml72-108For details, see CI/CD Pipeline.
Sources: .github/workflows/unit-tests.yml1-150 .github/workflows/wheels.yml1-173
The testing system uses pytest with parallel execution via pytest-xdist.
Key Features:
check_estimator validates that estimators follow scikit-learn conventions doc/developers/develop.rst88-100SKLEARN_SEED environment variable README.rst148-150thread_unsafe_fixtures to prevent race conditions during parallel execution pyproject.toml120-125For details, see Testing Infrastructure.
Sources: pyproject.toml112-132 README.rst140-150
The documentation is built using Sphinx and sphinx-gallery.
numpydoc for docstring formatting and sphinx-gallery for generating the example gallery doc/conf.py56-78build_tools/circle/build_doc.sh determines if a full build or "quick" build (skipping examples) is needed based on changed files and commit markers like [doc quick] build_tools/circle/build_doc.sh46-70For details, see Documentation System.
Sources: doc/conf.py1-100 build_tools/circle/build_doc.sh1-100
The project follows a standard GitHub fork-and-pull-request model with strict quality requirements.
For details, see Development Workflow & Contributing.
Sources: doc/developers/contributing.rst1-85 pyproject.toml134-160
Refresh this wiki