Python Project Best Practices¶

Use this guide when creating sibling repositories that should feel like this project: a Python-first package with clear package boundaries, typed public APIs, focused documentation, and repeatable validation commands.

Replace project_name with the distribution name and package_name with the import package. They are often the same, but keep the distinction explicit.

Repository Shape¶

Prefer this top-level layout for new Python package repositories:

project-name/
  README.md
  CONTRIBUTING.md
  LICENSE
  pyproject.toml
  setup.py                  # only when build-time customization is needed
  tox.ini
  docs/
    conf.py
    index.md
    development.md
    architecture.md
    api.md
  examples/
    01-basics/
      README.md
      00-package-tour.py
  tests/
    conftest.py
    core/
    workflows/
  package_name/
    __init__.py
    version.py
    py.typed
    core/
    workflows/

This repository uses a flat package layout: the import package lives directly under the repository root rather than under src/. Keep that style for sibling projects when the goal is consistency with this project.

Keep local-only or generated directories out of the source contract:

virtual environments and local package builds;
__pycache__, .pytest_cache, .mypy_cache, Ruff cache, and docs builds;
generated notebooks, checkpoints, prediction outputs, cache arrays, and large experiment artifacts;
scratch directories, backup directories, and one-off planning notes.

Packaging¶

Use pyproject.toml as the source of packaging metadata.

Recommended baseline:

[project]
name = "project_name"
dynamic = ["version"]
requires-python = ">=3.10"
authors = [
    {name = "Project Developers"},
]
dependencies = [
]
readme = "README.md"
license = {file = "LICENSE"}
description = "Short project description"

[project.optional-dependencies]
docs = [
    "myst-parser>=2",
    "sphinx>=7.2",
]
dev = [
    "mypy>=1.13",
    "pytest>=8",
    "ruff>=0.6",
    "tox>=4.39",
]

[build-system]
requires = [
    "setuptools>=77",
]
build-backend = "setuptools.build_meta"

[tool.setuptools.packages.find]
include = ["package_name*"]
namespaces = false

[tool.setuptools.package-data]
package_name = ["py.typed"]

[tool.setuptools.dynamic]
version = {attr = "package_name.version.__version__"}

Keep required dependencies small. Put optional integrations behind extras such as docs, dev, backend names, and all. Base imports should work without optional engines, plotting stacks, native extensions, or development tools.

Use setup.py only when setuptools needs executable build logic, such as optional C++ or CUDA extensions. Keep it thin and make native builds opt-in with environment variables. A pure-Python package usually does not need setup.py.

Put the package version in package_name/version.py:

"""Project version."""

__version__ = "0.1.0"

Add package_name/py.typed for PEP 561 typing support.

Development Toolkit¶

Use tox as the command surface for common checks:

[tox]
requires =
    tox >=4.39

envlist =
    lint
    type
    docs
    docs-tests
    tests

[testenv]
deps =
    pytest
commands =
    python -m pytest {posargs}

[testenv:lint]
package = skip
deps =
    ruff
commands =
    ruff format --diff package_name tests examples setup.py
    ruff check package_name tests examples setup.py

[testenv:format]
package = skip
deps =
    ruff
commands =
    ruff format package_name tests examples setup.py
    ruff check --fix-only package_name tests examples setup.py

[testenv:type]
deps =
    mypy
commands =
    mypy package_name {posargs}

[testenv:docs-tests]
deps =
    pytest
commands =
    pytest --doctest-modules package_name {posargs}

[testenv:docs]
extras =
    docs
commands =
    sphinx-build -W --keep-going -b html docs docs/_build/html {posargs}

Install locally from the repository root:

python -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade pip
python -m pip install -e ".[dev,docs]"

Run these commands before review:

tox -e lint
tox -e type
tox -e docs-tests
tox -e docs
tox -e tests

Use direct pytest commands for focused iteration:

python -m pytest tests/core
python -m pytest tests/workflows/test_example.py

Ruff And Mypy¶

Use Ruff for formatting, linting, and import sorting:

[tool.ruff.lint]
select = ["E", "F", "B", "I"]

[tool.ruff.lint.isort]
lines-after-imports = 2
known-first-party = ["package_name"]

[tool.ruff.format]
docstring-code-format = true

Use mypy for the package, with a strict-but-gradual baseline:

[tool.mypy]
python_version = "3.10"
files = ["package_name"]
check_untyped_defs = true
no_implicit_optional = true
pretty = true
show_error_codes = true
strict_equality = true
warn_redundant_casts = true
warn_unused_configs = true
warn_unused_ignores = true

Prefer precise types for public APIs, dataclasses, protocol-like contracts, and configuration objects. If a module must be baselined with ignore_errors, keep that override narrow and documented. Do not use broad type ignores to hide unclear ownership or unstable interfaces.

Source Organization¶

Organize by responsibility, not by implementation convenience. A small project does not need every directory below, but the dependency direction should remain clear.

package_name/
  core/          # normalized inputs, outputs, validation, shared primitives
  terms/         # domain units or model components
  workflows/     # user-facing orchestration across lower layers
  adapters/      # optional external integrations
  analysis/      # diagnostics and plotting helpers
  benchmarks/    # performance smoke tests and expert entry points

Recommended dependency direction:

core
  -> terms or domain modules
  -> training, fitting, or workflows
  -> adapters, analysis, examples, benchmarks

Keep the lower layers free of optional engine objects and UI concerns. Adapters should translate external types into normalized internal types at the boundary. Workflows may compose lower layers for examples or common tasks, but they should not own kernels, file formats, cache layouts, or core validation rules.

Use underscore-prefixed modules for implementation details:

package_name/terms/_shared.py
package_name/terms/_parameters.py

Expose stable APIs through package __init__.py files and documentation. Keep __init__.py thin. If imports are expensive or require optional dependencies, use lazy exports rather than importing heavy modules at package import time.

Public API Conventions¶

Follow these naming and API rules:

classes use PascalCase;
functions, methods, modules, and files use snake_case;
constants use UPPER_SNAKE_CASE;
public modules avoid surprising import-time work;
public functions validate arguments early and raise clear ValueError, TypeError, or domain-specific exceptions;
deprecations use warnings.warn(..., DeprecationWarning, stacklevel=2);
expensive optional integrations stay behind extras and adapter modules.

Prefer small documented public surfaces. A helper is not public just because it is imported by another internal module. If users should rely on it, export it intentionally, add tests, and mention it in API docs.

Documentation¶

Use Markdown documentation with Sphinx and MyST.

Recommended docs pages:

docs/index.md: navigation and project overview;
docs/getting-started.md: installation and first working example;
docs/concepts.md: user-facing mental model;
docs/architecture.md: package boundaries and dependency direction;
docs/development.md: setup, validation, and generated artifacts;
docs/api.md: documented public API.

Recommended docs/conf.py choices:

extensions = [
    "myst_parser",
    "sphinx.ext.autodoc",
    "sphinx.ext.napoleon",
    "sphinx.ext.viewcode",
]

source_suffix = {
    ".md": "markdown",
}

napoleon_google_docstring = True
napoleon_numpy_docstring = False
autodoc_typehints = "description"

Use Google-style docstrings for Python APIs:

def load_record(path: str) -> Record:
    """Load a record from disk.

    Args:
        path: File path to read.

    Returns:
        Parsed record.

    Raises:
        ValueError: If the file contents are invalid.
    """

Keep docs factual and close to implementation. When behavior depends on an optional backend, platform, or performance benchmark, state the scope and the fallback behavior.

Examples¶

Examples should be runnable and should double as lightweight integration coverage.

Use numbered example folders for guided learning:

examples/
  01-basics/
  02-domain-case/
  03-advanced-workflow/

Keep tracked .py files as the source of truth. Generated notebooks can be useful local presentation artifacts, but they should not become the canonical implementation unless the project is intentionally notebook-first.

Track only small, stable input data that is required for examples or tests. Generated models, predictions, plots, caches, and checkpoints should normally be ignored.

Testing¶

Mirror package areas under tests/:

tests/
  core/
  terms/
  workflows/
  adapters/
  analysis/
  speed/

Use pytest configuration in pyproject.toml:

[tool.pytest.ini_options]
testpaths = [
    "tests/core",
    "tests/workflows",
]
filterwarnings = [
    "error",
    "ignore::DeprecationWarning",
    "error::DeprecationWarning:package_name.*",
]

Testing expectations:

core data structures need shape, dtype, validation, and error-path tests;
public APIs need behavior tests and import-surface tests;
workflows need small end-to-end tests with stable fixtures;
adapters need optional dependency skips and boundary-conversion tests;
performance-sensitive paths need targeted speed gates or benchmark smoke tests;
bug fixes should include a regression test unless the behavior is already covered by a higher-level test.

Use warnings-as-errors for package deprecations. This keeps accidental warning regressions from becoming accepted behavior.

Optional Native Code¶

Keep native extensions optional. A sibling project should import and run in pure Python unless the user explicitly builds native code.

Recommended pattern:

place sources under package_name/csrc/;
use environment variables such as PROJECT_BUILD_NATIVE=1;
fail clearly when a requested compiler, CUDA toolkit, or build dependency is unavailable;
choose automatic runtime fallback by default;
provide a strict runtime mode only when users need native code to be required.

Do not let native extension availability change public API semantics. It should only affect performance or explicitly documented backend behavior.

Review Checklist¶

Use this checklist before opening a change:

package imports cleanly from a fresh editable install;
README.md explains the project, install command, and minimal example;
pyproject.toml has accurate dependencies, extras, package discovery, and tool configuration;
public APIs are typed, documented, exported intentionally, and tested;
optional dependencies are isolated behind extras and adapter modules;
generated files are ignored unless they are small, stable source artifacts;
tox -e lint, tox -e type, tox -e docs-tests, tox -e docs, and tox -e tests pass or have documented, scoped exceptions;
architecture docs match the actual dependency direction.

Bootstrap Checklist For A Sibling Repository¶

Create the repository with README.md, LICENSE, pyproject.toml, tox.ini, docs/, examples/, tests/, and package_name/.
Add package_name/version.py, package_name/py.typed, and a thin package_name/__init__.py.
Configure setuptools package discovery and dynamic versioning.
Configure Ruff, mypy, pytest, and tox.
Add a minimal docs build with MyST Markdown and Google-style docstrings.
Add one import test, one behavior test, and one runnable example.
Install with python -m pip install -e ".[dev,docs]".
Run the tox validation commands and fix failures before adding advanced features.