Your Linter Just Broke Production
Contract tests for the gaps between your linter and production
We ran ruff check --fix on a Python service. 71 unused imports removed. Tests passed. CI green. We nearly shipped a change that would have crashed every API endpoint in production.
The imports weren't unused. They were re-exports — names imported into one module so that other modules could lazily import them at runtime. The linter looked at each file in isolation and saw dead code. Production, which runs all the files together, saw load-bearing infrastructure.
Twelve names deleted. Every SSE connection, every health check, every task submission would have hit ImportError on the first request.
We caught it during code review. Then we asked: why did every safety net fail? And what would catch this automatically next time?
The answer was cheap: a handful of tests, about 50 lines total, that encode invisible contracts as executable specifications. They run in under two seconds, require zero mocks, and maintain themselves — when someone adds a new import, the tests discover it automatically.
The Gap
Here's what each tool in our pipeline could see:
None of them failed. They all did exactly what they were designed to do. The problem is that correctness here depends on a relationship between files that no single-file tool can see.
This is the shape of the bug:
The lazy import — from main import KEEPALIVE inside a function body — exists specifically to break a circular dependency. It's a common pattern in Python web frameworks: the main module imports routers at the top level, and routers import shared state from main lazily, inside functions, to avoid the cycle.
The linter sees the top-level import in main.py. It looks unused because nothing in that file references KEEPALIVE after importing it. The lazy import in router.py happens inside a function that the linter can't follow across files.
This isn't a Python-specific problem. The same class of invisible contract exists everywhere:
JavaScript/TypeScript: barrel files (
index.ts) re-export from internal modules. Tree-shaking or lint rules can remove "unused" re-exports that other packages depend on.Go: build-tag variants must define the same symbols. A cleanup in one tag file can break compilation under a different tag.
Rust: feature-gated code behind
#[cfg(feature = "...")]must maintain consistent public APIs across feature combinations.
The pattern is always the same: one module promises to expose certain names, and other modules depend on that promise, but no tool enforces the promise.
The AST in 60 Seconds
Python gives you a built-in way to read source code as structured data. The ast module parses any .py file into a tree — the Abstract Syntax Tree. You don't need to be a compiler engineer to use it. We'll use a handful of node types — ImportFrom, Import, Assign, FunctionDef, and a few more — but the core idea fits in ten lines.
Take this line of code:
from main import KEEPALIVE, _db_pool
Python's AST sees:
ImportFrom
├── module: "main"
├── level: 0 (absolute import)
└── names:
├── alias(name="KEEPALIVE")
└── alias(name="_db_pool")
The important part: ast.walk() visits every node in the tree, at every depth. So if an import is hidden inside a function — exactly where lazy imports live — the walker finds it:
Module
├── FunctionDef "get_constants"
│ └── ImportFrom ← ast.walk finds this
│ ├── module: "main"
│ └── names: [KEEPALIVE, SEEN_TTL, ...]
└── FunctionDef "handle_request"
└── ...
The core discovery loop is ten lines:
import ast
from pathlib import Path
for py_file in Path("src").rglob("*.py"):
tree = ast.parse(py_file.read_text())
for node in ast.walk(tree):
if isinstance(node, ast.ImportFrom) and node.module == "main":
for alias in node.names:
print(f"{py_file}:{node.lineno} imports {alias.name}")
This finds every from main import X in your codebase — top-level or buried inside functions, classes, or conditionals. It gives you the file path, the line number, and the exact name being imported. That's all we need to build a contract test.
Pattern 1: Re-export Contract Test
The first pattern guards the exact bug that started this story. One module imports names and re-exports them (explicitly or implicitly). Other modules depend on those names existing. A linter sees the re-exports as unused and wants to delete them.
The test:
"""Every lazy 'from main import X' must resolve at runtime."""
import ast
from pathlib import Path
import main as main_module
# Adjust parents[] index based on your test file's depth relative to project root
PROJECT_ROOT = Path(__file__).resolve().parents[2]
def _collect_lazy_imports():
"""Find every 'from main import X' across the codebase."""
results = []
for py_file in sorted((PROJECT_ROOT / "src").rglob("*.py")):
if py_file.name == "main.py":
continue
try:
tree = ast.parse(py_file.read_text())
except SyntaxError:
continue
for node in ast.walk(tree):
if isinstance(node, ast.ImportFrom) and node.module == "main":
for alias in node.names:
results.append((str(py_file), node.lineno, alias.name))
return results
def test_all_lazy_imports_resolve():
imports = _collect_lazy_imports()
assert imports, "Should find at least one lazy import"
missing = []
for path, lineno, name in imports:
if not hasattr(main_module, name):
missing.append(f" {path}:{lineno} -> {name}")
assert not missing, (
"Names imported from main but not defined:\n"
+ "\n".join(missing)
+ "\nAdd them back with a # noqa: F401 comment."
)
That's the complete test. Here's what makes it work well:
Self-maintaining. When someone adds a new from main import NEW_THING in any file, the AST walker discovers it automatically. No test case to update.
Clear failure messages. If KEEPALIVE is deleted from main.py, you see every dependent location:
Names imported from main but not defined:
src/routers/stream.py:35 -> KEEPALIVE
src/routers/health.py:28 -> KEEPALIVE
Add them back with a # noqa: F401 comment.
You know exactly which files depend on the name and where.
Cheap. No database, no network, no mocks. It parses files and checks hasattr. hasattr(main_module, name) checks the live module object — it sees everything the module defines or imports, which is exactly the set of names available to other modules at runtime. Total runtime: about 5 milliseconds.
Universal. This pattern works for any module that serves as a re-export hub:
Python
__init__.pyfiles that re-export from submoduleA
constants.pythat other modules import fromJavaScript
index.tsbarrel files: same concept, different syntax, same AST approach withts-morph
Python also has __all__ — the canonical way to declare a module's public API. Some linters respect __all__ and won't remove names listed in it. Contract tests complement __all__ by verifying the names actually exist on the module, not just that they're declared.
The # noqa: F401 comment (which tells ruff/pyflakes to suppress the "imported but unused" warning for that line) is the fix on the exporting side. But comments can be removed too. The test is the safety net for the safety net.
Pattern 2: Conditional Import Fallback Symmetry
Many Python libraries support optional dependencies with try/except at the module level:
try:
from some_sdk import Client, Config, ToolID
SDK_AVAILABLE = True
except ImportError:
SDK_AVAILABLE = False
ToolID = "default_tool_id"
def Client(*args, **kwargs):
raise RuntimeError("SDK not installed")
The contract: every code path must define the same public names. The rest of the module uses ToolID, Client, and SDK_AVAILABLE without checking which branch ran. If any name is missing from the fallback, you get a NameError — but only in environments where the SDK isn't installed. Which is usually production.
This is the "works on my machine" bug in its purest form. Your test suite runs with the SDK installed. CI runs with the SDK installed. The only place the fallback branch executes is production.
The test parses the try/except blocks and compares the names defined in each branch:
"""try/except branches must define the same public names."""
import ast
from pathlib import Path
MODULE_PATH = Path("src/tools/__init__.py")
def _names_defined_in(stmts):
"""Extract names assigned or defined in a block of statements."""
names = set()
for node in stmts:
if isinstance(node, ast.Assign):
for target in node.targets:
if isinstance(target, ast.Name):
names.add(target.id)
elif isinstance(node, ast.AnnAssign) and isinstance(node.target, ast.Name):
names.add(node.target.id)
elif isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef, ast.ClassDef)):
names.add(node.name)
elif isinstance(node, ast.Import):
for alias in node.names:
names.add(alias.asname or alias.name.split(".")[0])
elif isinstance(node, ast.ImportFrom):
for alias in node.names:
names.add(alias.asname or alias.name)
return names
def test_fallback_branches_define_same_names():
tree = ast.parse(MODULE_PATH.read_text())
for node in ast.walk(tree):
if not isinstance(node, ast.Try):
continue
def public(names):
return {n for n in names if not n.startswith("_")}
try_names = public(_names_defined_in(node.body))
for handler in node.handlers:
handler_names = public(_names_defined_in(handler.body))
# Filter flags only set in the try branch (e.g., AVAILABLE = True)
try_only = {n for n in try_names - handler_names
if not n.endswith("_AVAILABLE")}
handler_only = handler_names - try_names
assert not try_only, f"Names in try but missing from except: {try_only}"
assert not handler_only, f"Names in except but not in try: {handler_only}"
_names_defined_in covers the common ways a name gets introduced in a fallback block: ast.Assign, ast.AnnAssign, ast.FunctionDef, ast.AsyncFunctionDef, ast.ClassDef, ast.Import, and ast.ImportFrom. This is a heuristic, not a full proof of semantic equivalence — it won't catch names produced by exec(), globals() manipulation, or decorator-generated attributes. In practice, fallback blocks don't use those patterns, so the heuristic covers the real-world cases well.
This test auto-discovers every try/except block in the file. Add a new optional dependency block and it's covered. Add a new name to the try branch but forget the fallback, and you'll see:
Names in try but missing from except: {'Config'}
Where this pattern appears in the wild:
Django database backends (try to import the driver, fall back to a stub)
ML frameworks (try
import torch, fall back to CPU-only paths)Cloud SDKs (try
import boto3, fall back to local filesystem)Any plugin or adapter system with optional dependencies
Beyond AST: The Simplest Contract Test
This last pattern doesn't use the AST. It doesn't need to. It guards against the most catastrophic failure: a circular import that prevents the entire service from starting.
Python web apps commonly manage circular dependencies with lazy imports: module A imports module B at the top level, and B imports from A inside functions. The cycle is broken because the function-level import only runs when called, by which time both modules are fully loaded.
This convention works until someone refactors a lazy import to the top level:
The circular dependency was always there. The lazy import was the only thing preventing it from manifesting. And the lazy import looks like a code smell to anyone who doesn't know the history.
The test:
"""All modules must be importable without circular dependency errors."""
import importlib
import pytest
MODULES = [
"myapp.routers.health",
"myapp.routers.stream",
"myapp.routers.tasks",
"myapp.services.file_service",
"myapp.services.task_service",
"myapp.main",
]
@pytest.mark.parametrize("module_path", MODULES)
def test_no_circular_import(module_path):
importlib.import_module(module_path)
That's it. Import every module. If there's a circular dependency, Python raises ImportError and the test fails with the exact module that broke and why:
FAILED test_no_circular_imports[myapp.routers.stream]
ImportError: cannot import name 'db_pool' from partially
initialized module 'myapp.main' (most likely due to a
circular import)
The failure mode it prevents is total: not a degraded endpoint, not a wrong value, but a process that won't start.
One caveat: ensure your entry point guards startup logic with if __name__ == "__main__". If main.py calls uvicorn.run() at module level, importlib.import_module("myapp.main") will try to start the server. The test should import the module, not run it.
Unlike the previous two tests, the module list isn't auto-discovered. This is deliberate. When you add a new module, you should consciously decide whether it participates in the circular dependency graph. The test serves as both a guard and documentation of which modules are entangled.
When (Not) To Write These
Contract tests are for invisible contracts — agreements between modules that no existing tool enforces. Before writing one, ask:
Does the type system already enforce this? In some configurations — TypeScript with strict mode, for example — re-export contracts can be enforced at compile time. If tsc already catches it, you don't need an AST test.
Does a linter rule already exist? Some linters can be configured to recognize re-exports. Ruff supports # noqa: F401 and __all__ lists. If the linter knows about the contract, use the linter.
Do existing tests cover the path? If your integration tests actually hit the endpoints (not mocking the import layer), they'll catch missing re-exports. Contract tests are most valuable when unit tests mock too aggressively.
Write a contract test when:
The contract is invisible to single-file analysis
The failure is delayed (only surfaces at runtime, in specific environments)
The test is cheaper than the bug (50 lines of test vs. a production incident)
You'd otherwise have to tell teammates "don't touch that" — and hope they listen
Good contract tests share four properties:
Property Why it matters
───── ───────────────────────
Cheap <100ms — runs on every commit
Self-maintaining Discovers its own test cases via AST
Zero mocks Tests the real module, not a fake
Clear failures Names the exact file:line that broke
The Linter Isn't Wrong
The linter did exactly what it was designed to do. It found imports with no references in the current file and flagged them. That's correct single-file analysis.
The gap isn't a bug in the tool. It's a category of correctness that single-file analysis can't reach. Cross-module contracts live in the space between files — in the implicit promise that one module will expose a name that another module needs.
If that promise exists only in someone's head — in the tribal knowledge that says "don't delete those imports, the routers need them" — it will eventually break. Someone who doesn't know the history will clean up the code, the tests will pass, and production will fail.
If you'd tell a new teammate "don't touch that," write a test instead.
