Scenario
Mode: Test Quality Review (Mode 4)
Risk: T2 Mock Overuse
Input
A test suite for a payment service where every external call (database, payment gateway, email sender) is mocked, including the database layer. Tests pass at 100% but the last three production incidents were caused by ORM query bugs that only appear against real data.
def test_charge_customer(mocker):
mocker.patch('payments.db.get_customer', return_value=fake_customer)
mocker.patch('payments.gateway.charge', return_value={'status': 'ok'})
mocker.patch('payments.email.send_receipt')
result = charge_customer(customer_id=1, amount=100)
assert result.success
# The actual SQL query that caused last month's outage is never executed
Expected finding
Brooks-lint should flag this as 🔴 T2 Mock Overuse:
- Symptom: database layer mocked out, ORM queries never tested against real schema
- Consequence: tests pass while production queries silently break
- Remedy: use a test database (SQLite in-memory or Docker) for repository layer tests
How to contribute
- Add this to
evals/evals.json as the next sequential entry
- See
CONTRIBUTING.md → "Add an eval test case" for the exact JSON format
- Open a PR
Scenario
Mode: Test Quality Review (Mode 4)
Risk: T2 Mock Overuse
Input
A test suite for a payment service where every external call (database, payment gateway, email sender) is mocked, including the database layer. Tests pass at 100% but the last three production incidents were caused by ORM query bugs that only appear against real data.
Expected finding
Brooks-lint should flag this as 🔴 T2 Mock Overuse:
How to contribute
evals/evals.jsonas the next sequential entryCONTRIBUTING.md→ "Add an eval test case" for the exact JSON format