Debugging workflow
Reproduce, isolate, inspect logs, form hypotheses, fix, and verify.
Debugging is disciplined evidence gathering. The goal is not to guess faster; it is to make the bug small, observable, and impossible to misunderstand.
The big idea
A reliable debugging loop is boring on purpose:
When the loop is clear, even hard bugs feel tractable.
Start with reproduction
Write the smallest command, request, or test that fails every time.
curl -i http://localhost:8000/api/orders/42
pytest tests/test_orders.py::test_missing_order_returns_404 -q
npm test -- --runInBand order-detailIf the bug only happens sometimes, your first task is to collect the conditions: input, user, time, version, environment, concurrency, and external dependencies.
Trace the request flow
Do not debug "the app" as one blob. Walk the path:
browser, curl, mobile app
nginx, CDN, gateway
method, path, auth, validation
business rule and branching
query, transaction, job state
timeout, retry, contract drift
At each layer, ask: what did it receive, what did it decide, and what did it send next?
Make hypotheses falsifiable
Weak hypothesis: "Auth is broken."
Strong hypothesis: "The route rejects this request because the Authorization header is
missing after the nginx proxy forwards it."
The strong version tells you what to inspect:
curl -v http://localhost:8000/api/me -H "Authorization: Bearer test"
docker compose logs api | rg "request_id|Authorization|401"Read evidence, not vibes
- Logs
- Best for exact events: request IDs, user IDs, branches taken, errors.
- Metrics
- Best for shape: latency, error rate, queue depth, saturation.
- Traces
- Best for multi-service latency and where time disappeared.
- Database state
- Best for whether the system stored what you think it stored.
Add temporary instrumentation only when needed, and remove noisy debug prints before the fix lands.
Verify the fix
A fix is not done when the error disappears once. It is done when:
- The original failing reproduction now passes.
- At least one adjacent failure case is checked.
- The right layer owns the fix.
- The behavior is covered by a test or a saved smoke command.
In practice
Take a past production bug and rewrite it as a debugging trace: request, layer, evidence, root cause, patch, verification. If the trace has gaps, that is exactly where your future logs or tests should improve.
Key takeaways
- First make the bug reproducible; then make it small.
- Trace the full flow instead of guessing which layer is broken.
- A useful hypothesis predicts what evidence would prove it wrong.
- Use logs, metrics, traces, and database state for different questions.
- Verification means rerunning the original repro and checking nearby failure cases.
Checkpoint questions
Use these to test whether the lesson is clear enough to explain without rereading.
- 1Can you reproduce the bug with the smallest reliable command or request?
- 2What layer owns the failure: client, route, service, database, queue, or infrastructure?
- 3Which log line, metric, trace, or curl output would falsify your current hypothesis?
- 4How do you prove the fix worked and did not only hide the symptom?
References
External resources for going deeper after the lesson above.