Observability before dashboards

Teams often buy an observability suite and stop after shipping default dashboards. The missing step is agreeing which user journeys matter and what “good” looks like numerically. Without that agreement, every chart becomes debatable and every incident turns into an argument about which graph is “real”.

We start with SLOs on critical paths, then allocate trace sampling and log budgets to those paths. Everything else stays intentionally thin to control cost. This makes the system easier to reason about because you’re collecting signals to answer specific questions, not to satisfy curiosity.

This approach makes incidents shorter because on-call knows which signals are authoritative — and stakeholders see reliability as a product feature, not an ops side quest. It also reduces “alert fatigue” because you’re paging on outcomes (journeys) instead of noisy subsystem-level fluctuations.

A practical starting point is a small “SLO map”: pick the 3–5 journeys that matter (signup, checkout, sync, report generation), then define what “good” is numerically. Tie each journey to a user-facing promise (speed, availability, correctness) so the numbers have meaning. Everything else becomes secondary by design.

Then allocate budgets: sampling rates, log volume, and retention. If a signal doesn’t help you make a decision during an incident, it probably doesn’t deserve full-fidelity collection. When you do need detail, prefer targeted, temporary boosts (for a canary, a single tenant, a specific endpoint) rather than collecting high volume forever.

The payoff is compound: lower spend, clearer dashboards, and faster incident response because everyone knows what the source of truth is. Over time you also get better prioritization: work that improves SLOs becomes obviously valuable, and work that doesn’t can be safely deprioritized.