Data Observability: What to Monitor and Where to Start
Data observability is the practice of watching the data itself, so that the team learns a table is late, short, reshaped or wrong before its users do. Five things are worth watching. A small team should start with two of them, on the tables that matter most.
A pipeline can finish successfully and deliver wrong data. A job that loads a tenth of its expected rows reports success, and so does one that loads yesterday’s file twice. Monitoring of the machinery sees nothing amiss in either case, and the first person to notice is whoever reads the report.
Data observability exists to close that gap. This guide explains what it is, what it adds to the monitoring and testing a team already has, the five things to monitor, how to avoid drowning in alerts and where to begin. The term was popularised by software vendors, and that is acknowledged where their definitions are used. The practices can be adopted with or without a specialist product.
What observability means for data
The word comes from software operations. The OpenTelemetry project, a vendor-neutral open-source standard, defines it this way: “Observability lets you understand a system from the outside by letting you ask questions about that system without knowing its inner workings”1.
Applied to data, the system being observed is the data in its tables and files, as distinct from the machinery that moves it. Monte Carlo, a vendor of observability software, says it created the category in 2019, and it frames the problem as data downtime: “periods of time when data is partial, erroneous, missing, or otherwise inaccurate”2. The definition comes from a company that sells a product in the category, and it should be read with that in mind. The phenomenon it names is real and does not depend on the product.
It helps to keep two things apart in what follows. The definition of the category and the list of pillars come from a vendor, and are used here as a convenient map of the ground. The operational recommendations do not come from that vendor. The two-threshold approach to freshness is documented by an open-source tool. The guidance on alerting is from Google’s published engineering practice, which was written about software systems and is applied to data here on DII’s judgement. The order in which to start is DII’s own recommendation. A reader can accept the practices and set the vendor’s framing aside.
What it adds to what you already have
Two existing practices might be thought to cover the ground. Each covers part.
Infrastructure monitoring watches the machinery: whether jobs ran, how long they took, whether servers are healthy. It is necessary and it cannot see a partial load. A job that loads a tenth of the expected rows succeeds.
Data testing checks stated expectations at defined points: this column is never null, this key is unique. Tests catch the problems someone thought of in advance. They do not catch the ones nobody anticipated, and they say nothing about data that never arrived to be tested. Where to place tests, and which should block a pipeline, is the subject of where to put data quality checks in a pipeline.
Observability fills the gap between them by watching the properties of the data continuously and comparing them with what is normal. The three are complementary. A team needs all of them in some measure, and a mature set of tests reduces what observability has to catch.
Five things to monitor
Monte Carlo’s framing lists five pillars: freshness, quality, volume, schema and lineage2. The list below follows the same ground, with the quality pillar expressed as distribution, a term that describes the check more exactly. Lineage differs in kind from the other four: it is the context that makes the others useful.
Five things to monitor in data, and what each catches
| Monitor | The question | A problem it catches |
|---|---|---|
| Freshness | Did the data arrive when expected? | A source that stopped sending; a job that silently did nothing |
| Volume | Did about the right amount arrive? | A partial load; a duplicated load |
| Schema | Has the structure changed? | A renamed, removed or retyped column |
| Distribution | Do the values look as they normally do? | A spike in nulls; a unit change; a default value flooding a field |
| Lineage | What feeds this, and what does it feed? | Not a detector: it tells you what an incident affects and where it began |
Freshness
Freshness asks whether data is as recent as it should be. Monte Carlo’s description is serviceable: “Freshness seeks to understand how up-to-date your data tables are, as well as the cadence at which your tables are updated”2.
It is the most valuable single check and the easiest to implement. Open-source tooling supports it directly. dbt’s documentation describes source freshness as “a critical component of defining Service Level Agreements (SLAs) for your warehouse”3, and it lets a team declare two thresholds for each source, one that warns and one that fails. The two-threshold idea is worth copying whatever the tool: a point at which someone should look, and a later point at which downstream work should stop.
Volume
Volume asks whether roughly the expected number of rows arrived. A load of zero rows is obvious once checked. A load of a tenth, or of double, is the case that passes unnoticed, and a simple comparison with the same day in previous weeks will catch it.
Schema
Schema monitoring detects changes to structure: columns added, removed, renamed or changed in type. The typical case is an upstream team making a reasonable change without knowing who depended on the old shape. Detection tells you it happened. Preventing it is a matter of agreement between the producing and consuming teams, which is a separate practice.
Distribution
Distribution asks whether the values themselves look normal: the proportion of nulls, the range of a numeric field, the set of distinct values in a category. This is where subtle errors show. A currency field that switches from pounds to pence does not change the row count or the schema.
Distribution checks are also prone to false alarms, because real data varies. They should be applied narrowly, to fields that matter, after the first three are in place.
Lineage
Lineage records what each table is built from and what is built from it. It detects nothing. Its value is in the minutes after an alert, when it answers two questions: what upstream change caused this, and which downstream reports are now wrong. Open standards exist for collecting lineage from running pipelines, OpenLineage being one4. Lineage gathered automatically is more likely to be current than a diagram maintained by hand.
Alert fatigue is the main way this fails
An observability effort is most exposed to one failure: too many alerts. Monitoring is switched on across every table, the channel fills with notifications, and within weeks nobody reads them. The team is then worse off than before, because it believes it is covered.
Google’s Site Reliability Engineering book, writing about systems monitoring, describes the mechanism: “When pages occur too frequently, employees second-guess, skim, or even ignore incoming alerts”5. It sets a standard for anything that interrupts a person: “Every time the pager goes off, I should be able to react with a sense of urgency”5. And it offers a test for each rule: “Does this rule detect an otherwise undetected condition that is urgent, actionable, and actively or imminently user-visible?”5.
Translated for data, three rules follow. They are DII’s application of that guidance.
- Alert only where someone will act. If nobody would do anything about a late table, record it and do not notify.
- Give every alert an owner and a first step. An alert that arrives in a shared channel with no named responder belongs to nobody.
- Review alerts regularly and delete the ones that were ignored. An alert that fired and prompted no action is evidence that it should not exist in that form.
The same book’s requirement that alert rules “should be simple to understand and represent a clear failure”5 argues for starting with plain thresholds on freshness and volume before adopting statistical anomaly detection, whose alerts are harder to explain.
Where to start
A team new to this does not need a platform to begin. A workable sequence follows.
- Choose the tables. List the ten or so on which important reports and decisions rest. If the organisation has identified its critical data, use that list; the approach is set out in starting with critical data elements.
- Add freshness and volume checks to those tables. Use whatever the existing stack provides. A scheduled query that compares the latest load time and row count with expectations is enough.
- Decide who is told and what they do. Write it down beside the check.
- Run for a month and tune. Remove alerts nobody acted on. Tighten those that fired too late.
- Add schema monitoring on the sources that change.
- Add distribution checks sparingly, on the specific fields where a silent error would do damage.
- Only then consider a dedicated tool, with a month or more of experience to say what it must do.
Building in this order means each step is justified by the previous one. It also means the team learns what a useful alert looks like while the number of alerts is small.
How it connects to the rest of the practice
Observability detects. It does not prevent, and it does not resolve. Prevention is the design of the pipeline, covered in what makes a data pipeline reliable. Resolution needs a practised response once an alert is confirmed as a real problem, covered in handling data incidents.
It also serves the people at the far end. A dashboard that is trusted on the days it is right and unannounced on the days it is wrong loses its audience, which is one of the causes examined in why dashboards go unused, and what to check before rebuilding. Telling users that data is late before they discover it is one of the cheapest ways to keep their confidence.
The Data Observability Checklist sets out the five monitors with the questions to answer for each table, and the Data Pipeline Reliability Checklist covers the design properties that reduce how much there is to detect.
Sources
- OpenTelemetry Authors (Cloud Native Computing Foundation project). Observability primer (OpenTelemetry documentation).
- Monte Carlo. What Is Data Observability? 5 Key Pillars To Know In 2026.
- dbt Labs. Add sources to your DAG (dbt Developer Hub).
- OpenLineage (LF AI & Data Foundation project). OpenLineage documentation: About OpenLineage and Object Model.
- Ewaschuk, R. (Google; Site Reliability Engineering, edited by Beyer, B. et al.). Monitoring Distributed Systems (chapter 6 of Site Reliability Engineering). 2017.
This article is general professional information. It is not legal or regulatory advice.