Bronze, silver and gold are responsibilities, not folders.

What each layer owes

What each layer owes the one after it

Bronze owes replayability: the data exactly as it arrived, so any downstream mistake can be corrected without going back to the source. Silver owes truth at the record level: one deduplicated, conformed, tested row per entity. Gold owes meaning: business measures defined once, shaped for a reader, with a named owner. A layer that owes nothing the previous one did not is a copy, not a layer.

Databricks frames the layers the same way, describing bronze as maintaining the raw state of the source in its original format, silver as including at least one validated non-aggregated representation of each record, and gold as aggregated data tailored to analytics and reporting. Its own documentation is careful about the status of the pattern: following medallion is a recommended best practice, not a requirement.

That sentence deserves more weight than it usually gets. The pattern is a default, and defaults are valuable precisely because you can depart from them deliberately. Treating it as a rule produces the characteristic estate where every dataset passes through three tables regardless of whether anything happens to it in two of them.

Why bronze exists: replay, not archaeology

Bronze is not an archive kept out of caution. It exists so that a transformation error discovered three weeks later can be corrected from data you already hold, rather than by asking the source system to reproduce a month of history it may no longer have. That single property is what justifies the storage, and it is the reason bronze must carry no business logic.

The rule follows directly: the moment a filter, a type coercion or a business decision is applied on the way into bronze, replay stops being lossless. If invalid rows are dropped before landing, no later fix can recover them. If a timestamp is converted to a single time zone on ingest, the original offset is gone. Bronze should append, record when each record arrived, and decide nothing.

The corollary is that bronze needs a retention policy stated in terms of recovery rather than compliance. The right question is how far back a correction might plausibly need to reach — usually a reporting period or two — not how long the storage happens to be affordable. An estate that keeps bronze forever without asking that question is usually paying for years of data it could not reprocess anyway, because the transformation code has changed beyond recognition.

• Append, do not update; keep the arrival time alongside the source's own timestamps
• No filtering, no type coercion, no deduplication — those are silver's decisions
• Retention set by how far back a correction must reach, and stated explicitly

What silver decides that bronze cannot

Silver is where the estate decides what is true. Duplicates are resolved to one row per entity, types are conformed, keys are made reliable, late and out-of-order records are reconciled, and quality rules are enforced rather than assumed. Everything downstream inherits these decisions, which is why silver is the layer most worth investing in and the one most often skipped.

The decisions here are genuinely contestable, and that is the point of having a layer for them. When two source systems disagree about a customer's address, something has to decide which wins — most recent update, most trusted system, or a survivorship rule per field. When a record arrives for a period already closed, something has to decide whether it restates the period or lands as a correction. These are not implementation details; they are the definitions the business will later be arguing about in a meeting, and they belong in a contract with the producing team.

Skipping silver is the most common and most expensive shortcut in the pattern. When bronze feeds gold directly, every gold model re-implements deduplication and conformance in its own SQL, and they diverge — not immediately, but the first time one of them is edited under deadline. The symptom arrives months later as two dashboards with different customer counts, and by then the cause is distributed across a dozen models.

Gold is where a measure gets a definition and an owner

Gold is not simply aggregated silver. It is the layer where a business measure — revenue, active account, gross margin — is defined once, in one place, with a named owner who can be asked what it includes. The aggregation is the visible part; the definition and the ownership are what make the number quotable in a meeting without someone reopening the argument.

This is also the boundary where the lakehouse meets the semantic layer, and the two overlap in ways worth deciding deliberately. If a BI semantic model also defines measures, you now have two places a definition can live, and the estate needs to state which one is authoritative. Neither answer is wrong; having both silently is.

A practical marker of a healthy gold layer is that its tables are named after decisions rather than after sources. A table called `daily_margin_by_category` invites a question about its definition. A table called `sales_final_v2` invites nothing, and nobody owns it.

When medallion is the wrong shape

Medallion is the wrong shape when the data needs almost nothing done to it, when there is a single consumer with a single definition, or when the cost of three copies is material against the value of the workload. A reference table of forty country codes does not need bronze, silver and gold. Neither does a one-off analytical dataset that will be read twice and discarded.

The honest test is whether each layer would carry a different responsibility for this particular dataset. Where the answer is no, collapse the layers instead of creating empty ones — a validated table read directly by its one consumer is a complete and defensible architecture, and calling it silver adds nothing but a naming convention.

The opposite mistake also exists. Teams sometimes add a fourth and fifth layer to represent staging, or a departmental mart, until the number of hops makes lineage unreadable and every change requires touching five models. The pattern has three layers because three responsibilities were identified. A fourth layer needs a fourth responsibility, not a fourth stage.

Three copies, one responsibility: the common failure

The characteristic failure is an estate where bronze, silver and gold contain nearly the same tables with nearly the same columns, and the only transformation between them is a rename. This costs three times the storage, three times the compute to maintain, and three times the surface area to debug, while answering exactly one question instead of three.

It usually arrives honestly. A team adopts the pattern as a standard, applies it uniformly because uniformity seems safer than judgement, and each new dataset gets three tables by convention. Nobody decides to do this; it is what happens when the pattern is a rule instead of a default with a test.

The diagnostic is simple and worth running on an existing estate: for each dataset, write down what silver decided that bronze did not, and what gold defined that silver did not. Where either answer is blank, the layer is a copy. That audit is usually the fastest cost reduction available in a lakehouse, and it improves lineage readability at the same time.

Medallion does not tell you how to model gold

The pattern says where data sits in its lifecycle, not how it is shaped for analysis. Choosing a grain, deciding what a fact row represents, conforming dimensions and handling slowly changing attributes are warehouse modelling decisions that medallion is silent on. An estate with three well-run layers and no agreed grain still produces conflicting numbers.

This is a frequent source of confusion, because the layer names feel like a data model. They are not. Gold can be a star schema, a set of wide denormalised tables, or a mix chosen per consumer — the pattern accommodates all three and recommends none. The decision belongs to whoever owns the measures, and it needs making explicitly instead of emerging from whatever the first gold table happened to look like.

File layout decides whether the lakehouse is affordable

In a lakehouse, query cost is driven largely by how many files a query has to open, which makes file layout an operating concern instead of a tuning afterthought. Databricks collects per-file statistics — minimum and maximum values, null counts, record counts — and uses them to skip files that cannot match a predicate. Layout determines how effective that skipping is.

The platform guidance has moved here, and it is worth following instead of carrying forward older habits. Databricks now recommends liquid clustering for new tables in place of Z-ordering, noting that Z-ordering is not idempotent, cannot be combined with liquid clustering, and does not guarantee improvement across repeated runs. For managed tables, predictive optimisation collects statistics on frequently filtered columns instead of relying on a fixed column limit.

The operational point underneath the specifics is that somebody has to own small-file accumulation, compaction and retention. Streaming writes and frequent small merges create many small files, and a table that is never compacted degrades gradually — which makes it hard to attribute, because no single day is when it broke. This ownership is exactly what a warehouse handles for you, and it is a real part of the cost difference between the two architectures.

How streaming changes the layers

Streaming does not add a layer, it changes what bronze is. With streaming ingestion, bronze becomes an append-only landing of events as they arrive, and silver becomes the place where ordering, deduplication and late arrival are resolved — which is harder in a stream than in a batch, because the window is open rather than closed.

The consequential decision is how long silver waits before treating a period as settled. Wait too little and late events force restatements of published figures; wait too long and the freshness advantage that justified streaming disappears. That watermark is a business decision about how much correction the reader tolerates, and it belongs in the same document as the late-arrival policy for every other source.

Gold is usually unaffected. A measure defined once over silver does not care whether silver was populated by a stream or a nightly load, which is one of the pattern's genuine strengths: it localises the streaming complexity in one layer instead of spreading it through the estate.

What this guide does not cover

This describes what the layers owe and when the shape is justified. It does not recommend a table format, a platform or a specific clustering strategy, because those depend on the engine, the query patterns and the volumes involved. It also assumes a lakehouse, and a warehouse-centred estate solves the same responsibilities differently and usually with less layout work.

Nor does it contain cost figures. The cost of three copies depends entirely on volume, retention and storage pricing, and any number quoted without those is misleading. What is stated here is structural: where the cost goes, and what has to be true for it to be worth paying.

Questions and trade-offs

Technical references

• Databricks: what is the medallion lakehouse architecture? ↗
• Databricks: data skipping, liquid clustering and Z-ordering ↗
• Databricks: Unity Catalog and lineage ↗