Almost nobody fails at building a medallion architecture. Bronze, Silver and Gold get drawn in twenty minutes, and a competent team ships a working pipeline in a fortnight.
What fails is the third year. By then Silver has fragmented, because nobody owned the conformance rules. Gold has multiplied into hundreds of near-duplicate datasets. Compute cost has grown faster than data volume, because a three-layer design writes every record several times and re-reads history whenever logic changes. And when the CFO asks who is accountable for the revenue number, the answer is a platform team that built the pipes and no business domain that accepted ownership of what flows through them.
None of that is a technology problem. It is an operating model problem wearing an architecture diagram as a disguise.
What Is Medallion Architecture?
Medallion architecture is a data design pattern that organizes a lakehouse into three layers, Bronze, Silver and Gold, progressively improving the structure and quality of data as it moves through them. Bronze holds raw source data. Silver holds cleaned and conformed data. Gold holds business-ready data products. It is also called multi-hop architecture.
That definition is where most articles stop, which is why so many teams can recite it and still end up with an estate they do not trust.
The problem it solves is this. An enterprise ingests from databases, APIs, SaaS platforms, files and event streams. Left alone, every consumer builds their own version of “clean”. One analyst filters null customer records, another does not. A data scientist drops duplicates in a notebook, a dashboard does not. The same source data produces four different revenue numbers depending on who ran the query.
The medallion data architecture makes cleaning a platform responsibility rather than a per-consumer one. Two properties follow, and they are the entire justification for the core medallion architecture principles.
- Reprocessing: if raw data is preserved, downstream datasets can be rebuilt when business logic changes, without returning to source systems that may have purged history.
- Reuse: if conformed definitions live in one shared place, five use cases consume the same definition of customer instead of implementing it five times.
If an implementation does not deliver both, it is not a medallion data architecture. It is three storage tiers and a naming convention.
The medallion data architecture pattern was popularized by Databricks but is not proprietary. Microsoft recommends it as a default design pattern for OneLake in Fabric, and equivalent layered approaches run on Snowflake and on plain cloud object storage.
Bronze, Silver and Gold Layers of Medallion Data Architecture
The most common misreading is that the layers are quality tiers, with data getting steadily nicer as it moves right. They are better understood as ownership and reprocessing boundaries. That reframe makes the core medallion architecture principle obvious.
| Layer | What it holds | What it protects | Owner | Access |
| Bronze | Raw source data, append only, plus ingestion timestamp, source system and batch identifier | The ability to rebuild everything downstream when logic changes | Platform team | Restricted |
| Silver | Cleaned, deduplicated, conformed data carrying enterprise definitions | The single agreed meaning of customer, product, patient, trade | Domain data owners | Engineering, analytics, data science |
| Gold | Aggregations, dimensional models, metrics and features shaped for consumption | The business outcome, and the ability to change presentation without re-litigating meaning | Consuming domain | Business, BI, AI, apps |
Bronze Layer: Preserve the Source
Bronze is the landing zone and the historical record. What gets added is technical metadata, not business logic.
Bronze is append only. If Bronze is mutable, the replay guarantee that justifies the whole architecture disappears, and you have paid for three layers to get the benefit of one.
The most common Bronze mistake is over-cleaning. Every field dropped at ingestion is information you cannot recover when someone asks a question in eighteen months that today’s logic did not anticipate. The rule is not “transform late”. It is do not irreversibly discard information you may need later. Masking a regulated field is legitimate. Dropping a column because no current use case needs it is not.
The second mistake is unbounded retention. Bronze retention should be set by a real replay requirement, expressed in months and justified by a business or regulatory need. “Keep everything forever” is a decision not to decide, and it shows up on the storage bill.
Silver Layer: Clean, Validate and Conform
Silver is where data becomes trustworthy: schemas standardized, duplicates removed, invalid records handled, quality rules applied, enterprise definitions established.
Silver determines whether the investment pays back, because it is the only layer where reuse happens. If five use cases need the same customer definition, that transformation belongs once in Silver, not five times in Gold.
That carries an implication most architecture documents avoid. Silver only works if somebody has the authority to say no: to refuse a new conformed table and arbitrate changes to an existing one. Where that authority is missing, Silver fragments. A project needs a conformed customer table, one exists but lacks two attributes, changing it means a conversation with three teams, so the project writes customer_conformed_v2. Six months later there are nine. No technical control prevents this. Only a named owner does.
Gold Layer: Business-Ready Data Products
Gold is designed around consumption: business rules, aggregations, dimensional models, metrics and features for dashboards, analytics, machine learning and applications.
Gold is not simply cleaner Silver. It is purpose driven. A Gold dataset should make a specific business question easier to answer, and if you cannot name that question, the dataset should not exist. That test sounds trivial and it is the most effective control against sprawl, because Gold is cheap to add and politically impossible to remove.
What Is a Lakehouse, and What Is Delta Lake?
A data warehouse gives you governed tables and strong query performance, but struggles with unstructured data and usually requires transformation before loading.
A data lake stores anything cheaply, but without transactional guarantees it degrades into a swamp: partial writes, no schema enforcement, no reliable way to know what a table contained yesterday.
A lakehouse combines them: cheap object storage underneath, with a transactional table format on top supplying warehouse-grade guarantees.
On Databricks that format is called as Delta Lake, and three properties make medallion data architecture viable rather than aspirational.
- ACID transactions mean a pipeline that fails halfway does not leave a half-written table for the next reader.
- Time travel lets you query a table as it existed at an earlier version, which makes an audit answerable and a bad load recoverable.
- Schema enforcement and evolution lets a table reject a malformed write, or accept a new column deliberately, rather than silently corrupting.
Without these, Bronze replay is not trustworthy and Silver corrections are not safe. The layers depend on the format underneath them.
A Worked Example of Medallion Architecture
A retailer ingesting point-of-sale transactions.
Bronze: raw records land as the till emits them, including malformed rows, test transactions and retry duplicates, with ingestion timestamp and store identifier appended.
Silver: retry duplicates collapsed on transaction identifier, test transactions flagged rather than deleted, product codes mapped to the enterprise master, currency normalised, store identifiers conformed to the corporate hierarchy. One transactions table every team agrees represents a sale.
Gold: daily_sales_by_store_and_category for merchandising, plus a rolling basket feature table for the recommendation model.
Had merchandising and data science each defined a sale separately, the dashboard and the model would eventually disagree, and nobody could say which was right.
ETL vs ELT: Where Transformation Belongs in a Lakehouse
ETL and ELT describe where transformation happens. Medallion describes how data is organized. They are different axes, and treating them as competing choices produces confused designs.
| Approach | Transformation happens | Where it fits | Main risk |
| ETL | Before loading | Traditional warehouses; regulatory transformation before landing | Information discarded before it can be reprocessed |
| ELT | After loading, on platform compute | Cloud lakehouses. Land first, refine through the layers | Raw retention costs accumulate without a policy |
| Hybrid | Light work at ingestion, business logic after | Most enterprise estates | Boundary drift, as business logic migrates into ingestion and escapes governance |
ELT fits a lakehouse medallion architecture for one reason: it preserves the option to reprocess. Hybrid is what most enterprises actually run, and its risk deserves attention. Once business logic is added at ingestion “just for now”, the boundary erodes, and six months later part of the definition of a customer lives in an ingestion job nobody governs. Write the medallion architecture principles or the boundaries down and audit it.
Building Medallion Architecture on Databricks
Medallion data architecture is not a Databricks product. Databricks supplies the technologies that implement it at scale, and that capability set has changed materially. If your internal standards still reference Delta Live Tables, they are out of date.
Delta Live Tables is now Lakeflow pipelines. Existing DLT code runs with no migration required, though some Python API names and SKUs have changed. The change was introduced at the 2025 Data and AI Summit. Lakeflow Jobs is the new name for Databricks Workflows, and Lakeflow Connect provides managed ingestion connectors.
| Concern | Databricks capability | Design note |
| Storage foundation | Delta Lake | ACID transactions and time travel make Bronze replay and Silver correction possible at all |
| File ingestion | Auto Loader | Incremental file discovery with schema evolution, instead of directory listing that degrades at scale |
| Managed ingestion | Lakeflow Connect | Records source lineage through to destination tables in Unity Catalog |
| Transformation | Lakeflow Declarative Pipelines | Databricks positions declarative pipelines as the way to build Bronze, Silver and Gold tables, with streaming tables in a medallion data architecture, with streaming tables and materialized views that refresh incrementally. |
| Incremental refresh | Streaming tables, materialized views | Chosen per table. The most consequential cost decision in the design |
| Change capture | Delta change data feed | Incremental Silver rebuilds without full Bronze re-reads |
| Quality enforcement | Pipeline expectations, constraints | Rules belong in pipeline code at the Silver boundary |
| Orchestration | Lakeflow Jobs | Layer boundaries should map to job boundaries so failure stays in one hop |
| Table maintenance | Liquid clustering, predictive optimization, vacuum | Three layers, three times the maintenance surface. Automate it |
| Governance and lineage | Unity Catalog | Captures lineage automatically for Databricks queries, to column level, across all workspaces on the metastore |
| Portability | Spark Declarative Pipelines | The declarative API now exists upstream in Apache Spark, which matters to procurement teams weighing lock-in |
Two decisions here carry most of the cost consequence. Streaming table or materialized view, which determines whether refresh is incremental or a full recompute. And change data feed or full re-read, which determines whether a Silver logic change costs an hour or a weekend at three years of history. Get those wrong and no amount of cluster tuning recovers the difference.
One clarification, because it produces bad access models: Unity Catalog is not a fourth layer. Medallion answers how data becomes trusted. Unity Catalog answers who may see it and how its movement is traced. Map access to the layer boundaries: Bronze restricted to platform teams, Silver open to engineering and analytics under policy, Gold exposed to business consumers. Broad Bronze access is the most common quiet governance failure, because it lets analysts bypass Silver and rebuild private definitions in notebooks. Every one of those is a future argument about a number.
Data Quality, Schema Drift and the Details That Decide Whether It Holds
Most reference architectures stop at the three layers. These decisions determine whether the implementation survives, and each needs a written position in your standards rather than a per-project improvisation.
Quality enforcement:
Rules belong at the Bronze-to-Silver boundary as expectations that warn, drop or fail. Settle in advance what happens to a failing record: quarantining preserves it for investigation, dropping silently destroys evidence, failing protects correctness at the cost of availability.
Schema drift
Source systems add columns without telling you. Bronze should accept them permissively, because rejecting them loses data. Silver should reject them by default, because propagating an unexamined field into conformed definitions is how meaning drifts. Permissive at the edge, strict at the conformance boundary.
Slowly changing dimensions
Silver is where SCD handling belongs. Decide per entity between current state only and full history. Getting it wrong changes the grain of every downstream Gold table.
Idempotency and replay
A pipeline that runs twice must not produce different results than one that runs once. Merge on a stable business key rather than appending, or every backfill risks duplication.
Late-arriving data
Events do not arrive in order. Decide how late is too late and what happens to records arriving after a Gold aggregate is published. Ignoring them by default means numbers change retroactively without explanation.
Table maintenance
Compaction, clustering and vacuum across three layers. Small-file accumulation in Bronze is the most common cause of unexplained query slowdown in a young estate.
What Medallion Architecture Costs
Most medallion decisions are made on architectural merit and revisited eighteen months later on cost. Bringing that conversation forward changes the design you choose.
| Cost driver | Why it appears | What controls it |
| Storage multiplication | The same record persists in Bronze, Silver and several Gold tables | Bronze retention tied to a real replay requirement; retirement process for unused Gold |
| Write amplification | Each hop is a materialisation | Consolidating layers in one pipeline; views instead of tables where latency permits |
| Reprocessing and backfill | Silver logic changes require re-reading Bronze history | Change data feed and incremental processing; forecast backfill frequency at design time |
| Refresh scheduling | Compute scales with asset count, not asset value | Cadence matched to consumption; retire unqueried assets |
| Table maintenance | Compaction, clustering and vacuum across three layers | Automated optimisation, not hand-tuned jobs per table |
| Idle compute | Clusters sized for peak backfill, left running for steady state | Right-sizing, autoscaling, cost attribution to the owning domain |
The last row matters more than it looks. If compute cost is not attributed to the domain that caused it, no domain has any reason to retire a Gold table or reduce a refresh frequency. Cost attribution is a governance control, not a finance report.
The cost business cases most reliably omit is reprocessing. Migration models forecast steady-state ingestion and ignore backfill, so the platform lands on budget in year one and materially over in year two, exactly when finance stops treating it as an investment and starts treating it as run cost. Model reprocessing at year-three volumes before you commit.
Pro Tip: Two numbers worth instrumenting from day one: Gold tables with no query activity in ninety days as a share of total Gold tables, and reprocessing compute as a share of total pipeline compute.
When to Choose Medallion Architecture
Medallion is a good default for a large class of enterprise workloads and a poor fit for a meaningful minority. Choose it according to your medallion architecture principles rather than by convention.
Choose medallion when most of these hold:
- Multiple downstream consumers need the same conformed definitions of core entities
- Business logic changes often enough that reprocessing from raw is routine
- You are subject to audit, lineage or reproducibility obligations
- Source systems are numerous, heterogeneous or unreliable
- Data volume and consumer count are both growing
- You have, or can create, named owners for conformed definitions
Choose something simpler when:
- The integration is small, single-source and single-consumer. Three layers add cost and produce no reuse
- Latency requirements are tight enough that each hop is a material delay
- Data arrives already trusted and conformed from an authoritative system
- No one will own the conformance rules. The layers will fragment and you will pay for the architecture without receiving its benefit
That last condition decides most outcomes and is the least often assessed before a programme starts. The architecture is rarely the hard part. Who owns conformance, what reprocessing costs at year-three volumes, and which assets can be retired rather than migrated are the questions that determine whether the estate is trusted three years on.
The goal is never three copies of every dataset. It is a clear boundary between ingestion, trusted reusable data, and business-facing products. If you can achieve that boundary with two layers, use two.
Taking the Next Steps
Medallion architecture looks identical on every whiteboard and behaves differently in every industry. A clinical data estate under GxP obligations, a bank with intraday risk reporting and a retailer with seasonal volume spikes reach different conclusions about layer boundaries, retention and ownership, all for good reasons.
Book a discovery call with Modak. As a certified Databricks partner, Modak offers a working session with their data engineering leads to assess your current estate, model the reprocessing cost profile for your volumes, and give you a clear view of what medallion architecture should look like for your industry and regulatory position.
Frequently Asked Questions About Medallion Architecture
Is medallion architecture the same as ETL?
No. ETL and ELT describe where transformation happens. Medallion describes how data is organized as it is refined. Medallion is usually implemented with an ELT approach, but they are separate decisions.
Do I always need three layers?
No. Three layers are justified when multiple consumers need shared conformed definitions and reprocessing from raw is routine. A single-source, single-consumer integration rarely justifies the cost.
Is medallion architecture only for Databricks?
No. It was popularized by Databricks but is a logical pattern, not a product. Microsoft recommends it as a default design pattern for OneLake in Fabric, and equivalent approaches run on Snowflake and cloud object storage.
Is Unity Catalog a fourth medallion layer?
No. Unity Catalog is a governance layer spanning the estate, providing access control, discovery, auditing and lineage. It is not a stage in the refinement path.
What replaced Delta Live Tables?
Delta Live Tables is now Lakeflow pipelines, and existing DLT code runs without migration. Databricks Workflows is now Lakeflow Jobs.
Should Bronze data ever be updated?
No. Bronze should be append only. A mutable Bronze layer loses the ability to rebuild downstream datasets from an unaltered record of what the source sent, which removes the main justification for the layer.
Who should own the Silver layer?
Domain data owners, with platform team support. If nobody holds authority over conformed definitions, Silver fragments into competing versions of the same entity.
What makes medallion architecture expensive?
Storage multiplication, write amplification at each hop, reprocessing when Silver logic changes, refresh scheduling that scales with asset count rather than value, maintenance across three layers, and compute not attributed to the domain causing it.
How long should data be kept in Bronze?
As long as your actual replay requirement demands, expressed as a policy with a business or regulatory justification.
Can medallion architecture work with data mesh?
Yes. Databricks states the two are compatible. Each domain can run Bronze, Silver and Gold internally while owning its published data products. The constraint is organizational readiness, not technical compatibility.