1. Inventory consumers, not only tables
A table may feed scheduled reports, extracts, models and undocumented spreadsheets. Map its consumers and classify the consequences of delay or incorrect values. Agree which reporting windows and operational processes cannot tolerate interruption. Group a migration wave around a usable data product with an accountable consumer.
The inventory that matters is the one that extends past the platform boundary. Query logs and lineage will find every model and dashboard inside the warehouse; they will not find the analyst who exports to a spreadsheet every Friday, the finance system pulling a nightly extract, or the partner receiving a file. Those are the consumers who discover a migration by being broken by it.
Classify each consumer by consequence rather than by volume. A low-traffic table feeding a regulatory return outranks a heavily queried dashboard nobody acts on, and the ranking changes the wave order. Volume is easy to measure, which is exactly why estates default to it and migrate the wrong things first.
- Query logs for in-platform consumers, and a manual register for everything beyond it
- Consequence of a wrong or late value, stated per consumer
- A named accountable consumer per wave, who signs the wave off
2. Define what equivalence means
Record types, null handling, rounding, time-zone conversions and historical corrections. Row counts are useful but insufficient. Two datasets can contain the same number of rows while assigning revenue to different periods. Create a validation matrix covering source counts, financial aggregates, unique keys, relationship integrity and representative business questions.
The failures that survive to production are almost always semantic rather than structural. A timestamp interpreted in a different time zone moves transactions across a period boundary; a numeric type with different rounding behaviour changes a total by a few units per million; a null treated as zero in one engine and excluded in another changes an average without changing a count. Each is invisible to a row-count check and obvious to a finance reviewer the following month.
Write the matrix before the first wave and have the business owner approve the tolerances in it. A tolerance agreed in advance is an acceptance criterion; a tolerance agreed after a discrepancy appears is a negotiation conducted under pressure, and it usually ends with the discrepancy being accepted because the deadline is closer than the fix.
- Count and completeness checks for ingestion
- Aggregate reconciliation by business period and entity
- Approved record samples for transformations and edge cases
- Tolerances agreed and signed before the wave, not after a discrepancy
3. Prove a representative workload
Select a workload that includes the difficult joins, late data and access rules that production will require. Measure throughput and query behavior under realistic concurrency. Record versions, compute configuration and the measurement window. Vendor conversion tools can speed mechanical work, but the business owner must still sign off the transformed output.
Representative means it contains the parts that are hard. A proof that runs the three cleanest models against a quiet platform proves that the platform exists. Include the join nobody understands, the table with the informal partition scheme, the report that depends on a manually maintained mapping, and the access rules that decide which rows a given team sees.
Concurrency deserves particular attention because it is the dimension most often tested last and most often responsible for the first production incident. A query that completes comfortably alone can queue behind a scheduled load; the target platform's cost and isolation model is part of the proof, not a tuning exercise for afterwards.
4. Design the dual-run boundary
Running old and new systems together improves comparison but adds cost and coordination. Choose the period, the source of truth during it and the rules for handling divergence. Record how late corrections propagate to both sides. Avoid an indefinite parallel run with no owner or retirement criterion.
The decision that has to be explicit is which system is authoritative during the window. If both are, two numbers exist for every question and the organisation learns to prefer whichever supports the argument being made. Name one, say so in writing, and make the other a comparison instrument rather than a source.
Dual running is also where cost estimates fail, because the overlap is nearly always longer than planned. Budget for the realistic period rather than the optimistic one, and attach a named owner and a dated retirement criterion to the window. Parallel runs do not end because they are finished; they end because somebody was accountable for ending them.
5. Rehearse rollback before cutover
A rollback plan should identify the trigger, decision owner, last recoverable state, maximum acceptable data loss and consumer routing procedure. Test it on a rehearsal wave. If users can write to the new platform, explain how those writes are preserved or reconciled after rollback. A document saying 'switch back' is not a recoverable architecture.
The trigger is the part most often left vague, and vagueness is expensive at three in the morning. "If things look bad" is not a trigger. "If the reconciliation for the daily financial aggregate is outside the agreed tolerance at 06:00" is, because it can be evaluated by whoever is awake without convening a discussion about whether the situation qualifies.
Rehearsing also produces the number nobody has otherwise: how long a rollback actually takes. That figure determines how late in the morning a cutover can safely be attempted, and it is routinely assumed to be much smaller than it is.
6. Risks that are consistently underestimated
Six risks recur across migrations and none of them is technical in the sense that a tool solves it. They are undiscovered consumers, semantic drift in the numbers, dual-run cost, access and permission differences between platforms, orchestration and scheduling assumptions, and the absence of anyone accountable for retiring the old system. Each has a control, and each control has to be decided before the first wave.
The pattern connecting them is that all six are cheap to address before a migration and expensive afterwards. A consumer register costs a week at the start and costs a broken regulatory return later. An agreed tolerance costs a meeting at the start and costs a month of reconciliation later. The sequencing, not the difficulty, is what makes them risks.
| Dimension | How it presents | Control |
|---|---|---|
| Undiscovered consumers | A process nobody listed breaks after cutover, often outside the platform. | Consumer register extending past the warehouse, plus a cutover notice period. |
| Semantic drift | Counts match, totals do not: rounding, nulls, time zones or corrections differ. | Validation matrix with tolerances approved by the business owner before the wave. |
| Dual-run cost | Two platforms run for longer than budgeted, with no retirement date. | A dated retirement criterion and a named owner for the window. |
| Permission differences | Row and column rules do not translate; a team sees more or less than before. | Test access with the identities real consumers hold, including role changes. |
| Scheduling assumptions | Jobs that implicitly depended on completion order run in a different sequence. | Declare dependencies explicitly rather than by start time. |
| Nobody retires the old platform | The source warehouse runs indefinitely because switching it off has no owner. | Retirement is a wave with sign-off, a date and an accountable owner. |
7. Retire the old platform deliberately
Collect consumer sign-off, preserve required history and audit material, remove obsolete credentials and stop scheduled jobs under a change plan. Confirm the final bill and release ownership. Retirement is a migration wave in its own right, with the same sign-off and the same accountable owner, not a task that happens once everyone has stopped paying attention.
This guide is an original scoping framework informed by platform migration guidance. It does not promise a fixed duration, zero downtime or a client outcome, and it contains no figures for how long a migration takes or what it costs, because both depend entirely on the estate. What it states is the sequence and the decisions, which are the parts that transfer.
A worksheet for a business-owned cutover decision.
Copy these fields into the migration's acceptance record, then replace the hypothetical entries with evidence and named owners. The example concerns a daily finance extract; it defines tests to design, not results already obtained. A completed worksheet connects each business invariant to a reproducible comparison and an explicit release decision.
Scope and assumptions: Set tolerances and expected results with the business owner for the actual source and destination. This worksheet supplies test-design fields rather than executable conversion code or measured test output.
| Dimension | Field to complete | Illustrative test or decision |
|---|---|---|
| Dataset / consumer | Object identifier, versioned input snapshot, business date, source and destination settings, consumer and owner. | Finance approves the extract contract and the interpretation of the business date before comparisons start. |
| Business invariant | The rule that must remain true, at the same grain and with the same population filters. | Compare amounts and business keys by accounting day and currency; a grand total or row count alone can hide offsetting differences. |
| Null boundary | The intended meaning of a missing amount, missing key or unmatched relationship. | Select cases with missing values and unmatched joins. Check whether filters and aggregates preserve the approved meaning; do not silently treat missing as zero. |
| Numeric boundary | Source and target precision, scale, casts and the point where rounding occurs. | Compare individual boundary values and grouped totals. An apparently equal display is insufficient if an intermediate conversion loses precision. |
| Time boundary | Source time semantics, target timestamp variant, session time zone and business-day rule. | Select events around day boundaries and any relevant clock changes; verify day assignment as well as the rendered timestamp. |
| Late correction | Correction policy, extraction watermark, snapshot consistency and replay behavior. | Compare a correction applied after the original close. Establish how both systems account for the same correction before accepting a discrepancy. |
| Method / tolerance | Versioned comparison logic, representative case selection, matching keys, discrepancy classification and business-approved tolerance. | Record row-level exceptions and aggregate checks on equivalent inputs. Leave tolerance undecided until the owner sets it; no universal financial threshold applies. |
| Evidence / release | Evidence location, reviewer, business owner, unresolved exceptions, decision date and go/no-go authority. | If any required invariant fails or evidence is incomplete, retain the existing consumer path and resolve the exception before requesting cutover approval. |
At narrow widths, scroll the worksheet to read every column. With the table focused, use the arrow keys to scroll.
Worked decision: the hypothetical totals match, but a late correction appears under different accounting days. Record the mismatched keys and input snapshots, verify the correction and time-zone rules, and rerun the agreed comparison. Matching totals do not override the unresolved daily-allocation invariant. This is an acceptance scenario, not a benchmark or a migration outcome.
- For a Snowflake destination, explicitly record timestamp variants and session settings. Snowflake distinguishes local-time, no-time-zone and offset-bearing timestamps; verify the source system's semantics independently.
- Record numeric types through the complete calculation, including intermediate casts. Snowflake documents that a lower-precision conversion can lose digits that a later higher-precision cast cannot recover.
For source-object intake and a Snowflake-specific assessment boundary, see the Snowflake migration intake example. Keep this worksheet as the cross-platform acceptance record.
Official references for this example:
Questions and trade-offs
Question
Should all historical data move at once?
Only when consumer requirements justify it. Compare the cost and risk of moving all history, moving an active window with archived access, or migrating by product. Retention and access obligations need an approved decision.
Question
How long should the dual-run window be?
Long enough to cover a full reporting cycle for the consumers in the wave, so the comparison includes a period close rather than only ordinary days. Beyond that, the window should have a dated retirement criterion and a named owner rather than an open end. Parallel runs rarely stop because the comparison finished; they stop because somebody was accountable for stopping them.
