Market Options: Migration Validation Tooling

A comparison of tools that diff a source dataset against a migrated target. Evaluated for Aliz migration projects where the target is usually BigQuery and the sources range from Oracle and PostgreSQL to Hive and Teradata.

At a glance

Tool	Status	Self-host	Licence	Connects via connector	Custom schema / column mapping	Free trial
Datafold data-diff (Cloud)	Active	No	From $799, custom	Yes	Yes	Yes
Datafold data-diff (OSS)	Archived since 2024	Yes	OSS	Yes	Yes	n/a
rg-alizai-datacomp-tool	Active (Aliz internal)	Yes	Internal	Yes	No	n/a
datacompy	Active	Yes	OSS	No (compares DataFrames)	No	n/a
dbt-audit-helper	Active	Yes	OSS	No (runs inside dbt)	No	n/a
Data Validation Tool by Google (DVT)	Active	Yes	OSS	Yes	Yes	n/a
Datagaps	Active	No	Commercial	not evaluated	not evaluated	Yes

None of the tools copies source data into the target. The connector-based tools read both sides in place, so the machine running them needs network access to both systems. datacompy compares DataFrames you have already loaded and dbt-audit-helper runs inside dbt, so for those two the extraction and the memory it takes are your responsibility.

Tool notes

Datafold data-diff (Cloud)

• Sources — any SQL database (vendor claim without specifics; at minimum the OSS list below).
• Functionality — column profiles with aggregate statistics, distributions, and histograms of column-level differences; column and type mapping between the two datasets with a per-column status (identical, percentage of values different); value-level diff of the rows that differ.
• Pro — cloud based, nothing to operate.
• Con — proprietary SaaS.
• Note — the free tier covers small teams on a modern data stack (cloud warehouse plus dbt) with column-level lineage and limited Data Diff usage.

Datafold data-diff (OSS)

• Sources — PostgreSQL 10 and newer, MySQL, Snowflake, BigQuery, Redshift, DuckDB, MotherDuck, Microsoft SQL Server, Oracle, Presto, Databricks, Trino, ClickHouse, Vertica.
• Functionality — command-line diff of two tables: the rows that differ and summary statistics, using the same diffing algorithm as the Cloud edition. The hosted UI, lineage, column profiles, and histograms are Cloud-only.
• Pro — self-hosted, CLI based.
• Con — no longer maintained; the repository was archived in 2024.

caution

Do not start new work on the archived OSS edition. It will drift away from current database drivers and Python releases with nobody fixing it.

rg-alizai-datacomp-tool (Aliz internal)

• Sources — BigQuery, Hive, Teradata, Oracle, SQL Server, PostgreSQL, MySQL.
• Functionality — column validation (count, sum, avg, min, max, quartile, percentile, group by); row validation (BigQuery, Hive, Teradata, Oracle, SQL Server, PostgreSQL, MySQL only); schema validation; table size validation; custom query validation; ad hoc SQL exploration.
• Pro — it is ours, so we can extend it.
• Con — last contribution was two years ago, so it may not work with current libraries and Python versions; it makes assumptions about table shape.
• Note — BigQuery-side processing requires both tables in BigQuery; otherwise the comparison runs in memory.

datacompy

• Sources — pandas or Spark DataFrames.
• Functionality — comparison summary (columns in common, columns only in original or only in new, rows in common, rows only on one side, rows with some or all compared columns unequal) and a detailed comparison (columns with unequal values or types, sample rows with unequal values, sample rows only in original or only in new).
• Pro — a plain library that runs wherever pandas or Spark runs.
• Con — DataFrames only; comparing large data needs a separate Spark cluster.

dbt-audit-helper

• Sources — whatever your dbt adapter supports.
• Functionality — compare_and_classify_query_results, compare_and_classify_relation_rows, quick_are_queries_identical, quick_are_relations_identical, compare_row_counts, compare_column_values, compare_all_columns, compare_which_query_columns_differ, compare_which_relation_columns_differ, compare_relation_columns.
• Pro — lives in the same layer as the ELT, so validations version and run with the models.
• Con — only runs inside dbt.

Data Validation Tool by Google (DVT)

• Sources — AlloyDB, BigQuery, Db2 LUW, Db2 z/OS, FileSystem, Hive, Impala, MSSQL, MySQL, Oracle, PostgreSQL, Redshift, Spanner, Snowflake, Sybase ASE, Teradata.
• Functionality — column validation (count, sum, avg, min, max, stddev, group by); row validation (not supported for FileSystem connections); schema validation; custom query validation; ad hoc SQL exploration.
• Pro — actively maintained by Google Professional Services, supports custom column mapping, pushes work down into the engines and persists results to BigQuery.
• Con — none recorded in the comparison. The practical gotchas we hit (identifier casing, allow-list matching, Python version pin) are in the DVT runbook.

Datagaps

Commercial ETL-testing suite with a free trial. Not evaluated beyond a first look.

How to choose

• Migrating into BigQuery from a heterogeneous source (Oracle, Teradata, SQL Server, Hive) — use DVT. It is the only actively maintained open-source option that covers these sources and writes results to BigQuery. Follow the runbook.
• The whole pipeline already runs in dbt — use dbt-audit-helper. Validations become dbt models and run in the same CI.
• Both datasets are already DataFrames in one process — use datacompy. Do not stand up infrastructure for a comparison you can do in a notebook.
• The client wants a SaaS UI with lineage and is willing to pay — Datafold Cloud. Confirm the source list with the vendor first.
• We need a feature none of the above has — consider reviving rg-alizai-datacomp-tool, but budget for a dependency refresh before relying on it.

Open items

• Estimated usage cost for a 100 GB comparison has not been measured for any of the tools.
• Datagaps has not been evaluated in depth.

Source

Migration validation tooling (Google Sheet, September 2026).