Trinium
Start free trial
Home Product Security Nodes & connectors Docs Use cases Pricing FAQ About us Start free trial
Browse the docs
Preparation node

Data Compare

Check one dataset against another and report what disagrees.

Category
Preparation
Also known as
compare, diff, difference, reconcile, reconciliation, migration, mismatch, validate, verify, match, audit, two datasets, before after, regression

What it does

Holds a Target dataset against a Primary one and reports every way they differ. Rows are paired on a key you choose; columns are paired by name. Three kinds of finding come back, deliberately kept apart: a row that is in one dataset and not the other, a column whose data TYPE changed, and an individual value that changed. The Summary output has one line per column (the type on each side, how many values were compared, how many disagreed) and appears in the Data tab even with nothing wired to it. The Differences output has one line per mismatched cell, always carrying the key and the column that disagreed, ready to write to CSV or an Excel report.

Inputs and outputs

InputsPrimary, Target.
OutputsSummary, Differences.

When to use it

  • You migrated a table to a new system and need to prove nothing changed
  • Two systems should agree every night and you want to be told when they don't
  • You cleaned up a dataset and want to see exactly what your changes touched

Examples

Did the migration survive?

old system → Primary
new system → Target
Key: order_id

Press Run, click the node, read the Summary in the Data tab.

A full report to send

Data Compare (Differences) → CSV File

One line per mismatched cell: the key, the column, and both values.

A composite key

Key: region, order_id

Rows pair only when every key column matches.

Only the columns that matter

Compare columns: amount, status

Leave it empty to compare every column the two datasets share.

Keep a history

Data Compare (Summary) → Database output (Append)

Then you can answer 'when did the two systems start drifting?'

Tips

  • Primary is your source of truth; Target is the dataset being checked, and the one the report is about.
  • Leave Differences unconnected and the Summary still appears in the Data tab; that is the quickest way to use it.
  • Values are compared as text, so a column that changed type but kept its data reports a type mismatch and ZERO value mismatches. That is the difference between 'it was re-typed' and 'it was corrupted'.
  • The key must identify a single row on each side. If a key value appears twice the run stops and names it, because comparing the wrong two rows would make every number here wrong.
  • A key column must be the same type on both sides. Retype it upstream with Manage Columns if it isn't.
  • Nothing is cleaned up for you: trim whitespace, fix casing or round numbers upstream with Data Cleanse first, or those differences will be reported, correctly but noisily.
  • Column names are matched exactly, including case. Rename with Manage Columns if the two systems spell a column differently.
  • Rows missing from Target and extra rows in Target are both reported. A row lost in a migration cannot show up as a Target row, and it is usually the most important finding.
  • There is no limit on the Differences output. For a very large report write CSV or a database table; Excel itself stops at about a million rows.
  • A Test run reads only 50 rows from each source, so rows past that look 'missing'. Press Run for the real figures.
Manage ColumnsData Cleanse