Preparation node
Data Validation
Check the data against rules. Rows that hold up leave Valid; the rest leave Invalid, tagged with what they broke.
What it does
Checks the data against rules you declare, and decides what a violation costs. Rows that pass every rule leave the Valid anchor; rows that break one leave Invalid with a `_validation_errors` column naming what they broke. Rules come in two kinds: row rules (not null, ranges, patterns, allowed values, a custom condition) judge each row, while structure rules (a column exists, a column's type) describe the whole dataset and fail the run instead of routing rows.
Inputs and outputs
| Inputs | One. |
| Outputs | Valid, Invalid. |
When to use it
- Stop bad data before it reaches the destination
- Quarantine reject rows to a file for someone to fix
- Catch it loudly when an upstream source changes shape
Examples
Require a value
email → Is not null
Sane numbers
age → Number is in range 0 … 120
Numeric columns; nulls pass.
A fixed vocabulary
status → Is one of: active, paused, closed
Format
sku → Matches a pattern ^[A-Z]{2}-\d{4}$Across columns
Custom expression: [ship_date] >= [order_date]
Guard the shape
customer_id → Column exists
A structure rule: it fails the run.
Tips
- Quarantine routes and succeeds; Fail the run routes and then fails with a summary; Warn only never fails. One setting for the whole node.
- Nulls PASS every row rule except Is not null / Is not blank: a missing value isn't a bad value. Add an Is not null rule when you need both.
- A row breaking several rules is listed once, with every rule it broke.
- Set a Message on a rule to say what it means in business terms. It replaces the rule name in the errors column and the run summary.
- Structure rules can't route rows (there's nothing row-specific to send), so they fail the run, or just warn under Warn only.
- Both anchors are optional to wire: connect only the side you need.