Preparation nodes
Preparation nodes work on one stream of rows: they keep or drop rows, add and rename columns, calculate new values, and check the data before it goes anywhere.
Every formula and custom filter is written in one SQL expression language, the same in every node and whatever the data came from.
Trim whitespace, remove characters, fix nulls, and standardize text.
Check one dataset against another and report what disagrees.
Describe what is actually in the data, one row per column.
Check the data against rules. Rows that hold up leave Valid; the rest leave Invalid, tagged with what they broke.
Rename every matching column at once. Add or strip a prefix, replace text, or fix the capitalisation.
Keep or drop columns by a rule: by type, by a name pattern, or by an expression.
Fill missing values with a constant or a column statistic (mean/median/mode).
Split rows by a condition: matches go to True and the rest to False.
Add or replace columns with SQL expressions.
Apply one expression across several fields at once.
Compute a column from other rows with SQL window functions, such as running totals, ranking, lag/lead and moving averages.
Synthesize rows from nothing: a sequence of numbers, dates or date-times.
Keep, drop, reorder, rename, and retype columns.
Stamp each row with a sequential number or a UUID.
Send each row down one of several named branches, where the first matching rule wins. For a single yes/no condition, use Filter.
Keep a subset of rows: the first or last N, skip N, every Nth, a 1-in-N chance, or exactly N at random.
Order rows by one or more columns.
Split rows into first-seen (Unique) and repeats (Duplicate) by key columns.