Preparation node
Dynamic Select
Keep or drop columns by a rule: by type, by a name pattern, or by an expression.
What it does
Keeps or drops columns by a RULE instead of by name. Manage Columns needs you to name every column; this one says “keep every numeric column” or “drop everything ending _tmp” once, and keeps meaning it when the source changes. Column order is never touched; that stays Manage Columns’ job.
Inputs and outputs
| Inputs | One. |
| Outputs | One. |
When to use it
- A source that gains or loses columns over time, and a load that must keep working
- Strip a family of columns: anything ending _tmp, anything starting ext_
- Keep just the numeric columns before a Multi-Field Formula or a Summarize
Examples
Survive a new column
Drop · name starts with utm_
A column added upstream next month is dropped too, so the table load does not break at 2am.
Keep every number
Keep · Type: Number
Whatever the numeric columns happen to be called today.
Numbers plus the keys
Keep · Type: Number · Always keep: id, customer_id
“Always keep” is the escape hatch when one rule cannot say it.
An expression when you need OR
[type] = 'number' OR [name] = 'id'
The Expression tab sees one row per column: [name], [type], [arrow_type], [index], [nullable].
Tips
- The preview lists every input column and marks it kept or dropped. It comes from the engine, so it is exactly what the run will do.
- A rule that keeps nothing is refused with a message naming what to change, in the builder, before you run.
- A name in “Always keep” that is not in the input is ignored rather than failing the run. That is deliberate: this node exists to tolerate drift.
- Basic mode ANDs the type test and the name test. For OR, switch to Expression.
- Name tests ignore case unless you ask them not to, matching the Filter node.