Nodes
A pipeline is built from nodes. Each one reads, shapes or writes rows, and passes them to the next node as they arrive.
Nodes that start a pipeline. They read files, cloud storage and SFTP, databases and warehouses, Apache Iceberg, SaaS APIs, or a table you type.
32 nodesPreparationFilter and route rows, choose and rename columns, calculate with formulas, clean, deduplicate, sort, sample, validate, profile and compare data.
18 nodesJoinCombine streams on keys, join several inputs at once, stack inputs, append fields, replace values from a lookup, and match records spelled differently.
6 nodesLookupAdd columns to each row from a system outside the pipeline, such as a web API, a service of your own, or a database table.
3 nodesParseTake text apart. Convert dates and times, apply regular expressions, split delimited text, and read JSON, URLs and XML held in a column.
6 nodesTransformReshape a whole table. Summarize rows into groups, pivot them into a cross-tabulation, or unpivot wide columns into name and value rows.
3 nodesOutputNodes that end a pipeline. They write files, cloud storage and SFTP, databases and warehouses, Apache Iceberg, SaaS APIs, Google Sheets and email.
26 nodesCanvas toolsA note pinned to the canvas, and a container that groups nodes and can switch a whole section off. Neither one runs or touches data.
Every node, A to Z
Read an Apache Iceberg table from AWS Glue, S3 Tables or a REST catalog, with time-travel and column pruning. Its columns show without a run.
Write rows to an Apache Iceberg table through AWS Glue, S3 Tables or a REST catalog, appending or fully refreshing, with auto-create and partitioning. No crawler needed.
Send your rows to an API many at a time and add its answers as new columns.
Look up each row against an API and add the answer as new columns.
Append every Source row's columns onto every Target row (a cross join).
Query data in S3 with Amazon Athena. Picking a table shows its columns without a run, because the catalog already knows them.
Write the pipeline's output to an Azure Storage container with a templated blob path (dates, run id) and a write mode.
Read a single blob or a whole prefix from an Azure Storage container. A Bookmark can read only the files no successful run has read.
Read a table or SQL query from BigQuery. Columns AND what the query will scan both show without a run.
Load rows into a BigQuery table with a load job, appending or replacing. Loading is free, and the table is created if needed.
Read a table or SQL query from ClickHouse. Columns show without a run, even for a query you write yourself. A Bookmark can read only the rows added or changed since the last successful run.
Load rows into a ClickHouse table (append or replace; auto-creates a MergeTree).
Pivot rows into a wide cross-tabulation.
Write the incoming rows to a CSV file.
Read rows from a CSV file and stream them into the pipeline.
Trim whitespace, remove characters, fix nulls, and standardize text.
Check one dataset against another and report what disagrees.
Describe what is actually in the data, one row per column.
Check the data against rules. Rows that hold up leave Valid; the rest leave Invalid, tagged with what they broke.
Look up each row's keys in a database table and add the matching row's columns.
Read a table or SQL query from a Databricks SQL warehouse. Results stream back columnar.
Load rows into a Databricks table (append or replace) through a Unity Catalog volume.
Convert a column between real date/time values and formatted text.
Rename every matching column at once. Add or strip a prefix, replace text, or fix the capitalisation.
Keep or drop columns by a rule: by type, by a name pattern, or by an expression.
Read one sheet, or every sheet, from an Excel workbook.
Write a styled Excel report with a table, number formats and an optional chart.
Copy, move or delete the files an input, List Files or File from URL hands it through its Files anchor, before the data is processed or after the run succeeds or fails.
Read one file from a web link: CSV, JSON, Parquet, Excel or XML.
Fill missing values with a constant or a column statistic (mean/median/mode).
Split rows by a condition: matches go to True and the rest to False.
Look up an input field against a mapping table to replace found text or append matched fields.
Add or replace columns with SQL expressions.
Apply one expression across several fields at once.
Compute a column from other rows with SQL window functions, such as running totals, ranking, lag/lead and moving averages.
Find records that mean the same thing despite typos, spelling, or word order. Cluster duplicates, or match against a reference list.
Write the pipeline's output to a Google Cloud Storage bucket with a templated key (dates, run id) and a write mode.
Read a single object or a whole prefix from a Google Cloud Storage bucket. A Bookmark can read only the files no successful run has read.
Synthesize rows from nothing: a sequence of numbers, dates or date-times.
Read a tab and range from a Google spreadsheet. Column types are inferred unless you read everything as text.
Write rows to a tab in a Google spreadsheet, overwriting or appending, and optionally creating the tab.
Read HubSpot CRM objects: contacts, companies, deals and tickets.
Read Jira issues with JQL.
Combine two inputs on matching keys, returning the matched rows plus the unmatched orphans from both sides.
Combine 2 or more inputs into one on a shared key (or record position). Every row is kept by default, null-filled where they don't match.
Parse a JSON text column into typed columns, a name/value table, or auto-flattened columns.
Write rows as an array or newline-delimited JSON.
Read array or newline-delimited JSON; nested objects become columns.
List the files in a folder as rows, with each one's path, name, size and modified time, without reading any of them.
Keep, drop, reorder, rename, and retype columns.
Read documents from a MongoDB collection into rows.
Write rows into a MongoDB collection as documents. Turn on All or nothing and a replace that fails leaves the old collection in place.
Read a table or SQL query from a MySQL database. A Bookmark can read only the rows added or changed since the last successful run.
Load rows into a MySQL table. Turn on All or nothing and a load that fails leaves the table as it was.
Write the pipeline's output to a Windows network share or a NAS over SMB with a templated path (dates, run id) and a write mode.
Read a single file or a whole folder from a Windows network share or a NAS over SMB. A Bookmark can read only the files no successful run has read.
Write the incoming rows to a columnar Parquet file.
Read one or more columnar Apache Parquet files.
Read the result of a SQL query from PostgreSQL. A Bookmark can read only the rows added or changed since the last successful run.
Load rows into a PostgreSQL table (append, truncate, or upsert). The load is one transaction by default, so a load that fails leaves the table as it was.
Stamp each row with a sequential number or a UUID.
Read a table or SQL query from Amazon Redshift. A Bookmark can read only the rows added or changed since the last successful run.
Load rows into an Amazon Redshift table (append, replace, or upsert). Best for small and medium loads. The load is one transaction by default, so a load that fails leaves the table as it was.
Match, extract, replace, or split text with a regular expression.
Read records from an HTTP/JSON API, following its pagination to the end.
Push rows to an HTTP API, one record per request or many batched into one.
Send each row down one of several named branches, where the first matching rule wins. For a single yes/no condition, use Filter.
Write the pipeline's output to an S3 bucket with a templated key (dates, run id) and a write mode.
Read a single object or a whole prefix from an S3 bucket. A Bookmark can read only the files no successful run has read.
Read Salesforce objects with SOQL, paged and typed automatically.
Write rows back to Salesforce: insert, update, upsert or delete, 200 records a request, with every rejected record reported.
Keep a subset of rows: the first or last N, skip N, every Nth, a 1-in-N chance, or exactly N at random.
Email the rows to people: one message per row, per group, or one for the whole run, with the data attached as CSV or Excel.
Write the pipeline's output to an SFTP server with a templated path (dates, run id) and a write mode.
Read a single file or a whole folder from an SFTP server. A Bookmark can read only the files no successful run has read.
Read a table or SQL query from Snowflake. Picking a table shows its columns without a run, and without waking a warehouse.
Load rows into a Snowflake table, creating it if it doesn't exist.
Order rows by one or more columns.
Read a table or SQL query from Microsoft SQL Server. A Bookmark can read only the rows added or changed since the last successful run.
Load rows into a SQL Server table. Turn on All or nothing and a load that fails leaves the table as it was.
Write the incoming rows into a table in a new SQLite database file.
Read a table, or the result of a SQL query, from a local SQLite database file.
Group by one or more columns and compute aggregate values: sum, average, count, median and more.
Type a small table straight onto the canvas: lookup values, constants, or a fixed set of rows, versioned with the pipeline.
Split a text field into multiple columns or rows by a delimiter.
Unpivot wide columns into tall Name/Value rows (the inverse of Cross Tab).
Read a table or SQL query from a Trino / Starburst cluster. A Bookmark can read only the rows added or changed since the last successful run.
Load rows into a Trino / Starburst table (append or truncate; auto-creates). Turn on All or nothing and an append that fails leaves the table as it was.
Stack the rows of 2 or more inputs into one, matched by field name (or record position), with the fields reconciled.
Split rows into first-seen (Unique) and repeats (Duplicate) by key columns.
Split a URL column into its parts (scheme, host, domain, path, query, fragment), pull query parameters into columns, or explode them into rows.
Parse an XML text column into typed columns, a name/value table, or auto-discovered columns.
Read one or more XML files, shredding a repeating element into typed rows.
Read Zendesk tickets, users, or organizations.