Redshift input

Read a table or SQL query from Amazon Redshift. A Bookmark can read only the rows added or changed since the last successful run.

Category

Input

Connection

Amazon Redshift

What it does

Reads from an Amazon Redshift cluster through a saved Connection and streams the result into the pipeline in batches. Pick a table with the live schema and table browser, or write your own SQL when you want the warehouse to do the heavy lifting first. Credentials live on the Connection and never enter the pipeline.

Inputs and outputs

When to use it

• Pull a warehouse table or query result into a pipeline
• Push joins and aggregation down to Redshift, which is far faster at them than moving rows out
• Move warehouse data out to a file, another database or a lakehouse table

Examples

public.events

SELECT user_id, COUNT(*) FROM public.events GROUP BY user_id

SELECT * FROM public.events WHERE created_at > current_date - 7

SELECT * FROM spectrum_schema.events

Bookmark column: updated_at · Start from: 2026-01-01 00:00:00

Tips

• Press ⤢ beside SQL query for the SQL Studio: a full-screen editor that browses the connection's schemas, tables and columns and runs your query with Run to here without closing.
• Pre-SQL runs on this same connection immediately before the query, so session settings stick: SET search_path, SET statement_timeout, or a procedure that prepares the extract. It also runs when you press Test, because a test run really does read from the database, so keep it to session setup.
• Columns appear when you press Refresh columns. A Serverless workgroup wakes up (and bills) when connected to, so Trinium only asks Redshift to describe the query when you press it. The answer is kept while you edit the pipeline, until you change this node.
• Aggregate in the query rather than pulling raw rows; Redshift is far faster at it than moving the rows out.
• Dates, times and timestamps come across as real dates, so you can sort, filter and do date math on them straight away. NUMERIC arrives as exact text; retype it with Manage Columns if you want a fixed decimal. SUPER, GEOMETRY and VARBYTE still need a cast in SQL (for example payload::varchar).
• A query that returns no rows still passes its columns downstream, so a Union or Join below it keeps working.
• The browser lists tables the catalog knows about; Spectrum (external) tables may not appear. Type the name directly and it still reads fine.
• A Serverless workgroup wakes up when it's connected to, so the first test after an idle period takes noticeably longer than the rest.
• The query is Redshift SQL, the source's own dialect. This is a source query, not a Trinium row expression.
• Bookmark: each Live run reads only the rows whose bookmark column is past where the last SUCCESSFUL run stopped. A failed or cancelled run never moves it, so after an outage one run reads everything that was missed. Staging and Production keep their own; a Test run always reads from Start from and never moves it.
• The bookmark column must move forward whenever a row is added or changed: an updated_at stamp, or an ever-increasing id. A row whose bookmark column is empty is never read, and a row DELETED in the database is never seen, so reload the table now and then if deletes matter.
• With a hand-written query, write WHERE updated_at >= ${bookmark} where the value goes. Editing the table, query, connection or column keeps the bookmark, and the run's Messages say when it last read something else. A Bookmark column of another kind (a date where it was a number) is refused until you Set or Reset it in the node's Bookmark panel.
• Each value of a text parameter the pipeline uses keeps its own bookmark, so region = EU and region = US each read only their own new rows. A number or true/false parameter in the table name is refused, because one bookmark would serve every value; make it a text parameter with a list of choices. A date in the table name (events_${date:%Y%m%d}) carries the bookmark into each new day's table, so bookmark a date-time column there.
• When the pipeline uses a text parameter anywhere, such as ${region} in this query or in a Filter after it, each value keeps its own bookmark, so a run for one region never moves another's. A value's first run reads from Start from, and date, number and true/false parameters share one bookmark. Starting or stopping using a text parameter pauses the bookmark the same way, until you Set where the value starts or Reset the old bookmark.
• Lookback (date-time columns only) re-reads the N minutes before where the last successful run stopped, to catch rows a long transaction committed late. A first run and every Test run read exactly from Start from. Re-read rows load again, so use it with an Upsert output.

Related nodes