S3 File

Write the pipeline's output to an S3 bucket with a templated key (dates, run id) and a write mode.

Category: Output
Connection: Amazon S3

What it does

Writes the pipeline's result straight to an Amazon S3 bucket through a saved Connection, instead of leaving a file to download. You choose the format (CSV, JSON, Parquet or Excel), the destination key, and what should happen if something is already sitting there. The key can carry tokens that are filled in at run time, so a nightly job writes a new object every night without anyone editing the name.

Inputs and outputs

Inputs	One.
Outputs	None. This node ends a pipeline.

When to use it

Publish transformed data to a consumption bucket for another team
Write a dated or per-run partition every night without hand-editing the key
Land results where a warehouse, a lake table or a downstream job will pick them up

Examples

Dated partition

consumption/sales/${date}/orders.parquet

${date} becomes 2026-08-26: one object a day, and no collisions.

Per-run object

consumption/sales/${run_id}.csv

Every run writes its own object, so nothing is ever overwritten.

Per-value

consumption/${region}/orders.csv

One template, one object per value: ${region} is the value of the pipeline parameter this run was started with.

A fixed “latest”

consumption/current/orders.parquet · If it exists: Overwrite

For something downstream that always reads from the same object.

Tips

The format settings are the same as on the CSV File, JSON File, Parquet File and Excel File outputs: the delimiter, the JSON layout and the Parquet compression.
Excel writes a real .xlsx with the same settings as Excel Report: a sheet name, a styled table, columns sized to their content, a frozen header, number formats and a chart. Excel's own limit is 1,048,576 rows per sheet; with Partition by, each folder gets its own workbook.
Tokens filled in at run time: ${date} (today in the instance timezone, 2026-08-26), ${date-1} / ${date+7} (a day offset), ${date:%Y%m%d} (a layout), ${datetime} (UTC, filename-safe) and ${run_id}. The same tokens, except ${run_id}, work in an input node's prefix, so a daily schedule can read yesterday's folder.
If it exists: Overwrite replaces it; Fail stops the run and names the key; Unique suffix writes alongside it with a token added before the extension.
Writes happen after a successful Staging or Production run. A Test run never touches the bucket, so you can press Test as often as you like.
An upload failure fails the run. Each file arrives whole or not at all; files that arrived before the failure stay where they are, and the run's error lists which files arrived and which did not, so you know what a re-run will send again.
Without Partition by, the whole result goes to a SINGLE object. With it, rows are routed into column=value/ folders beneath the destination (which then ends in /), the partition columns are left out of the files, and sorted input gives one file per folder. This is the Hive layout Athena, Spark and Trino read.
Replace the folder: replaces the folder(s) this run writes into. With Partition by, that is each partition folder written; without it, the destination's parent folder. What was there is removed only once every new file has arrived, so a run that fails part way leaves the old files in place. Re-running a late-arriving day then replaces that day and nothing else. A destination at the bucket root is refused.
Querying the partitioned folders from Athena? Partition projection lets Athena see each new folder with no crawler. See Help → Guides → Athena without crawlers in the app.