Trinium
Start free trial
Home Product Security Nodes & connectors Docs Use cases Pricing FAQ About us Start free trial
Browse the docs
Input node

CSV Files

Read rows from a CSV file and stream them into the pipeline.

Category
Input

What it does

Reads rows from one or more CSV files in the pipeline's file library and streams them into the pipeline. Column types are detected from the first 10,000 rows, and you can set the delimiter, header, encoding and rows to skip for files that came from Excel or older systems.

Inputs and outputs

InputsNone. This node starts a pipeline.
OutputsOne.

When to use it

  • Load a plain-text .csv or .tsv export
  • Combine several CSVs into one stream
  • Read a file that isn't UTF-8 or uses an unusual delimiter

Examples

Standard CSV

Comma-delimited, first row is a header

The default: just pick the file.

Tab-separated export

Delimiter: Tab

For .tsv files.

Skip a title banner

Skip rows: 3

Ignores 3 lines above the header row.

Legacy encoding

Encoding: Windows-1252

For files from older Windows tools.

Late-changing column

Inference rows: 0

Reads every row before deciding types. Slower on big files.

Tips

  • Several files are stacked into one stream. The first file that can be read sets the columns, and later files are matched to it BY NAME: a missing column is filled with nulls, an extra one is dropped, and a file with no columns in common is skipped. Every difference is named in the run's Messages.
  • A file that cannot be read (corrupt, or not really a CSV) is skipped and named, wherever it sorts, and an empty file is passed over. Under Stop the run, an unreadable file stops the run. If no file can be read, the run fails and names the file.
  • Change that with “If a file is different”: Skip reads only files matching the first exactly, and Stop the run fails on any difference.
  • Turn on “Add source file name as column” to track which file each row came from.
  • Types come from the first 10,000 rows. A value further down that does not fit its column's type stops the run with a message naming it: raise Inference rows (0 reads the whole file), or set the column types yourself.
  • Set the column types yourself if a code like 00123 must stay text. That skips detection entirely and is the fastest option on a large file.
  • Turn on 'Run each file separately' to process every file as its own run, handy when one bad file shouldn't fail the rest. Every start does this, a schedule and a flow step included. A Test run still reads them together. The runs take turns, one at a time.
CSV File