REST API (Read)

Read records from an HTTP/JSON API, following its pagination to the end.

Category

Input

Connection

REST API

Also known as

http, api, web service, endpoint, json, paginate, pagination, webhook, graphql, feed

What it does

Rows come FROM the API. It calls one endpoint (through a saved REST API Connection), follows the pagination to the end, and streams the response records in as typed rows. Nested objects become dotted columns (user.name), arrays come through as JSON text. Declare the columns (the “Fetch sample” button fills them in from a real call) and the builder shows them WITHOUT a run. Which of the three REST nodes do you want? • REST API (Read): rows come FROM the API; one fixed request, then pagination. • API Lookup: rows come from somewhere else and the API ADDS A COLUMN to each one. • REST API (Write): rows go TO the API and nothing comes back.

Inputs and outputs

When to use it

• Load records from a product, partner, or internal API
• Pull a paginated feed on a schedule
• Read an endpoint that has no database or file export

Examples

path: orders · records path: data.items

Pagination: Page number · page parameter: page

Next-cursor path: paging.next.after · parameter: after

Method: POST · body: { "filter": { "status": "paid" } }

Tips

• The URL is FIXED: you type it once and every run asks the same question. If the URL has to change per row (customers/${id}), that is API Lookup, in the Lookup tab.
• The base URL and every credential live on the Connection, so a pipeline never carries a key. A request only ever goes to that connection's own host: a full URL on another host is refused, so make a connection for that host instead.
• Column types are decided by the FIRST page, then enforced: a later page that disagrees fails loudly rather than mistyping the column.
• Rate limits (429), server errors (5xx) and dropped connections are retried automatically, honouring Retry-After, because a read is safe to ask again. A page whose response breaks part way is retried, and fails the run if it never arrives whole; it is never taken as the end of the data.
• “Stop after” caps the total records read, handy while you're still shaping the pipeline.
• For Jira, HubSpot, Zendesk or Salesforce use their own Input nodes instead: the endpoint, paging and records path are pre-filled, and their connection Test actually checks the credential rather than just the address.

Related nodes