API Batch Lookup

Send your rows to an API many at a time and add its answers as new columns.

Category

Lookup

Connection

REST API

Also known as

python, lambda, function, script, udf, custom code, model, predict, score, machine learning, llm, ai, batch, bulk, enrich, api, http

What it does

API Lookup's twin for code and models you run yourself. Each request carries up to a hundred rows (or as many as you set) and the answer carries one entry per row, in the same order, which becomes new columns on those rows. Same rows out, wider. Say you have a churn model behind a function, an AWS Lambda with a Function URL, an Azure Function or a container, and a million customers to score. API Lookup would call it a million times; this node calls it ten thousand times, four requests at once, and each row gets its own score back. The function can be written in any language, with any library, and runs on your own infrastructure: Trinium only sends it rows and reads its answers. The request is {"rows": [row, row, …]}, each row an object of its column values. The answer is {"rows": [answer, answer, …]}, one answer per row in the order they were sent, each an object holding the declared answer columns. Both field names can be changed, so an existing model endpoint works as it is: a Databricks model serving endpoint takes dataframe_records and answers predictions. Which one? • API Lookup: one request per row, built from that row's values (an id in the path, a body template); for someone else's API. • API Batch Lookup: many rows per request, one answer per row; for your own function, a hosted model or an LLM gateway. • REST API (Write): rows go out and nothing comes back.

Inputs and outputs

When to use it

• Score rows with a model you have deployed: your own, SageMaker or Bedrock behind a function, Databricks model serving
• Run your own Python, or any language, on every row, without Trinium running the code
• Price, classify or check rows with your company's own library behind a function
• Send text to an LLM through your own gateway and keep each row's answer

Examples

Path: score · Rows per request: 100 · Answer columns: score (Number)

Path: serving-endpoints/churn/invocations · Rows go in: dataframe_records · Answers are at: predictions · one answer column

import json

def handler(event, context):
    if event.get("headers", {}).get("x-api-key") != "your-secret":
        return {"statusCode": 401, "body": ""}
    rows = json.loads(event["body"])["rows"]
    answers = [{"score": len(r["name"]) / 10} for r in rows]
    return {"statusCode": 200, "body": json.dumps({"rows": answers})}

Tips

• Choose what the endpoint does. Only calculates: a Test run calls it, so the preview shows real answers, and a request cut off by the network is sent again. Creates, changes or sends something: a Test run sends nothing, and a request that may already have reached the endpoint is never sent again, so nothing happens twice.
• Every row must get exactly one answer, in the order sent. A request whose answer holds more or fewer fails the run naming its rows, or, with If a request fails set to continue, keeps those rows with empty answer columns.
• To report a problem with one row, have the function answer that row with a column of its own, such as error, and route those rows with a Filter. An answer of {"error": "…"} means the whole request failed.
• A request is kept under 5 MB: when rows are wide, it carries fewer of them, and the run says so. Send only the columns the function needs (Manage Columns before this node) to keep requests small.
• Batching saves the cost of each call, not the work on each row. A function that calls an LLM once per row, one row at a time, takes as long here as it would one row per request; let it work on its rows in parallel.
• A decimal goes out with every digit, as a JSON number (in Python, json.loads(body, parse_float=Decimal) keeps them). Coming back, answer a decimal column as text, "12345.6789", to keep every digit; a JSON number is read as a floating-point value first, which holds about 15 significant digits.
• If a Function URL has no sign-in of its own, anyone who knows its address can call it, so have the function check a secret header, as the example does.
• The path is the same on every request. To put a row's value in the path or build a body from a template, use API Lookup.

Related nodes