Connector
Apache Iceberg
Read and write Apache Iceberg tables through the AWS Glue Data Catalog, Amazon S3 Tables, or an Iceberg REST catalog such as Apache Polaris.
Settings
What the connection form asks for. Every connection has a Staging side, which it needs, and an optional Production side with values of its own; both ask for the same settings.
| Catalog Type | REST catalog: Polaris, Lakekeeper, Nessie, Unity, Snowflake Open Catalog and any other Iceberg REST catalog. AWS Glue Data Catalog: the catalog Athena, EMR and Redshift read Iceberg tables through. Amazon S3 Tables: AWS's managed Iceberg table buckets. Choices: REST catalog; AWS Glue Data Catalog; Amazon S3 Tables. |
| Catalog URI | The Iceberg REST catalog base URL, e.g. https://catalog.example.com or http://localhost:8181. Shown when Catalog Type is REST catalog. |
| Warehouse | REST: optional, the warehouse identifier some catalogs require. Glue: required, the s3://bucket/path new tables are written under. Shown when Catalog Type is REST catalog or AWS Glue Data Catalog. |
| Catalog ID | Optional. The AWS account id that owns the Glue catalog, to use another account's catalog. Leave blank for this account's own. Shown when Catalog Type is AWS Glue Data Catalog. |
| Table Bucket ARN | The table bucket, e.g. arn:aws:s3tables:us-east-1:123456789012:bucket/lake. Shown when Catalog Type is Amazon S3 Tables. |
| Catalog Endpoint | Optional. A VPC endpoint for the catalog API. Leave blank to use AWS's own endpoint for the region. Shown when Catalog Type is AWS Glue Data Catalog or Amazon S3 Tables. |
| Method | How the catalog authenticates: a static bearer token, or OAuth2 client credentials (Polaris uses OAuth2). Choices: Bearer token; OAuth2 client credentials; No authentication. Shown when Catalog Type is REST catalog. |
| Bearer Token | The catalog API token. Leave blank only if the catalog accepts unauthenticated calls. Shown when Method is Bearer token, and Catalog Type is REST catalog. Stored encrypted, and never shown again after you save it. |
| Client ID | The OAuth2 client (service principal) id. Leave blank for catalogs that issue a secret-only credential. Shown when Method is OAuth2 client credentials, and Catalog Type is REST catalog. |
| Client Secret | Shown when Method is OAuth2 client credentials, and Catalog Type is REST catalog. Stored encrypted, and never shown again after you save it. |
| Token URL | Leave blank to use the catalog's own token endpoint. Set it when tokens come from an external identity provider (Keycloak, Okta, Entra). Shown when Method is OAuth2 client credentials, and Catalog Type is REST catalog. |
| Scope | Leave blank to request the default 'catalog' scope. Polaris wants PRINCIPAL_ROLE:ALL. Shown when Method is OAuth2 client credentials, and Catalog Type is REST catalog. |
| Audience | Optional, and needed only by identity providers that require an audience or resource claim. Shown when Method is OAuth2 client credentials, and Catalog Type is REST catalog. |
| Resource | Shown when Method is OAuth2 client credentials, and Catalog Type is REST catalog. |
| Credentials | Access key: this connection's own key; for Glue and S3 Tables it signs the catalog calls too. Vended by the catalog: a REST catalog hands out short-lived credentials for each table (Glue and S3 Tables do not vend). The server's IAM role: use the server's own identity, on admin-managed shared connections only. Choices: Access key; Vended by the catalog; The server's IAM role. |
| S3 Endpoint | Set for S3-compatible warehouses (MinIO, Cloudflare R2, Wasabi). Leave blank for AWS S3. |
| Region | The AWS region. Required with an access key and for Glue; S3 Tables reads it from the table bucket's ARN when left blank. |
| Access Key ID | Shown when Credentials is Access key. |
| Secret Access Key | Shown when Credentials is Access key. Stored encrypted, and never shown again after you save it. |
Testing the connection
Test lists the catalog's top-level namespaces.
Good to know
- Three kinds of catalog: a REST catalog, the AWS Glue Data Catalog, and Amazon S3 Tables. Tables Trinium writes through Glue can be queried in Athena.
- How the data files are reached is a separate choice: this connection's own access key, short-lived credentials handed out by a REST catalog for each table, or the server's IAM role. The server's role is offered only on a shared connection an administrator manages.
- Part of Enterprise, as are the Iceberg input and output nodes.
Nodes that use it
Apache Iceberg input
Read an Apache Iceberg table from AWS Glue, S3 Tables or a REST catalog, with time-travel and column pruning. Its columns show without a run.
Apache Iceberg output
Write rows to an Apache Iceberg table through AWS Glue, S3 Tables or a REST catalog, appending or fully refreshing, with auto-create and partitioning. No crawler needed.