# mountOS Skill: Iceberg lake (query engines over the REST catalog)

> Point a query engine (DuckDB, Trino, Spark, PyIceberg) at a mountOS iceberg volume. mountOS serves an Iceberg REST catalog plus an S3 data endpoint from the `mountos` client; engines read and write tables directly, no Hive Metastore or external catalog.

This skill is for iceberg (lake) volumes. For general object volumes over S3 (and the
multipart part-size rule that also applies here), see https://mountos.io/skills/s3.md. To create a
volume or a key first, see https://mountos.io/skills/volumes.md.

## How it works

Mounting an iceberg-typed volume with `mountos` starts two local listeners and a read-only
catalog tree at the mount path:

- an **Iceberg REST catalog** (default `:18181`)
- a **lake S3 data endpoint** (default `:18282`) for the table data and metadata files

Both authenticate with the volume access key pair (`apiKey` / `apiSecret`). The catalog is
read-write for tables; the data plane is AWS SigV4-compatible (service `s3`). The general S3
gateway (`mountos gateway --gateway s3`) does not serve iceberg volumes; reach them through
this lake endpoint.

## Connection essentials (every engine)

- **Catalog URI is the REST base** (for example `http://host:18181`), NOT `.../v1/<volume>`.
  The Iceberg client appends `/v1/config` and the rest of the path itself. The catalog returns
  the volume name as `overrides.prefix`, so the client builds `/v1/<volume>/...` automatically.
  Baking the volume into the URI yields `.../v1/<volume>/v1/config` and a 404.
- **Auth is OAuth2 `client_credentials`**, the access key as `client_id` and the secret as
  `client_secret` (both HTTP Basic and form-body are accepted). Token endpoint
  `/v1/oauth/tokens`.
- **The S3 data endpoint is auto-discovered** from the catalog's `/v1/config` response
  (`s3.endpoint`), so engines do not need a separate endpoint setting for data files.
- **Warehouse** is `s3://<volume>/`.

## Per-engine configuration

### DuckDB (iceberg + httpfs extensions)

```sql
INSTALL iceberg; LOAD iceberg; INSTALL httpfs; LOAD httpfs;
CREATE SECRET mos (TYPE ICEBERG, CLIENT_ID '<apiKey>', CLIENT_SECRET '<apiSecret>',
                   OAUTH2_SERVER_URI 'http://<host>:18181/v1/oauth/tokens');
ATTACH '<volume>' AS lake (TYPE ICEBERG, SECRET mos, ENDPOINT 'http://<host>:18181');
CREATE SCHEMA lake.demo;
CREATE TABLE lake.demo.orders (id BIGINT, name VARCHAR);
INSERT INTO lake.demo.orders VALUES (1,'a'),(2,'b');
SELECT * FROM lake.demo.orders;
```

`ENDPOINT` is the REST base (no `/v1`). The S3 endpoint comes from `/v1/config`.

### PyIceberg

```python
from pyiceberg.catalog import load_catalog
cat = load_catalog("mos", **{
    "type": "rest",
    "uri": "http://<host>:18181",
    "warehouse": "s3://<volume>/",
    "credential": "<apiKey>:<apiSecret>",
})
```

PyIceberg picks up the prefix and S3 endpoint from `/v1/config`.

### Trino (iceberg connector, native S3)

`etc/trino/catalog/iceberg.properties`:

```properties
connector.name=iceberg
iceberg.catalog.type=rest
iceberg.rest-catalog.uri=http://<host>:18181
iceberg.rest-catalog.security=OAUTH2
iceberg.rest-catalog.oauth2.server-uri=http://<host>:18181/v1/oauth/tokens
iceberg.rest-catalog.oauth2.credential=<apiKey>:<apiSecret>
iceberg.rest-catalog.warehouse=s3://<volume>/
iceberg.unique-table-location=false
fs.native-s3.enabled=true
s3.endpoint=http://<host>:18282
s3.path-style-access=true
s3.aws-access-key=<apiKey>
s3.aws-secret-key=<apiSecret>
s3.region=us-east-1
```

Two Trino-specific settings matter:

- **`iceberg.unique-table-location=false`** is required. mountOS materializes a table at the
  canonical `s3://<volume>/<namespace>/<table>` location. Trino's default appends a random UUID
  to the location, which the lake data endpoint does not resolve, so writes fail. Disabling it
  makes Trino use the canonical location.
- Views work with default settings. mountOS advertises the view endpoints in `/v1/config`, so
  Trino drives CREATE VIEW, CREATE OR REPLACE VIEW, and DROP VIEW with no extra configuration.

### Spark (iceberg runtime, S3FileIO)

Spark session config (equivalently `--conf` flags or `spark-defaults.conf`):

```properties
spark.sql.extensions=org.apache.iceberg.spark.extensions.IcebergSparkSessionExtensions
spark.sql.catalog.mos=org.apache.iceberg.spark.SparkCatalog
spark.sql.catalog.mos.type=rest
spark.sql.catalog.mos.uri=http://<host>:18181
spark.sql.catalog.mos.warehouse=s3://<volume>/
spark.sql.catalog.mos.credential=<apiKey>:<apiSecret>
spark.sql.catalog.mos.io-impl=org.apache.iceberg.aws.s3.S3FileIO
spark.sql.catalog.mos.s3.endpoint=http://<host>:18282
spark.sql.catalog.mos.s3.path-style-access=true
spark.sql.catalog.mos.s3.region=us-east-1
spark.sql.catalog.mos.s3.access-key-id=<apiKey>
spark.sql.catalog.mos.s3.secret-access-key=<apiSecret>
spark.sql.defaultCatalog=mos
```

One Spark-specific setting matters:

- **`s3.region` is required** (any value, for example `us-east-1`). Spark's `S3FileIO` uses the
  AWS SDK v2, which refuses to start without a region even against a custom endpoint. DuckDB,
  Trino, and PyIceberg supply a default, Spark does not. Add `spark.sql.catalog.mos.client.region`
  as well for older `iceberg-aws` builds. With `s3.path-style-access=true` the region value is
  otherwise unused.

## Capabilities and limits

- **Tables: full read and write.** Create, insert, select, delete, and CREATE TABLE AS SELECT
  (CTAS) all work, including engines that write data and manifests before committing the table.
- **Multipart writes** follow the same 8 MiB minimum part size as the general S3 surface
  (https://mountos.io/skills/s3.md). Default part sizes for DuckDB, Trino, and PyIceberg are already
  above the floor.
- **Views: full support (SQL views).** Create, load, replace (CREATE OR REPLACE), rename, and
  drop all work. A view stores one SQL definition per dialect; engines that send a query in their
  own dialect (Trino, Spark) read it back unchanged. View definitions are versioned, each replace
  adds a version and the catalog keeps the history.
- **Server-side scan planning is not supported.** Engines plan scans client-side (the default
  for DuckDB, Trino, Spark, and PyIceberg), so this is not a functional limit.
- **Time travel** is available through table snapshots and through mountOS forks of the volume
  (a fork gives a point-in-time view of the catalog and data; see
  https://mountos.io/skills/volumes.md).

## Read next

- S3 client tuning and the multipart rule: https://mountos.io/skills/s3.md
- Create and use volumes, keys, and forks: https://mountos.io/skills/volumes.md
- Topic index: https://mountos.io/llms.txt
- Support: support@mountos.io and https://mountos.io/support