S3 / R2 — Integration

Overview

Import CSV/Parquet/JSONL from S3-compatible storage and export scheduled datasets to your buckets with file manifests.

Capabilities

• Ingest from prefixes with schema inference & previews

• Scheduled exports with partitioned folders (dt=YYYY-MM-DD)

• Signed URLs and server-side encryption

• Checksum manifests for completeness

1. 1

   Step 1

   Create an IAM user with limited bucket access; copy Access Key/Secret.

   Back to top
2. 2

   Step 2

   Add a connection in InsightLake with endpoint/region and credentials.

   Back to top
3. 3

   Step 3

   Register import prefixes and map to staging tables.

   Back to top
4. 4

   Step 4

   Configure export jobs and choose formats (CSV/Parquet/JSONL).

   Back to top
5. 5

   Step 5

   Run a test import/export and verify files in your bucket.

   Back to top

Limitations

• Very large single files are multipart-processed; ensure part size limits fit your provider.

• Cross-account buckets require proper bucket policies.

FAQs

Pricing