S3

​

Overview

S3 foreign tables have been tested against the following object stores that implement the S3 API: Amazon S3, MinIO, Cloudflare R2, and Google Cloud.

This code block demonstrates how to create a foreign table over a Parquet file stored in Google Cloud Storage (GCS). The file path must start with s3, r2, or gs.

-- Parquet format is assumed
CREATE FOREIGN DATA WRAPPER parquet_wrapper
HANDLER parquet_fdw_handler
VALIDATOR parquet_fdw_validator;

CREATE SERVER parquet_server
FOREIGN DATA WRAPPER parquet_wrapper;

CREATE FOREIGN TABLE parquet_table ()
SERVER parquet_server
OPTIONS (files 's3://<bucket>/<path>/<file>.parquet');

The glob pattern can be used to query a directory of files.

CREATE FOREIGN TABLE parquet_table ()
SERVER parquet_server
OPTIONS (files 's3://<bucket>/<path>/*.parquet');

​

Providing Credentials

CREATE USER MAPPING is used to provide S3 credentials. These credentials are tied to a specific Postgres user, which enables multiple users to query the same foreign table with their own credentials.

CREATE USER MAPPING FOR <current_user>
SERVER <server_name>
OPTIONS (
  type 'S3',
  key_id '<key_id>',
  secret '<secret>',
  region 'us-east-1'
);

​

Credentials Options

Below is a complete list of options for CREATE USER MAPPING.