Delta Sharing Server – Operations Guide

Overview

The Delta Sharing Server is an open-source implementation of the Delta Sharing protocol, maintained by the Delta Lake community. WebShield uses this server to share EA Consent Delta Tables with Privacy Network participants.

Key characteristics:

  • Exposes Delta tables via REST API without copying data
  • Token-based authorization
  • Currently supports S3 storage backend

Docker Image

The Delta Sharing Server is available as a Docker image:

# Pull the latest image
docker pull deltaio/delta-sharing-server:latest

Resources:

Configuration

The server is configured via a YAML configuration file that defines shares, schemas, tables, and authorization.

Example Configuration File

Create a file named config.yaml:

version: 1
shares:
  - name: "local-webshield-ea-consent-share"
    schemas:
    - name: "ea-consent-schema"
      tables:
      - name: "ea-consent-tb"
        location: "s3a://webshield-delta-sharing/laptop/ea-consent-schema/ea-consent-tb"
        id: "00000000-0000-0000-0000-000000000001"
      - name: "ea-consent-verification-keys"
        location: "s3a://webshield-delta-sharing/laptop/ea-consent-schema/ea-consent-verification-keys"
        id: "00000000-0000-0000-0000-000000000002"
authorization:
  bearerToken: "your-secure-token-here"

Configuration notes:

  • Replace your-secure-token-here with a strong, unique bearer token
  • Update S3 bucket name and paths to match your environment
  • Each table requires a unique UUID in the id field
  • The s3a:// protocol is used for S3 access

S3 Credentials

The server requires AWS credentials to access S3 buckets. Provide credentials via:

  • Environment variables: AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY
  • IAM role: If running on EC2 or ECS with an IAM role attached
  • AWS credentials file: Mount ~/.aws/credentials into the container

For detailed S3 configuration options, see the official Delta Sharing documentation.

Running the Server

Basic Deployment

Run the server with Docker, mounting the configuration file and providing S3 credentials:

docker run -d \
  --name delta-sharing-server \
  -p 8080:8080 \
  -v $(pwd)/config.yaml:/conf/delta-sharing-server-config.yaml \
  -e AWS_ACCESS_KEY_ID=your-aws-access-key \
  -e AWS_SECRET_ACCESS_KEY=your-aws-secret-key \
  deltaio/delta-sharing-server:latest

Port configuration:

  • 8080 (default): REST API endpoint for Delta Sharing protocol

Important: The configuration file must be mounted at /conf/delta-sharing-server-config.yaml inside the container.

Verification

After starting the server, verify it’s running correctly:

1. List Available Shares 

curl -H "Authorization: Bearer your-secure-token-here" \
  http://localhost:8080/shares

Expected response:

{
  "items": [
    {
      "name": "local-webshield-ea-consent-share"
    }
  ]
}

2. List Schemas in Share 

curl -H "Authorization: Bearer your-secure-token-here" \
  http://localhost:8080/shares/local-webshield-ea-consent-share/schemas

Expected response:

{
  "items": [
    {
      "name": "ea-consent-schema",
      "share": "local-webshield-ea-consent-share"
    }
  ]
}

3. List Tables in Schema 

curl -H "Authorization: Bearer your-secure-token-here" \
  http://localhost:8080/shares/local-webshield-ea-consent-share/schemas/ea-consent-schema/tables

Expected response:

{
  "items": [
    {
      "name": "ea-consent-tb",
      "schema": "ea-consent-schema",
      "share": "local-webshield-ea-consent-share"
    },
    {
      "name": "ea-consent-verification-keys",
      "schema": "ea-consent-schema",
      "share": "local-webshield-ea-consent-share"
    }
  ]
}

Security Considerations

  • Bearer Token: Store bearer tokens securely (AWS Secrets Manager, environment variables, etc.)
  • Network Isolation: Deploy in a private network or behind an API gateway
  • TLS/HTTPS: Configure SSL/TLS for production deployments (see official documentation)
  • S3 Permissions: Use IAM roles with least-privilege access to S3 buckets

Official Documentation

For advanced configuration, SSL/TLS setup, monitoring, and troubleshooting, refer to the official Delta Sharing documentation:

Last Updated: 2026-01-29 22:23:30 UTC