Serving Private S3 Files with CloudFront Signed URLs

TL;DR: Needed to give external clients temporary access to files stored in a private S3 bucket. Built a system using CloudFront signed URLs that generates per-client HTML index pages with expiring download links. No authentication system to maintain, no pre-signed URL juggling — just a Python script, a CloudFront distribution, and a key pair.

I had a bunch of files sitting in a private S3 bucket, organized into directories by client. Each client needed to download their files, but I didn’t want to:

Make the bucket public (obviously)
Give clients AWS credentials
Build a whole web app with authentication just to serve files
Manually generate S3 pre-signed URLs every time someone needed a download

What I wanted was simple: a link per client that shows their files as a nice download page, with every link expiring after a set period. No logins, no credentials, just “here’s your link, it works for 7 days.”

The Architecture

flowchart LR
    S3["S3 Bucket<br/>(private)"] --> CF["CloudFront<br/>(OAC + signed URLs)"]
    CF --> Client["Client Browser"]
    Script["Python Script"] --> S3
    Script --> CF

    style S3 fill:#ff922b,stroke:#e8590c,color:#fff
    style CF fill:#339af0,stroke:#1971c2,color:#fff
    style Script fill:#51cf66,stroke:#2f9e44

The setup has three pieces:

Private S3 bucket with files organized by client directory
CloudFront distribution locked down with a key group — every request must have a valid signature
Python script that lists each client’s files, generates signed URLs, creates an HTML index page, uploads it to S3, and spits out a signed URL to that index page

The client gets one URL. That URL loads a page listing all their files with individual signed download links. Everything expires at the same time.

Step 1: CloudFront Infrastructure

The CloudFront distribution sits in front of the S3 bucket so the bucket stays fully private. There are two ways to grant CloudFront access to S3:

Origin Access Control (OAC) — the newer, recommended approach. Supports all regions, SSE-KMS encryption, and all HTTP methods.
Origin Access Identity (OAI) — the legacy approach. Still works, but AWS recommends migrating to OAC. Doesn’t support SSE-KMS or regions launched after December 2022.

I’m showing OAC below since that’s what you should use for new setups. If you’re on an existing distribution with OAI, see AWS’s migration guide.

Origin Access Control and Bucket Policy

resource "aws_cloudfront_origin_access_control" "oac" {
  name                              = "${var.bucket_name}-oac"
  description                       = "OAC for ${var.bucket_name}"
  origin_access_control_origin_type = "s3"
  signing_behavior                  = "always"
  signing_protocol                  = "sigv4"
}

data "aws_iam_policy_document" "s3_policy" {
  statement {
    actions   = ["s3:GetObject"]
    resources = ["arn:aws:s3:::${var.bucket_name}/*"]

    principals {
      type        = "Service"
      identifiers = ["cloudfront.amazonaws.com"]
    }

    condition {
      test     = "StringEquals"
      variable = "AWS:SourceArn"
      values   = [aws_cloudfront_distribution.s3_distribution.arn]
    }
  }
}

resource "aws_s3_bucket_policy" "bucket_policy" {
  bucket = var.bucket_name
  policy = data.aws_iam_policy_document.s3_policy.json
}

The bucket is locked down — all public access blocked:

resource "aws_s3_bucket_public_access_block" "bucket_access" {
  bucket = var.bucket_name

  block_public_acls       = true
  block_public_policy     = true
  ignore_public_acls      = true
  restrict_public_buckets = true
}

CloudFront Distribution

resource "aws_cloudfront_distribution" "s3_distribution" {
  origin {
    domain_name              = "${var.bucket_name}.s3.${var.region}.amazonaws.com"
    origin_id                = "S3-${var.bucket_name}${var.origin_path}"
    origin_path              = var.origin_path
    origin_access_control_id = aws_cloudfront_origin_access_control.oac.id
  }

  enabled             = true
  is_ipv6_enabled     = true
  default_root_object = "index.html"
  price_class         = "PriceClass_100"

  default_cache_behavior {
    allowed_methods  = ["GET", "HEAD", "OPTIONS"]
    cached_methods   = ["GET", "HEAD"]
    target_origin_id = "S3-${var.bucket_name}${var.origin_path}"

    cache_policy_id = aws_cloudfront_cache_policy.signed_urls.id

    viewer_protocol_policy = "redirect-to-https"

    trusted_key_groups = [aws_cloudfront_key_group.key_group.id]
  }

  restrictions {
    geo_restriction {
      restriction_type = "none"
    }
  }

  viewer_certificate {
    cloudfront_default_certificate = true
  }

  tags = {
    Environment = terraform.workspace
    Purpose     = "client-file-access"
  }
}

# Cache policy that forwards query strings (needed for signed URL parameters)
resource "aws_cloudfront_cache_policy" "signed_urls" {
  name        = "${terraform.workspace}-signed-urls-cache-policy"
  default_ttl = 3600
  max_ttl     = 86400
  min_ttl     = 0

  parameters_in_cache_key_and_forwarded_to_origin {
    cookies_config {
      cookie_behavior = "none"
    }
    headers_config {
      header_behavior = "none"
    }
    query_strings_config {
      query_string_behavior = "all"
    }
  }
}

A few things to note:

trusted_key_groups tells CloudFront that every request must be signed with a key from the specified key group. No valid signature, no access.
query_string_behavior = "all" in the cache policy is important — the signed URL parameters (Policy, Signature, Key-Pair-Id) are passed as query strings, so CloudFront needs to forward them for validation.
origin_access_control_id replaces the legacy s3_origin_config block. With OAC, the bucket policy uses the CloudFront service principal with a condition on the distribution ARN.

Key Pair and Key Group

CloudFront uses RSA key pairs for URL signing. You generate the key pair yourself, upload the public key to CloudFront, and keep the private key safe for signing.

# Generate the key pair
openssl genrsa -out private_key.pem 2048
openssl rsa -pubout -in private_key.pem -out public_key.pem

Then register it with CloudFront:

resource "aws_cloudfront_public_key" "public_key" {
  encoded_key = file("${path.module}/keys/public_key.pem")
  name        = "${terraform.workspace}-client-access-key"
  comment     = "Public key for client directory access"
}

resource "aws_cloudfront_key_group" "key_group" {
  name    = "${terraform.workspace}-client-access-key-group"
  comment = "Key group for client directory access"
  items   = [aws_cloudfront_public_key.public_key.id]
}

Important: Keep the private key out of source control. Store it in a secrets manager or pass it in at runtime. If the private key leaks, anyone can generate valid signed URLs to your files.

Step 2: The Python Script

The script does all the heavy lifting: it walks through each client directory in S3, generates signed URLs for every file, builds an HTML index page, uploads it, and prints out a signed URL to the index page itself.

URL Signing

I’m using the AWS CLI for signing since it handles the RSA signature math:

import subprocess
from datetime import datetime, timedelta

def sign_url(url, key_pair_id, private_key_path, expiry_time):
    """Sign a CloudFront URL using the AWS CLI"""
    try:
        expiry_str = expiry_time.strftime('%Y-%m-%dT%H:%M:%SZ')

        cmd = [
            'aws', 'cloudfront', 'sign',
            '--url', url,
            '--key-pair-id', key_pair_id,
            '--private-key', f'file://{private_key_path}',
            '--date-less-than', expiry_str
        ]

        result = subprocess.run(cmd, capture_output=True, text=True, check=True)
        return result.stdout.strip()
    except subprocess.CalledProcessError as e:
        print(f"Error signing URL: {e.stderr}")
        return url

The key_pair_id is the ID of your CloudFront public key (not the key group). The --date-less-than flag sets when the signed URL expires.

Generating the Index Page

For each client directory, the script generates a clean HTML page with download links:

def generate_index_html(client_id, files, cloudfront_domain,
                        key_pair_id, private_key_path, expiry_time):
    """Generate an HTML index page with signed download links"""

    file_entries = []
    for file_obj in files:
        file_path = file_obj['Key']
        file_name = file_path.split('/')[-1]

        if file_name == "index.html":
            continue

        # Build the CloudFront URL and sign it
        relative_path = file_path.replace('client_files/', '', 1)
        url = f"https://{cloudfront_domain}/{relative_path}"
        signed_url = sign_url(url, key_pair_id, private_key_path, expiry_time)

        file_entries.append({
            'name': file_name,
            'url': signed_url,
            'size': format_size(file_obj['Size']),
            'modified': file_obj['LastModified'].strftime('%Y-%m-%d %H:%M:%S')
        })

    # Build HTML with file listing
    rows = ""
    for entry in file_entries:
        rows += f"""
            <div class="file-item row">
                <div class="col-md-6">
                    <a href="{entry['url']}" download="{entry['name']}">
                        {entry['name']}
                    </a>
                </div>
                <div class="col-md-2 file-size">{entry['size']}</div>
                <div class="col-md-4 timestamp">{entry['modified']}</div>
            </div>"""

    return f"""<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>Files for {client_id}</title>
    <link href="https://cdn.jsdelivr.net/npm/bootstrap@5.3.0/dist/css/bootstrap.min.css"
          rel="stylesheet">
</head>
<body>
    <div class="container" style="padding: 20px;">
        <h1>Files for Client: {client_id}</h1>
        <p class="text-muted">
            Links expire on {expiry_time.strftime('%Y-%m-%d %H:%M:%S')} UTC
        </p>
        <div class="row fw-bold">
            <div class="col-md-6">Filename</div>
            <div class="col-md-2">Size</div>
            <div class="col-md-4">Last Modified</div>
        </div>
        {rows}
    </div>
</body>
</html>"""

Nothing fancy — Bootstrap for styling, a table-like layout with file names as signed download links. Each link includes the download attribute so the browser downloads instead of trying to render the file.

Putting It All Together

The main function walks the S3 bucket, processes each client directory, and outputs the signed URLs:

import boto3
import os

def format_size(size_bytes):
    """Format file size in a human-readable format"""
    for unit in ['B', 'KB', 'MB', 'GB']:
        if size_bytes < 1024:
            return f"{size_bytes:.1f} {unit}"
        size_bytes /= 1024
    return f"{size_bytes:.1f} TB"

def main():
    # Configuration
    cloudfront_domain = '<your-distribution>.cloudfront.net'
    key_pair_id = '<your-cloudfront-public-key-id>'
    private_key_path = 'private_key.pem'
    bucket_name = '<your-bucket-name>'
    prefix = 'client_files/'

    if not os.path.exists(private_key_path):
        print(f"Error: Private key not found at {private_key_path}")
        return

    expiry_time = datetime.utcnow() + timedelta(days=7)

    session = boto3.Session(region_name='us-east-1')
    s3 = session.client('s3')

    # List client directories
    response = s3.list_objects_v2(
        Bucket=bucket_name,
        Prefix=prefix,
        Delimiter='/'
    )

    if 'CommonPrefixes' not in response:
        print("No client directories found")
        return

    for obj in response['CommonPrefixes']:
        client_prefix = obj['Prefix']
        client_id = client_prefix.split('/')[-2]

        print(f"Processing client: {client_id}")

        # List files for this client
        file_response = s3.list_objects_v2(
            Bucket=bucket_name,
            Prefix=client_prefix,
            MaxKeys=1000
        )

        if 'Contents' not in file_response:
            print(f"  No files found for {client_id}")
            continue

        # Generate and upload the index page
        index_html = generate_index_html(
            client_id, file_response['Contents'],
            cloudfront_domain, key_pair_id, private_key_path, expiry_time
        )

        index_key = f"{client_prefix}index.html"
        s3.put_object(
            Bucket=bucket_name,
            Key=index_key,
            Body=index_html,
            ContentType='text/html',
            ServerSideEncryption='AES256'
        )

        # Generate a signed URL for the index page itself
        relative_path = index_key.replace(f'{prefix}', '')
        index_url = f"https://{cloudfront_domain}/{relative_path}"
        signed_index_url = sign_url(
            index_url, key_pair_id, private_key_path, expiry_time
        )

        print(f"  Client: {client_id}")
        print(f"  URL: {signed_index_url}")
        print("-" * 80)

    print(f"\nAll URLs expire on {expiry_time.strftime('%Y-%m-%d %H:%M:%S')} UTC")

if __name__ == "__main__":
    main()

Run it, and you get output like:

Processing client: acme-corp
  Client: acme-corp
  URL: https://dxxxxxxxx.cloudfront.net/acme-corp/index.html?Policy=...&Signature=...&Key-Pair-Id=...
--------------------------------------------------------------------------------
Processing client: globex
  Client: globex
  URL: https://dxxxxxxxx.cloudfront.net/globex/index.html?Policy=...&Signature=...&Key-Pair-Id=...
--------------------------------------------------------------------------------

All URLs expire on 2026-02-14 12:00:00 UTC

Send each client their URL. They open it, see their files, click to download. Done.

How It Works End to End

sequenceDiagram
    participant Op as Operator
    participant Script as Python Script
    participant S3 as S3 Bucket
    participant CF as CloudFront
    participant Client as Client

    Op->>Script: Run script
    Script->>S3: List client directories
    S3-->>Script: Directory listing
    loop For each client
        Script->>S3: List files in directory
        S3-->>Script: File listing
        Script->>Script: Generate signed URLs<br/>+ build HTML index
        Script->>S3: Upload index.html
        Script->>Script: Sign index.html URL
        Script-->>Op: Print signed URL
    end
    Op->>Client: Send signed URL
    Client->>CF: Open URL (with signature)
    CF->>CF: Validate signature
    CF->>S3: Fetch index.html (via OAC)
    S3-->>CF: index.html content
    CF-->>Client: Rendered page with download links
    Client->>CF: Click download (signed URL)
    CF->>S3: Fetch file (via OAC)
    S3-->>CF: File content
    CF-->>Client: File download

The flow is:

Script runs — walks S3, generates signed URLs for each file, builds HTML index pages
Index pages uploaded to S3 inside each client’s directory
Operator gets a signed URL per client pointing to their index page
Client opens the URL — CloudFront validates the signature, serves the page via OAC
Client downloads files — each download link is also a signed URL with the same expiry

Every URL in the chain is signed. The index page URL is signed. Each individual download link inside the page is signed. If any of them expire, access stops.

S3 Bucket Structure

The bucket layout is straightforward:

<bucket>/
  client_files/
    acme-corp/
      report-2026-01.zip
      report-2026-02.zip
      index.html          ← generated by script
    globex/
      data-export.csv
      audit-log.zip
      index.html          ← generated by script

The client_files/ prefix maps to the CloudFront origin_path, so CloudFront URLs don’t include it. A file at client_files/acme-corp/report.zip becomes https://<distribution>.cloudfront.net/acme-corp/report.zip.

Encryption Note

If you’re using OAC (as shown above), SSE-KMS encryption works out of the box — OAC supports it natively. Just make sure the KMS key policy allows the cloudfront.amazonaws.com service to use the key.

If you’re on the legacy OAI, SSE-KMS doesn’t work. The OAI doesn’t have permission to use the KMS key, and there’s no clean way to grant it. You’ll need to use S3-managed encryption (SSE-S3/AES256) instead:

s3.put_object(
    Bucket=bucket_name,
    Key=index_key,
    Body=index_html,
    ContentType='text/html',
    ServerSideEncryption='AES256'  # SSE-S3, not KMS
)

This is one of the reasons AWS recommends migrating to OAC — it removes the encryption headache entirely. If you’re on OAI and hitting “Access Denied” errors that don’t make sense, check if the bucket has KMS encryption enabled. That one cost me a few hours.

Security Considerations

All URLs expire: both the index page and individual download links have the same expiry. After the TTL, everything stops working
No public access: the S3 bucket is fully locked down, CloudFront is the only way in, and every CloudFront request requires a valid signature
Per-client isolation: each client only sees their own directory. There’s no way to enumerate other clients or their files
Private key security: the private key used for signing should never be committed to source control. Store it in a secrets manager and inject it at runtime
CloudFront cache: be aware that CloudFront caches responses. If you regenerate URLs with a shorter expiry, cached pages with the old URLs might still be served until the cache TTL expires. Consider invalidating the cache after regenerating

Wrapping Up

This pattern works well when you need to share files with external parties without building a full authentication system. The total infrastructure cost is minimal — a CloudFront distribution and an S3 bucket you probably already have. The Python script runs in seconds even for dozens of clients with hundreds of files.

The nice thing is it’s stateless. There’s no database of users, no session management, no password resets. Generate the URLs, send them out, they expire on schedule. Need to revoke access early? Rotate the key pair and all existing URLs become invalid instantly.

The Problem: Sharing Private Files Without Sharing Access