SSL Required: Getting CA Certificates Through AWS, Azure, and GCP Secret Managers Intact
TL;DR: Every cloud’s managed databases can enforce SSL, and for real server verification your pods need the CA certificate. The hard part isn’t the database - it’s getting a multi-line PEM file through a secret manager without destroying it. Inside JSON, newlines must be escaped as
\n. In AWS Secrets Manager the certificate goes in the Plaintext tab, not the key/value editor. In Azure Key Vault and GCP Secret Manager a secret is a single value, so the PEM can live on its own or as a field in a JSON map. External Secrets Operator turns all three into a normal Kubernetes Secret you can mount. And trust root CAs only - never intermediates or server certificates.
The Error That Starts It
Sooner or later a managed database gets locked down to encrypted connections only - usually because of a compliance audit, sometimes because it was never optional in the first place. Azure Database for PostgreSQL, for example, requires TLS on every connection by default. Then the apps start failing:
# Azure Database for PostgreSQL
FATAL: SSL connection is required. Please specify SSL options and retry.
# MySQL with require_secure_transport=ON (Azure, RDS, self-hosted)
Connections using insecure transport are prohibited while --require_secure_transport=ON.
# Amazon RDS for PostgreSQL with rds.force_ssl=1
FATAL: no pg_hba.conf entry for host "10.0.3.17", user "app", database "app", SSL off
The quick fix is sslmode=require. That encrypts the connection, but it doesn’t verify who you’re talking to - any server with any certificate is accepted. Azure’s own docs list require under “supported, but not recommended” because it leaves you open to man-in-the-middle attacks. If you’re doing this for SOC 2 or ISO 27001, the auditor wants the client to verify the server:
| Mode | Encrypts | Verifies CA | Verifies hostname |
|---|---|---|---|
sslmode=require | yes | no | no |
sslmode=verify-ca | yes | yes | no |
sslmode=verify-full | yes | yes | yes |
MySQL’s equivalents are --ssl-mode=VERIFY_CA and --ssl-mode=VERIFY_IDENTITY.
Use verify-full where you can. With private networking it isn’t always possible: Azure notes that private endpoints and VNet integration can prevent hostname verification, and Cloud SQL instances using the per-instance CA only support hostname verification through Private Service Connect. In those cases verify-ca is the fallback - it still proves the certificate came from the right CA.
Either way, every pod needs the database’s CA certificate. Downloading it is easy. Getting it into a secret manager and back out in one piece is where things go wrong.
The Flow
flowchart LR
CA[Root CA bundle<br/>ca.pem] -->|escape newlines<br/>if stored in JSON| SM[(Secret manager<br/>AWS / Azure / GCP)]
SM -->|External Secrets<br/>Operator| KS[Kubernetes Secret<br/>ca.pem]
KS -->|volume mount| POD[App pod<br/>/etc/ssl/db/ca.pem]
POD -->|sslmode=verify-full| DB[(Managed database)]
Step 1: Get the Right CA Certificate
AWS RDS and Aurora
AWS publishes a global bundle that covers every commercial region, plus per-region bundles:
curl -sO https://truststore.pki.rds.amazonaws.com/global/global-bundle.pem
The bundle contains all three RDS root CAs (rds-ca-rsa2048-g1, rds-ca-rsa4096-g1, rds-ca-ecc384-g1). The default for new databases is rds-ca-rsa2048-g1, and RDS rotates the server certificate automatically before it expires - yearly on engine versions that support rotation without a restart.
Azure Database for PostgreSQL and MySQL
Azure’s server certificates chain up to two root CAs, and Microsoft recommends trusting both: DigiCert Global Root G2 and Microsoft RSA Root Certificate Authority 2017. The catch: both download as .crt files in DER (binary) format, not PEM. Convert them and combine into one bundle:
curl -sO https://cacerts.digicert.com/DigiCertGlobalRootG2.crt
curl -s -o MicrosoftRSARoot2017.crt \
"https://www.microsoft.com/pkiops/certs/Microsoft%20RSA%20Root%20Certificate%20Authority%202017.crt"
openssl x509 -inform DER -in DigiCertGlobalRootG2.crt -out digicert-g2.pem
openssl x509 -inform DER -in MicrosoftRSARoot2017.crt -out microsoft-2017.pem
cat digicert-g2.pem microsoft-2017.pem > ca.pem
Paste a DER file into a secret manager and you’ll get binary garbage. If head -1 on the file doesn’t print -----BEGIN CERTIFICATE-----, it isn’t PEM yet.
GCP Cloud SQL
Cloud SQL’s CA depends on the instance’s server CA mode:
# Per-instance CA
gcloud sql ssl server-ca-certs list --instance=INSTANCE \
--format="value(cert)" > ca.pem
# Shared CA
gcloud sql ssl server-certs list --instance=INSTANCE \
--format="value(ca_cert.cert)" > ca.pem
During a CA rotation Cloud SQL keeps upcoming, active, and previous certificates side by side. Ship all of them in the bundle until the rotation is done, and clients keep working on both sides of the switch.
Enforce encryption on the instance with:
gcloud sql instances patch INSTANCE --ssl-mode=ENCRYPTED_ONLY
Self-hosted (Hetzner, VMs, on-prem)
Use your own root CA - whatever signed the database’s server certificate. Same storage rules apply.
The rule for every cloud: roots only
Both AWS and Azure are explicit about this: put only root CA certificates in your trust store. Intermediate CAs and server certificates rotate - Azure rotates its intermediates routinely and without announcement (a rotation ran through Q1 2026). If you pinned an intermediate or a server certificate, your app breaks the day it changes. Root changes are announced well ahead of time.
Step 2: The Rule for Every Secret Manager - JSON Means \n
A PEM certificate is multi-line:
-----BEGIN CERTIFICATE-----
MIIDjjCCAnagAwIBAgIQAzrx5qcRqaC7KGSxHQn65TANBgkqhkiG9w0BAQsFADBh
...
-----END CERTIFICATE-----
JSON strings can’t contain raw line breaks. If you store the certificate inside a JSON map, every newline has to become the two characters \n. Don’t do it by hand - let jq do it:
# Turn ca.pem into a correctly escaped JSON string
jq -Rs . ca.pem
# "-----BEGIN CERTIFICATE-----\nMIIDjjCCAnagAwIBAgIQAzrx5qcRqaC7...\n-----END CERTIFICATE-----\n"
# Or build the whole secret in one go
jq -n --rawfile ca ca.pem --arg user app --arg password "$DB_PASSWORD" \
'{username: $user, password: $password, ca_pem: $ca}' > secret.json
--rawfile reads the file as one string and jq escapes it on output. When External Secrets Operator later reads the property, it parses the JSON and the \n sequences become real newlines again.
There are three ways to break this:
- Newlines stripped - the certificate ends up on one line. Nothing can parse it.
- Raw newlines inside JSON - invalid JSON, so the property can’t be extracted.
- Double-escaped - escaped by hand, then escaped again by a tool. The mounted file contains a literal
\ninstead of line breaks.
Step 3a: AWS Secrets Manager - Use the Plaintext Tab
A Secrets Manager secret is a single string (up to 64 KB). The console gives you two ways to edit it: a Key/value editor and a Plaintext tab. The key/value editor is built for short, single-line values - paste a multi-line PEM into a value field and the line breaks don’t survive. AWS’s own docs show certificates and private keys entered as plaintext.
You have two layouts to choose from, both in the Plaintext tab:
Raw PEM on its own - paste the certificate exactly as it is. No JSON, no escaping.
JSON map with the other connection details - paste the output of the jq command above:
{
"username": "app",
"password": "...",
"ca_pem": "-----BEGIN CERTIFICATE-----\nMIIDjjCCAnagAwIBAgIQAzrx5qcRqaC7...\n-----END CERTIFICATE-----\n"
}
Or skip the console entirely:
aws secretsmanager create-secret \
--name production/app/db \
--secret-string file://secret.json # or file://ca.pem for the raw layout
The file:// prefix sends the file exactly as written - no shell quoting surprises.
One thing to watch: if you open a JSON secret in the console and click back into the Key/value view to “just fix one field”, you’re editing the certificate in a single-line field again. Keep certificate secrets in the Plaintext tab for their whole life.
Step 3b: Azure Key Vault and GCP Secret Manager - One Value, Two Options
In Azure Key Vault and GCP Secret Manager, a secret is one value in one place. No key/value editor, no tabs. Same two layouts:
Option A - a dedicated secret holding just the PEM. No escaping at all, because there’s no JSON. Upload the file directly so the newlines are kept:
# Azure Key Vault (secret names: letters, numbers, and dashes only)
az keyvault secret set --vault-name my-vault --name app-db-ca --file ca.pem
# GCP Secret Manager
gcloud secrets create app-db-ca --data-file=ca.pem
Option B - a JSON map with the other connection details. Same rule as AWS: escape with jq first.
az keyvault secret set --vault-name my-vault --name app-db --file secret.json
gcloud secrets create app-db --data-file=secret.json
Option A keeps the certificate readable and easy to rotate on its own. Option B keeps everything for one database in one secret. Pick one convention per project and stick to it.
Azure Key Vault also has a separate Certificates object type. It’s built for certificates with private keys - it manages the key, renewal, and export. For a public root CA bundle that clients only use to verify the server, a plain secret is simpler.
Step 4: Pull It Into Kubernetes With External Secrets
External Secrets Operator works the same way for all three clouds - only the ClusterSecretStore differs. With a JSON map, use property. With a single-value secret, leave it out:
apiVersion: external-secrets.io/v1
kind: ExternalSecret
metadata:
name: app-db
spec:
refreshInterval: 1h
secretStoreRef:
name: cloud-secrets # AWS, Azure, or GCP ClusterSecretStore
kind: ClusterSecretStore
target:
name: app-db
creationPolicy: Owner
data:
# JSON map layout (any cloud)
- secretKey: ca.pem
remoteRef:
key: production/app/db
property: ca_pem
- secretKey: password
remoteRef:
key: production/app/db
property: password
# Single-value layout (any cloud) - no property
# - secretKey: ca.pem
# remoteRef:
# key: app-db-ca
If you stored the certificate base64-encoded instead (some teams do, to sidestep newline problems entirely), add decodingStrategy: Base64 to the remoteRef and ESO decodes it for you.
Then mount it:
volumes:
- name: db-ca
secret:
secretName: app-db
items:
- key: ca.pem
path: ca.pem
containers:
- name: app
volumeMounts:
- name: db-ca
mountPath: /etc/ssl/db
readOnly: true
env:
- name: DATABASE_URL
value: "postgresql://app@db.internal:5432/app?sslmode=verify-full&sslrootcert=/etc/ssl/db/ca.pem"
Mount the directory, not a single file with subPath. Kubernetes never updates subPath mounts when the Secret changes, so a new CA bundle - say, during a Cloud SQL CA rotation - would never reach a running pod.
For MySQL clients the equivalent is --ssl-mode=VERIFY_IDENTITY --ssl-ca=/etc/ssl/db/ca.pem. For Java apps, the bundle needs to be imported into a truststore; the same secret works as the source.
Step 5: Verify Before You Trust It
Check the certificate that actually landed in the cluster, not the one on your laptop:
kubectl get secret app-db -o jsonpath='{.data.ca\.pem}' | base64 -d | head -1
# -----BEGIN CERTIFICATE----- <- good: header on its own line
kubectl get secret app-db -o jsonpath='{.data.ca\.pem}' | base64 -d \
| openssl storeutl -noout -text -certs /dev/stdin | grep -E "Subject:|Not After"
# Lists every certificate in the bundle with its expiry
What the failures look like:
head -1prints the whole certificate on one line - newlines were stripped.- You see a literal
\nin the output - double-escaped. opensslcan’t load it at all - one of the two above, or a DER file that was never converted.
Finally, test the connection from inside a pod with verify-full. If the hostname in your connection string doesn’t match the server certificate, this is where you’ll find out - better now than during an incident.
Takeaways
-
requireencrypts,verify-fullverifies. Auditors care about the second one. Fall back toverify-caonly when private networking makes hostname verification impossible. -
Trust root CAs only. AWS and Azure both say so. Intermediates and server certificates rotate - Azure’s without notice.
-
Azure’s root CAs download as DER. Convert with
openssl x509 -inform DERbefore they go anywhere near a secret manager. -
Inside JSON, newlines must be
\n. Usejq -Rsor--rawfileinstead of escaping by hand. -
AWS Secrets Manager: certificates go in the Plaintext tab (or
--secret-string file://). The key/value editor doesn’t keep multi-line values intact. -
Azure Key Vault and GCP Secret Manager store one value per secret. Store the raw PEM on its own with
--file/--data-file, or as an escaped field in a JSON map. -
External Secrets Operator makes all three look the same.
propertyfor JSON maps, nothing for single values,decodingStrategy: Base64for base64. -
Mount the directory, not
subPath, so CA rotations reach running pods. -
Verify what’s in the cluster.
head -1andopenssltake ten seconds and catch every mistake above.