Skip to content

CAPIF Certificate Generation Architecture

Summary

This document describes the changes made to the certificate generation and management architecture in CAPIF, implemented in the OCF182-certs-generation branch. The main objective is to allow a single Vault server to serve multiple CAPIF instances efficiently and securely.

Main Changes

Previous Architecture

In the previous implementation, Vault was responsible for:

  • Generating the root CA and intermediate CA
  • Generating service certificates
  • Storing and distributing certificates

Problems:

  • Vault generated service certificates (less secure)
  • Difficult scalability for multiple CAPIF instances
  • Strong coupling between Vault and each CAPIF instance

New Architecture

In the new implementation:

┌──────────────────────────────────────────────────────────────┐
│                          VAULT                               │
│  ┌────────────────────────────────────────────────────────┐  │
│  │  1. Generates Root CA                                  │  │
│  │  2. Generates Intermediate CA                          │  │
│  │  3. Configures signing role                            │  │
│  │  4. Stores CA bundle in secret/ca                      │  │
│  └────────────────────────────────────────────────────────┘  │
│                                                              │
│  Functions:                                                  │
│  ✓ SIGNS certificates (receives CSR)                         │
│  ✓ STORES signed certificates                                │
└──────────────────────────────────────────────────────────────┘
                             ▲
                             │
        ┌────────────────────┼────────────────────┐
        │                    │                    │
        ▼                    ▼                    ▼
┌──────────────┐    ┌──────────────┐    ┌──────────────┐
│  CAPIF #1    │    │  CAPIF #2    │    │  CAPIF #N    │
│  CCF_ID: A   │    │  CCF_ID: B   │    │  CCF_ID: C   │
└──────────────┘    └──────────────┘    └──────────────┘

Benefits:

  • ✅ Enhanced security: each service generates its own private key
  • ✅ Scalability: one Vault serves multiple CAPIF instances
  • ✅ Isolation: certificates organized by CCF_ID
  • ✅ PKI best practices: separation between generation and signing

Modified Components

1. Vault (services/vault/vault_prepare_certs.sh)

Main changes:

  • No longer generates service certificates, only PKI infrastructure
  • Generates only root CA and intermediate CA
  • Configures PKI signing endpoint at /v1/pki_int/sign/my-ca
  • Stores CA bundle in secret/ca (KV v2)
  • Configures my-ca signing role with flexible policies that allow signing any CSR

2. NGINX (services/nginx/nginx_prepare.sh)

New primary certificate generation component:

Implemented flow:

  1. Fetches CA certificate from Vault (secret/ca)
  2. Generates its own private key (server.key) if it doesn't exist
  3. Creates a CSR (Certificate Signing Request) with CAPIF hostname
  4. Sends CSR to Vault for signing
  5. Receives signed certificate (server.crt)
  6. Extracts public key from certificate (server_pub.pem)
  7. Obtains unique CCF_ID from Helper
  8. Stores all certificates in Vault under secret/capif/${CCF_ID}/nginx

Important note: This script includes retries with timeout to handle ordered service startups.

3. Register (services/register/register_prepare.sh)

Similar pattern to NGINX:

  • Generates its own private key (register_key.key)
  • Creates CSR with complete organization information
  • Obtains CA from Vault
  • Requests certificate signing from Vault via PKI endpoint
  • Saves signed certificate locally

4. Helper Service (services/helper/helper_service/app.py)

Modifications at application startup:

  • Creates a dedicated certs/ directory with restrictive permissions
  • Generates key pair and CSR for superadmin certificate using pyOpenSSL
  • Saves private key locally with 600 permissions
  • Requests certificate signing from Vault
  • Downloads and stores signed superadmin certificate
  • Downloads and saves CA root from Vault

Security improvements: permission validation and robust error handling.

5. API Services (Invoker, Provider, Security)

Modified prepare_*.sh scripts:

New consumption pattern (no longer generate certificates):

  1. Query Helper to obtain CAPIF instance's CCF_ID
  2. Retrieve server's public key from Vault using path secret/capif/${CCF_ID}/nginx
  3. Save public key to local location
  4. Implement robust retries to wait for NGINX to generate and store certificates

Affected files:

  • prepare_invoker.sh
  • prepare_provider.sh
  • prepare_security.sh

Certificate Organization in Vault

KV (Key-Value v2) Structure

Root level:

  • secret/ca: Stores intermediate CA certificate (accessible by all CAPIF instances)

Per-CAPIF level:

  • secret/capif/<CCF_ID>/nginx/: Contains certificates for each CAPIF instance
  • server_crt: Server certificate
  • server_key: Server private key
  • server_pub: Extracted public key
  • ca: Copy of CA bundle

Note: In the future, sub-levels for register/ and helper/ will be added under each CCF_ID.

PKI Endpoints

Root PKI Engine (pki/):

  • root/generate/internal: Generates root CA
  • config/urls: Configures issuance and CRL URLs
  • root/sign-intermediate: Signs intermediate CA

Intermediate PKI Engine (pki_int/):

  • intermediate/generate/internal: Generates intermediate CA CSR
  • intermediate/set-signed: Installs signed intermediate certificate
  • roles/my-ca: Defines signing role with its policies
  • sign/my-ca: Endpoint used by services to sign their CSRs

Deployment Flow

1. Vault Initialization

Execute once per Vault cluster:

  • In Docker: run vault_prepare_certs.sh script
  • In Kubernetes: apply ConfigMap and Job from helm/vault-job/vault-job.yaml

Expected result:

  • ✅ Root CA generated
  • ✅ Intermediate CA generated and signed
  • ✅ my-ca role configured
  • ✅ CA bundle stored in secret/ca

2. CAPIF Instance Deployment

Each CAPIF instance (identified by unique CCF_ID):

  1. Helper starts → Generates unique CCF_ID

  2. NGINX starts:

  3. Generates server.key and server.csr

  4. Requests signing from Vault
  5. Receives server.crt
  6. Stores in secret/capif/${CCF_ID}/nginx

  7. Register starts:

  8. Generates registration certificate

  9. Requests signing from Vault

  10. API Services start:

  11. Retrieve certificates from Vault using CCF_ID

  12. Use public key for validation

Advantages of New Architecture

Security

  1. Private keys never transit: each service generates its private key locally
  2. Principle of least privilege: Vault only has CA role, doesn't manage service keys
  3. Isolation per CAPIF: certificates separated by CCF_ID

Scalability

  1. One Vault, multiple CAPIF: total decoupling
  2. Namespace per instance: secret/capif/${CCF_ID}/
  3. No collisions: unique CCF_ID guarantees separation

Operations

  1. Simplified rotation: each CAPIF can regenerate its certificates independently
  2. Improved auditing: can track which CAPIF requested which certificate
  3. Independent deployment: a CAPIF can restart without affecting others

Helm/Kubernetes Compatibility

The helm/vault-job/vault-job.yaml file has been significantly simplified:

Main changes:

  • ❌ No longer generates certificates for specific services
  • ✅ Only executes basic PKI setup (Root CA, Intermediate CA, signing role)
  • ✅ Lighter: 144 lines removed
  • ✅ ConfigMap contains a self-contained script that enables PKI engines, generates CAs and configures signing role
  • ✅ Job runs only once when deploying Vault namespace

Docker Compose Configuration

Changes in services/docker-compose-capif.yml:

NGINX:

  • No longer mounts external certificate volumes
  • Certificates are generated dynamically in /etc/nginx/certs at startup
  • New environment variables: VAULT_HOSTNAME, VAULT_PORT, VAULT_ACCESS_TOKEN, CAPIF_HOSTNAME

Helper:

  • Manages its own certificate directory internally
  • certs/ directory is automatically created by app.py at startup
  • No longer requires external volumes for certificates

Migration from Previous Architecture

Migration steps:

  1. Backup existing certificates: Export current certificates from Vault (optional, for history)
  2. Deploy new Vault: Execute updated vault_prepare_certs.sh script
  3. Clean old certificates: Remove certificate directories in NGINX and Register services
  4. Redeploy CAPIF services: Execute run.sh so services automatically generate their new certificates

Important considerations:

  • ⚠️ Downtime: There will be a service interruption during migration
  • ⚠️ New CCF_ID: Each instance will get a new unique CCF_ID
  • ⚠️ Regenerated certificates: All service certificates will be new
  • ✅ No API changes: CAPIF's public interface remains the same
  • ✅ Compatibility: Existing clients will continue working after obtaining new CA

Testing

Recommended verifications:

  1. Verify CA in Vault: Query secret/data/ca with curl to confirm CA bundle is available
  2. Start CAPIF: Execute startup script and observe logs from each service
  3. Verify NGINX certificate: Use openssl x509 inside container to inspect generated certificate
  4. Verify storage in Vault: Obtain CCF_ID from Helper and query path secret/capif/${CCF_ID}/nginx in Vault
  5. Run test suite: Run run_capif_tests.sh to validate complete functionality

Troubleshooting

Error: "Unable to retrieve CA certificate from Vault"

Cause: Vault has not been properly initialized with PKI infrastructure

Solution: Execute Vault preparation script (vault_prepare_certs.sh) inside container

Error: "Failed to sign server certificate"

Cause: my-ca signing role is not configured or Vault token doesn't have sufficient permissions

Solution: Verify role exists and token has write permissions on pki_int/sign/my-ca

Error: "Unable to retrieve CCF_ID from Helper"

Cause: Helper service hasn't completed initialization yet

Solution: Wait 30-60 seconds. Service scripts implement automatic retries with incremental delays

References

Modified Files (Branch OCF182-certs-generation)

Document Version: 1.0
Date: February 2026
Branch: OCF182-certs-generation
Author: CAPIF Team