In modern cloud-native applications, secrets like database passwords, API keys, and TLS certificates are everywhere. Yet, manually rotating these credentials is error-prone, time-consuming, and risky. A single leaked credential can lead to catastrophic security breaches.
Example Incident:
In 2021, a major cloud provider suffered a breach due to an exposed API key that hadn’t been rotated for months. Attackers used it to access sensitive customer data.
To prevent such scenarios, automated secret rotation is essential. This article explores how to implement secure, zero-downtime secret rotation in Spring Boot microservices using HashiCorp Vault and Kubernetes.
Why HashiCorp Vault + Kubernetes?
1. The Problem with Static Secrets
- Hardcoded credentials in config files or environment variables.
- Manual rotation leads to human errors and service disruptions.
- Long-lived secrets increase breach risks.
2. How Vault Solves This
- Dynamic Secrets: Generate short-lived credentials on-demand.
- Automatic Rotation: Revoke and reissue secrets without downtime.
- Lease Management: Secrets expire automatically.
3. Kubernetes Integration
- Inject secrets directly into pods via Vault Agent Sidecar.
- CSI (Container Storage Interface) Provider mounts secrets as volumes.
- Service Accounts for secure authentication.
Implementation: Step-by-Step Guide
Step 1: Deploy Vault in Kubernetes
helm install vault hashicorp/vault --set server.dev.enabled=true
(For production, use HA mode with Consul storage backend.)
Step 2: Configure a Secrets Engine
Enable the Database Secrets Engine for dynamic MySQL credentials:
vault secrets enable database vault write database/config/mysql \ plugin_name=mysql-database-plugin \ connection_url="root:password@tcp(mysql:3306)/" \ allowed_roles="app-role"
Step 3: Spring Boot Integration
Add the Spring Cloud Vault dependency:
<dependency> <groupId>org.springframework.cloud</groupId> <artifactId>spring-cloud-starter-vault-config</artifactId> </dependency>
Configure application.yml:
spring:
cloud:
vault:
uri: http://vault:8200
authentication: KUBERNETES
kubernetes:
role: app-role
service-account-token-file: /var/run/secrets/kubernetes.io/serviceaccount/token
database:
enabled: true
role: app-role
backend: database
Step 4: Automate Rotation with Kubernetes CronJob
Create a CronJob to rotate secrets daily:
apiVersion: batch/v1
kind: CronJob
metadata:
name: vault-secret-rotator
spec:
schedule: "0 0 * * *"
jobTemplate:
spec:
template:
spec:
containers:
- name: vault-cli
image: vault:latest
command: ["vault", "write", "database/rotate-root/mysql"]
restartPolicy: OnFailure
Best Practices for Zero Downtime Rotation
✅ Use Vault Agent for Sidecar Injection
- Avoids app restarts by refreshing secrets in-memory.
✅ Implement Retry Logic in Apps
- Handle temporary credential failures gracefully.
✅ Monitor Lease Expiry
- Alert if secrets near expiration without renewal.
✅ Audit Secret Access
- Track who accessed what and when.
Conclusion
Manual secret rotation is no longer viable in cloud-native environments. By combining HashiCorp Vault’s dynamic secrets with Kubernetes-native integration, Java microservices can achieve:
🔒 Enhanced security with short-lived credentials.
⚡ Zero-downtime rotation for high availability.
🤖 Full automation eliminating human errors.
