Written September 21, 2021
Zero Trust or BeyondProd approaches require authenticated and encrypted communications everywhere. TLS is the cryptographic protocol that powers encryption for all your technologies. For TLS, you need certificates. This practitioner's tutorial provides instructions for automating Kubernetes Ingress TLS certificate renewal and enabling server-side encryption.
Before you can configure Kubernetes Ingress TLS, you will need to be able to request a certificate issued by a trusted certificate authority (CA). If you already have a private CA and root certificate, you can skip to the automated renewal section below. If you need to create a CA, you can:
step-ca
Request a copy of your CA root certificate, which will be used to make sure each application can trust certificates presented by other applications.
step ca root ca.crt
Your certificate will be saved in ca.crt
.
Smallstep CAs use provisioners to authenticate certificate requests using passwords, one-time tokens, single sign-on, and a variety of other mechanisms.
step
CLI and do not require a local network agent. The instructions below focus on the JWK provisioner, but can be repurposed with small tweaks to operationalize all non-ACME provisioners.To learn more, see Configuring step-ca
Provisioners.
The right provisioner depends on your operational environment.
The JWK provisioner is the most general-purpose provisioner. It supports password and one-time token-based authentication. To add a JWK provisioner called kubernetes-ingress
to a hosted Certificate Manager authority (if you haven't already), run:
step ca provisioner add kubernetes-ingress --type JWK --create --x509-default-dur 720h
For instructions on adding provisioners to open source step-ca
, or to learn more about other provisioner types, see Configuring step-ca
Provisioners.
There are various approaches for configuring TLS certificate renewal in Kubernetes. For this tutorial, we will assume you are using Helm and will manage certificate renewal using the open source cert-manager project. You may repurpose various pieces for your own configuration by following the cert-manager or Kubernetes Ingress documentation.
First, install cert-manager in your cluster with Helm:
helm repo add jetstack https://charts.jetstack.io
helm repo update
helm install cert-manager jetstack/cert-manager --namespace cert-manager --create-namespace --set installCRDs=true
In cert-manager, a custom resource type Certificate
represents
each TLS certififcate and specifies how to store it.
Create a new Certificate
resource myserver-certificate.yaml
for your Kubernetes Ingress server. Make sure you specify the same DNS name configured above, and keep track of the Kubernetes secret name where cert-manager will store your certificate and private key.
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
name: myserver
spec:
secretName: myserver-tls
duration: 720h
dnsNames:
- myserver.example.net
issuerRef:
name: my-ca
kind: StepIssuer
group: certmanager.step.sm
Customize the Certificate
name, secretName
, and issuer name to suit your own naming style in your Kubernetes cluster, then apply it with kubectl
.
kubectl apply -f myserver-certificate.yaml
This configuration specifies that cert-manager should issue and renew a TLS certificate with the DNS name myserver.example.net
and store the certificate and private key in a Kubernetes secret named myserver-tls
. The certificate is valid for 720 hours, and cert-manager will automatically renew it before expiration and update the myserver-tls
secret.
You'll also notice the above Certificate
resource has an issuerRef
specifying a cert-manager StepIssuer
resource named my-ca
.
Before cert-manager knows how to use your CA
to issue and renew your certificate, we'll need to create
that StepIssuer
resource and configure it to point to your private CA.
First, let's grab the name
and key kid
from the JWK provisioner we created earlier.
step ca provisioner list
Next, get a base64-encoded version of your CA root certificate:
step ca root -f >(step base64) 2> /dev/null
Create my-ca-issuer.yaml
and fill in the CA URL and provisioner details
with your own. caBundle
refers to the base64-encoded version
of your root certificate.
apiVersion: certmanager.step.sm/v1beta1
kind: StepIssuer
metadata:
name: my-ca
namespace: default
spec:
url: https://my-ca-url.my-team.ca.smallstep.com
caBundle: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUJpekNDQVRHZ0F3SUJBZ0lRTytFQWg4eS8wVjlQMFhwSHJWajVOVEFLQmdncWhrak9QUVFEQWpBa01TSXcKSUFZRFZRUURFeGxUZEdWd0lFTmxjblJwWm1sallYUmxjeUJTYjI5MElFTkJNQjRYRFRFNU1EZ3hNekU1TVRVdwpNbG9YRFRJNU1EZ3hNREU1TVRVd01sb3dKREVpTUNBR0ExVUVBeE1aVTNSbGNDQkRaWEowYVdacFkyRjBaWE1nClVtOXZkQ0JEUVRCWk1CTUdCeXFHU000OUFnRUdDQ3FHU000OUF3RUhBMElBQkFNVkw3VzBQbTNvSlVmSTR3WGQKa2xERW5uNVhTbWo4NlgwYW1DQTBnY08xdElUUG1DVzNCcGU0cE9vV1V2WlZlUWRvU2NxN3pua1V0Mi9HMnQxTgo3MWlqUlRCRE1BNEdBMVVkRHdFQi93UUVBd0lCQmpBU0JnTlZIUk1CQWY4RUNEQUdBUUgvQWdFQk1CMEdBMVVkCkRnUVdCQlJ1Y1ByVm5QdlpOMHI0QVU5TGcyL2VCcng3a2pBS0JnZ3Foa2pPUFFRREFnTklBREJGQWlCUlJBdGsKNXpMY0doQ2FobVBuVzIwZExpdEMzRVdNaVE0bERwN2FFeitFUEFJaEFJOWZWczVxb0l0bVQ4anA2WktVNVEydQphRFBrOGsyQ25OMjdyRnNZV3VwTAotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCg==
provisioner:
name: kubernetes-ingress
kid: N6I99Yuk7iGDMk_eW3QaN2admCsrC9UuDN27dlFXUOs
passwordRef:
name: my-ca-provisioner-password
key: password
You'll notice the passwordRef
configuration above, which declares a secret
where step-issuer can expect to find your JWK provisioner password.
You can name this whatever you'd like.
Create the secret in your cluster, then apply your StepIssuer
resource.
kubectl create secret generic my-ca-provisioner-password --from-literal=password=Y4nys7f11
kubectl apply -f my-ca-issuer.yaml
If all went as expected now that both your Certificate
and your StepIssuer
are configured, cert-manager should have reached out to your Smallstep CA, issued (or begun issuing) the certificate, and created the myserver-tls
secret with your certificate and private key.
Telling your ingress to make use of the certificate and private key
in the myserver-tls
secret is as easy as adding a tls
block
to your exsting Ingress
resources.
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: myserver
labels: ...
annotations: ...
spec:
rules:
- host: myserver.example.net
http:
paths:
- backend:
serviceName: my-service
servicePort: 5000
path: /
tls:
- hosts:
- myserver.company.net
secretName: myserver-tls
Voila! Your ingress controller will pick up this tls
configuration
and automatically serve the renewed certificate each time
the myserver-tls
secret is updated.
Use curl
to check that your TLS configuraiton is working as expected.
You'll need to pass your CA root certificate as an argument
so curl
can verify the ingress certificate.
$ curl --cacert ca.crt https://myserver.example.net
HTTP/2 200
...
Once Kubernetes Ingress TLS is configured, you'll need to make sure that clients know to trust certificates signed by your CA. For certificates signed by a public CA (like Let's Encrypt), most clients already include the CA root certificate in their trust stores for certificate verification. But, for a private CA, you will need to explicitly add your CA's root certificate to your clients' trust stores.
The step
CLI includes a utility command for this purpose on many systems:
step certificate install ca.crt
Rather than manually running the above for each machine that needs to trust your CA, most teams will use some form of automation to distribute the root certificate. Depending on your needs and your IT or DevOps team's approach, this may be a configuration management tool (like Ansible or Puppet), a Mobile Device Management (MDM) solution, or something else. Some examples:
ca.crt
directly to the ca-ceritficates
bundle on linux VMs so running applications trust the API servers they callca.crt
directly into base Docker images for gRPC so gRPC clients can always reference the trusted CAca.crt
in a Kubernetes Secret
and inject it into an environment variable for access from application codeca.crt
in the trust stores of every employee Macbook so their web browsers trust internal websitesstep certificate install ca.crt
on target machines that want curl
to implicity trust the CAca.crt
in a Kubernetes ConfigMap
and mount it to pods for reference on the filesystemAlternatively, many clients support passing the CA root certificate as a flag or argument at runtime.
The Practical Zero Trust project is a collection of living documents detailing TLS configuration across a broad spread of technologies. We'd love to make this document better. Feel free to contribute any improvements directly on GitHub.
Unsubscribe anytime. See our privacy policy.
© 2023 Smallstep Labs, Inc. All rights reserved.