Open Source

Integrate Certificate Manager, step-issuer, and cert-manager for Kubernetes TLS

When connecting things outside Kubernetes to Kubernetes services, we recommend using Smallstep Certificate Manager in combination with step-issuer and Kubernetes’ cert-manager utility.

Before you begin

You will need:

An account on the Smallstep platform. Need one? Register here.

An Authority in Certificate Manager that will act as your upstream CA.

Additionally, we assume that you have a functioning Kubernetes Cluster already running and accessible.

kubernetes diagram

Dependencies

step

kubectl

helm

Add a Certificate Manager Provisioner

First, we'll add a Certificate Manager provisioner for step-issuer to use. In this case, we will create a JWK provisioner. Running the following command will start an authentication flow to Certificate Manager. Upon success, it will request a password for the new provisioner. Make sure that this password is saved in your password manager. You'll need it for later steps.

$ step ca provisioner add "step-issuer" --type JWK --create

Upon success, you will see the provisioner’s configuration returned as a successful response.

Run the following command, and make a note of the new provisioner’s “kid” field (the 5th line listed; it will be required in later steps):

$ step ca provisioner list | grep step-issuer -A 9
"name": "step-issuer",
      "key": {
         "use": "sig",
         "kty": "EC",
         "kid": "c39XHcunqE...BbR0xhl7I",
         "crv": "P-256",
         "alg": "ES256",
         "x": "abQRrRWF6cMlhRvpQlAZNLWUmwYjWi0MJvspgw",
         "y": "jeIdFtUu5lZwZacDeX8nElNtZPpQrW70WyUKOo"
},

Install and Configure `cert-manager`

cert-manager is a Kubernetes certificate management controller. This important add-on helps issue and renew certificates. Switch to the correct kubectl context for your Kubernetes Cluster, and run the following commands to install cert-manager to the cluster:

$ helm repo add jetstack https://charts.jetstack.io
"jetstack" has been added to your repositories

$ helm repo update
Hang tight while we grab the latest from your chart repositories...
...Successfully got an update from the "jetstack" chart repository
Update Complete. ⎈Happy Helming!⎈

Next, run:

$ helm install \
     cert-manager jetstack/cert-manager \
     --namespace cert-manager \
     --create-namespace \
     --version v1.7.1 \
     --set installCRDs=true
NAME: cert-manager
LAST DEPLOYED: Thu Mar 17 14:19:59 2022
NAMESPACE: cert-manager
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
cert-manager v1.7.1 has been deployed successfully!
  
In order to begin issuing certificates, you will need to set up a ClusterIssuer
or Issuer resource (for example, by creating a 'letsencrypt-staging' issuer).
  
More information on the different types of issuers and how to configure them
can be found in our documentation:
  
https://cert-manager.io/docs/configuration/
  
For information on how to configure cert-manager to automatically provision
Certificates for Ingress resources, take a look at the 'ingress-shim'
documentation:
  
https://cert-manager.io/docs/usage/ingress/

Now that cert-manager is installed, confirm your pods are running:

$ kubectl get pods -n cert-manager
NAME                                       READY   STATUS    RESTARTS   AGE
cert-manager-6d6bb4f487-gdtrj              1/1     Running   0          3m17s
cert-manager-cainjector-7d55bf8f78-hqhnm   1/1     Running   0          3m17s
cert-manager-webhook-577f77586f-n5q9h      1/1     Running   0          3m17s

Install & Connect `step-issuer`

step-issuer is a cert-manager CertificateRequest controller that signs the certificate requests from the cluster. We’ll need to connect it to your Certificate Manager authority before it can start signing certificates.

To install step-issuer into your cluster, first add the Smallstep repository:

$ helm repo add smallstep  https://smallstep.github.io/helm-charts
"smallstep" has been added to your repositories

Next, run:

$ helm repo update
Hang tight while we grab the latest from your chart repositories...
...Successfully got an update from the "jetstack" chart repository
...Successfully got an update from the "smallstep" chart repository
Update Complete. ⎈Happy Helming!⎈

Next, run:

$ helm install \
     step-issuer smallstep/step-issuer \
     --namespace step-issuer \
     --create-namespace
NAME: step-issuer
LAST DEPLOYED: Thu Mar 17 15:39:29 2022
NAMESPACE: default
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
⚙️  To get step-issuer up and running follow the next steps:
....

Now that step-issuer is installed, let's confirm its pods are running:

$ kubectl get pods -n step-issuer
NAME                           READY   STATUS    RESTARTS   AGE
step-issuer-56cc6c4498-g4pl2   2/2     Running   0          63s

We start initializing step-issuer using the base64 representation of the Certificate Manager authority’s Root Certificate. Do so by running the following command:

$ step ca root | step base64
LS0tLS1CRUdJTWhrak9....pzdnZ3bHpHcFg4eFJFRVJUSUZJQ0FUtCg==

In addition to this value, also gather your authority URL, provisioner password, and provisioner kid. Use these values to populate issuer-config.yaml below:

apiVersion: certmanager.step.sm/v1beta1
kind: StepIssuer
metadata:
 name: step-issuer
 namespace: step-issuer
spec:
 # The CA URL.
 url: https://${authority name}.${team name}.ca.smallstep.com
 # The base64 encoded version of the CA root certificate in PEM format.
 caBundle: ${base64 Value Above}
 # The provisioner name, kid, and a reference to the provisioner password secret.
 provisioner:
   name: step-issuer
   kid: ${Provisioner `kid`}
   passwordRef:
     name: step-issuer-provisioner-password
     key: password

Create a Kubernetes Secret holding the provisioner password for the step-issuer provisioner you’ve set up in Certificate Manager:

$ kubectl create secret \
		-n step-issuer generic step-issuer-provisioner-password \
		--from-literal=password=<your provisioner password>
secret/step-issuer-provisioner-password created

Apply issuer-config.yaml to the cluster:

$ kubectl apply -f issuer-config.yaml
stepissuer.certmanager.step.sm/step-issuer created

Moments later, you should be able to see the status property in the resource:

$ kubectl get stepissuers.certmanager.step.sm step-issuer \
    -n step-issuer \
    -o yaml
apiVersion: certmanager.step.sm/v1beta1
kind: StepIssuer
...
status:
  conditions:
  - lastTransitionTime: "2022-03-17T16:09:51Z"
    message: StepIssuer verified and ready to sign certificates
    reason: Verified
    status: "True"
    type: Ready

At this point, step-issuer is ready to begin signing certificates.

step-issuer has a controller watching for CertificateRequest resources. To create this request we first need to create a CSR with step. This command will ask for a password to encrypt the private key; generate a password in your password manager for this step:

$ step certificate create --csr \
    internal.smallstep.com internal.csr internal.key
Your certificate signing request has been saved in internal.csr.
Your private key has been saved in internal.key.

Encode the new CSR using base64:

$ cat internal.csr | step base64
LS0tLxRWFJIdH....LS0tLS0=

Now, create certificate-request.yaml and replace the request value with the base64 value of your new CSR:

apiVersion: cert-manager.io/v1
kind: CertificateRequest
metadata:
 name: internal-smallstep-com
 namespace: step-issuer
spec:
 # The base64 encoded version of the certificate request in PEM format.
 request: ${base64 Value from Above}
 # The duration of the certificate
 duration: 24h
 # If the certificate will be a CA or not.
 isCA: false
 # A reference to the issuer in charge of signing the CSR.
 issuerRef:
   group: certmanager.step.sm
   kind: StepIssuer
   name: step-issuer

And now apply certificate-request.yaml to the cluster:

$ kubectl apply -f certificate-request.yaml
certificaterequest.cert-manager.io/internal-smallstep-com created

And moments later the bundled signed certificate with the intermediate as well as the root certificate will be available in the resource:

$ kubectl get certificaterequests.cert-manager.io internal-smallstep-com     -n step-issuer     -o yaml
apiVersion: cert-manager.io/v1
kind: CertificateRequest
metadata:
....
conditions:
  - lastTransitionTime: "2022-03-18T09:59:26Z"
    message: Certificate request has been approved by cert-manager.io
    reason: cert-manager.io
    status: "True"
    type: Approved
  - lastTransitionTime: "2022-03-18T09:59:27Z"
    message: Certificate issued
    reason: Issued
    status: "True"
    type: Ready

Now, you are ready to use the TLS certificate in your app.

Using the Certificate Resource

Before supporting CertificateRequest, cert-manager supported the resource Certificate; this allows you to create TLS certificates providing only X.509 properties like the common name, DNS, or IP address SANs. cert-manager now provides a method to support Certificate resources using CertificateRequest controllers like step-issuer.

The YAML for a Certificate resource looks like example certificate.yaml below:

apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: backend-smallstep-com
  namespace: step-issuer
spec:
  # The secret name to store the signed certificate
  secretName: backend-smallstep-com-tls
  # Common Name
  commonName: backend.smallstep.com
  # DNS SAN
  dnsNames:
    - localhost
    - backend.smallstep.com
  # IP Address SAN
  ipAddresses:
    - "127.0.0.1"
  # Duration of the certificate
  duration: 24h
  # Renew 8 hours before the certificate expiration
  renewBefore: 8h
  # The reference to the step issuer
  issuerRef:
    group: certmanager.step.sm
    kind: StepIssuer
    name: step-issuer

To apply the certificate resource, run:

$ kubectl apply -f certificate.yaml
certificate.cert-manager.io/backend-smallstep-com created

Moments later, a CertificateRequest will be automatically created by cert-manager:

$ kubectl get certificates.cert-manager.io \
        -n step-issuer
NAME                    READY   SECRET                      AGE
backend-smallstep-com   True    backend-smallstep-com-tls   18s

step-issuer gets this CertificateRequest and sends the signing request to Certificate Manager and stores the signed certificate in the same resource. cert-manager retrieves the signed certificate and stores the signed certificate/key pair in the Secret denoted in the YAML file property secretName.

$ kubectl get secrets backend-smallstep-com-tls \
       -n step-issuer \
	   -o yaml
apiVersion: v1
data:
  ca.crt: LS0tLS1CRtLS....0FURS0tLS0tCg==
  tls.crt: LS0tKVjJXTj....dNSd3Qc3a0tLQo=
  tls.key: LS0tLS1CRUd....JTiBSU0LS0tLQo=
kind: Secret
metadata:
  annotations:
    cert-manager.io/alt-names: localhost,backend.smallstep.com
    cert-manager.io/certificate-name: backend-smallstep-com
    cert-manager.io/common-name: backend.smallstep.com
    cert-manager.io/ip-sans: 127.0.0.1
    cert-manager.io/issuer-group: certmanager.step.sm
    cert-manager.io/issuer-kind: StepIssuer
    cert-manager.io/issuer-name: step-issuer
    cert-manager.io/uri-sans: ""
  creationTimestamp: "2022-03-22T12:37:57Z"
  name: backend-smallstep-com-tls
  namespace: step-issuer
  resourceVersion: "305291"
  uid: fc54d1e7-4501-409b-b71b-e2c560504709
type: kubernetes.io/tls

That's it. You can now reference this secret from your app to access the certificate. No headaches involved, no custom tools required, and one less problem to think about. Zero Trust doesn’t have to be hard to implement: With the right tools, it’s simple to “set it and forget it” the first time around and automatically have mutual TLS at hand.