Redis TLS — Practical Zero Trust

How to get and renew Redis TLS certificates

Written September 20, 2021

Zero Trust or BeyondProd approaches require authenticated and encrypted communications everywhere. TLS is the cryptographic protocol that powers encryption for all your technologies. For TLS, you need certificates. This practitioner's tutorial provides instructions for automating Redis TLS certificate renewal and enabling server-side encryption.

Try it

Create a private key and request a certificate

Before you can configure Redis TLS, you will need a certificate issued by a trusted certificate authority (CA). If you already have a certificate, private key, and CA root certificate from your organization's existing CA, you can skip to the Redis TLS configuration section below. If you need to generate a certificate, you can:

Use a free hosted Smallstep Certificate Manager authority

Roll your own with open source step-ca

This tutorial is designed for private networks, not publicly trusted websites or endpoints. If you are looking to automate certificates for public websites, servers, or endpoints, we recommend using Let's Encrypt and the ACME instructions below.

To request a certificate from your CA using the step CLI, bootstrap your CA with step ca bootstrap and run the following command (sub the server name for the actual name / DNS name of your Redis server).

step ca certificate "redis.example.net" server.crt server.key

Your certificate and private key will be saved in server.crt and server.key respectively.

Request a copy of your CA root certificate, which will be used to make sure each application can trust certificates presented by other applications.

step ca root ca.crt

Your certificate will be saved in ca.crt.

Configure Redis to use the certificate

You can test your certificate by starting up Redis with TLS enabled.

Native "SSL Support" (TLS) was added to Redis 6.0.0, which was released GA on April 30, 2020. TLS in Redis is an optional feature and not all Redis server binaries will have it enabled. The official Docker image, and many OS distribution images, have TLS enabled.

redis-server \
    --tls-port 6379 --port 0 \
    --tls-cert-file /path/to/server.crt \
    --tls-key-file /path/to/server.key \
    --tls-ca-cert-file /path/to/ca.crt \
    --tls-auth-clients no

6379 is the default port that redis-cli (and probably SDKs) connect to.

Setting --port 0 disables the non-TLS TCP socket.

--tls-auth-client no disables client authentication (which is on by default).

The server's private key can be password-protected using PEM encryption. Use step crypto change-pass to encrypt the private key. Then, the server will prompt for the password at startup, or you can provide it using --tls-key-file-pass <password>.

Test Redis TLS configuration

Send a PING command to Redis to test your TLS configuration:

$ redis-cli --tls --cacert /path/to/ca.crt ping
PONG

Operationalize It

Select a provisioner

Smallstep CAs use provisioners to authenticate certificate requests using passwords, one-time tokens, single sign-on, and a variety of other mechanisms.

ACME (RFC8555) is an open standard, used by Let's Encrypt, for authenticating certificate requests. To use ACME on a private network you need to run an ACME server. ACME is harder to setup, but has a large client ecosystem (some software even has built-in support).

Other provisioners use the open source step CLI and do not require a local network agent. The instructions below focus on the JWK provisioner, but can be repurposed with small tweaks to operationalize all non-ACME provisioners.

To learn more, see Configuring step-ca Provisioners.

Show me instructions for...

ACME

Other provisioners

The right provisioner depends on your operational environment.

The JWK provisioner is the most general-purpose provisioner. It supports password and one-time token-based authentication. To add a JWK provisioner called redis to a hosted Certificate Manager authority (if you haven't already), run:

step ca provisioner add redis --type JWK --create --x509-default-dur 720h

For instructions on adding provisioners to open source step-ca, or to learn more about other provisioner types, see Configuring step-ca Provisioners.

Configure Redis TLS Certificate Automation

We've created a systemd-based certificate renewal timer that works with step. Check out our documentation on Renewal using systemd timers for background on how these timers work.

To install the certificate renewal unit files, run:

cd /etc/systemd/system
sudo curl -sL https://files.smallstep.com/cert-renewer@.service \
     -o cert-renewer@.service
sudo curl -sL https://files.smallstep.com/cert-renewer@.timer \
     -o cert-renewer@.timer

The renewal timer will check your certificate files every five minutes and renew them after two-thirds of their lifetime has elapsed.

To renew and hot-reload the Redis server certificate, we will need a Redis-specific systemd override file that can tell Redis to refresh its certificates. To install the override, run:

sudo mkdir /etc/systemd/system/cert-renewer@redis-server.service.d
cat <<EOF | sudo tee /etc/systemd/system/cert-renewer@redis-server.service.d/override.conf
[Service]
; "Environment=" overrides are applied per environment variable. This line does not
; affect any other variables set in the service template.
Environment=CERT_LOCATION=/var/lib/redis/redis.crt \\
            KEY_LOCATION=/var/lib/redis/redis.key \\
            REDIS_USERNAME=default \\
            REDISCLI_AUTH=""
; Empty ExecStartPost (Don't attempt to restart redis-server.service)
ExecStartPost=
ExecStartPost=redis-cli --user "\$REDIS_USERNAME" --tls --cacert /var/lib/redis/ca.crt config set tls-cert-file \$CERT_LOCATION
EOF

If you use password authentication in Redis, you'll need to update the REDIS_USERNAME and/or REDISCLI_AUTH. Otherwise, leave them alone.

To start the renewal timer, run:

sudo systemctl daemon-reload
sudo systemctl enable --now cert-renewer@redis-server.timer

You'll see that the timer is active, by checking the output of systemctl list-timers.

Distribute your root certificate to end users and systems

Once Redis TLS is configured, you'll need to make sure that clients know to trust certificates signed by your CA. For certificates signed by a public CA (like Let's Encrypt), most clients already include the CA root certificate in their trust stores for certificate verification. But, for a private CA, you will need to explicitly add your CA's root certificate to your clients' trust stores.

The step CLI includes a utility command for this purpose on many systems:

step certificate install ca.crt

Rather than manually running the above for each machine that needs to trust your CA, most teams will use some form of automation to distribute the root certificate. Depending on your needs and your IT or DevOps team's approach, this may be a configuration management tool (like Ansible or Puppet), a Mobile Device Management (MDM) solution, or something else. Some examples:

Use Ansible to add ca.crt directly to the ca-ceritficates bundle on linux VMs so running applications trust the API servers they call

Bake ca.crt directly into base Docker images for gRPC so gRPC clients can always reference the trusted CA

Store ca.crt in a Kubernetes Secret and inject it into an environment variable for access from application code

Use Jamf to install ca.crt in the trust stores of every employee Macbook so their web browsers trust internal websites

Use Puppet to run step certificate install ca.crt on target machines that want curl to implicity trust the CA

Store ca.crt in a Kubernetes ConfigMap and mount it to pods for reference on the filesystem

Alternatively, many clients support passing the CA root certificate as a flag or argument at runtime.

Research notes

In researching Redis TLS, we did some thorough investigation. Here are our rough notes if you are interested in diving deeper.

Full research notes

General notes

Redis client & server (redis-cli and redis-server) do not use your system's trust store. They require explicitly configuration of trusted CA root(s) (i.e., trust store).

I tested two popular SDKs (node-redis & go-redis/redis) and they both used the language-default behavior.

golang uses the system truststore

node ships their own trust store, but you can use the EXTRA_CA_CERTS env var to add more

redis-cli doesn't do name validation on the server certificate. The server's hostname does not need to match the certificate subject (CN/SANs).

Anyone who is able to get a certificate from your trusted CA could MITM connections from the redis-cli client.

I tested two popular SDKs (node-redis & go-redis/redis) and they both implement proper certificate path validation and verify the server name.

I don't want to endorse what redis-cli is doing here, but I suppose it isn't too terrible if you trust everything that can get a certificate from your CA. However, I'd check client libraries carefully to make sure there aren't any that 1) trust the system trust store, and 2) disable server name verification. In that scenario, anyone with a Web PKI certificate would be able to MITM your Redis server.

User authentication is "partial" because, while the Redis server supports optional or mandatory client certificates (mTLS), it does not do any authorization or mapping between certificate subject and Redis user.

Functionally this establishes a sort of "cryptographic boundary", where any client can connect to Redis as long as it's able to get a certificate from the trusted CA.

The expectation appears to be that the standard username/password authentication would occur over the [m]TLS-encrypted channel.

Client authentication can also be enabled for clustered / replicated deployments. Client certificates are then required for establishing cluster bus and replication connections.

You can provide a different certificate for Redis servers to use as client certs for these connections (e.g., with a different key use). However, you can't configure a different root CA (i.e., the same root CA is used to verify client certs for normal clients, replicas, and cluster nodes).

For replication, you can use the masterauth directive to specify a password / shared secret to authenticate replicas. This would allow you to authenticate replicas and would prevent normal clients from connecting to the replication API.

The cluster bus does not appear to have any additional authentication mechanism. Presumably, that means normal clients could use client certs to connect to authenticate to the cluster bus API as cluster nodes. I haven't tested this though.

Refreshing the Certificate

CERT_FILE=$(redis-cli --tls --cacert .step/certs/root_ca.crt --raw config get tls-cert-file | tail -n1)
REDIS_USERNAME=default
REDISCLI_AUTH=""
redis-cli --user "$REDIS_USERNAME" --tls --cacert /path/to/root_ca.crt config set tls-cert-file ${CERT_FILE}

This is a little quirky. Redis doesn't reload its config on a SIGHUP. But it does support dynamic configuration, and if you set the tls-cert-file directive it will reload the cert. This is the recommended approach to force a certificate reload. Note that the value here is the filesystem path to the server's certificate. We're not actually changing it, we're literally reading the existing value and then writing that value again.

Note also this can be a relative path, which is super weird since you're setting it via an API and don't really know the server's working directory. Still, we're just setting it to the same value it was at startup.

Another quirk here is that, if the server's certificate expires, the client won't connect and we can't get the server to reload its cert without killing it and restarting it. An --insecure option was added to redis-cli in version 6.2 which fixes this.

Root Distribution

"Adding SAN validation also makes sense (again, optionally) but will probably be more difficult to do, as Redis tends to use IPs rather than hostnames in all sorts of places. https://github.com/redis/redis/issues/7946#issuecomment-714660881

"The best option is to disable matching of hostname to certificate name. As long as your CA is private the security implications should be minor" https://github.com/redis/redis/issues/7928#issuecomment-714462007

The tls-ca-cert-dir directive is complicated: https://github.com/redis/redis/issues/7392

Redis TLS — Practical Zero Trust

How to get and renew Redis TLS certificates

Try it

Create a private key and request a certificate

Configure Redis to use the certificate

Test Redis TLS configuration

Operationalize It

Select a provisioner

Show me instructions for...

Configure Redis TLS Certificate Automation

Distribute your root certificate to end users and systems

Research notes

Full research notes

General notes

Refreshing the Certificate

Root Distribution

Further reading

Contribute to this document

Subscribe to updates

Learn

Products

Pricing

Documentation

Open Source

Company