Understanding the Role of Cert-manager in Cluster API
The Cluster API team has started cutting release candidates of the latest version of the project: Cluster API v0.3. I have been looking forward to this new version, as it adds a set of powerful capabilities, such as declarative control plane management and cluster upgrades. I plan to explore those features in future posts.
As I started testing the latest release candidate, I noticed that I had to install cert-manager into my management cluster. I wasn’t quite sure why this was the case, so I decided to learn more about how Cluster API leverages cert-manager.
What is cert-manager?
Cert-manager is a controller and set of Custom Resource Definitions (CRDs) that enable automated certificate management on Kubernetes. Some of the CRDs that cert-manager installs are
Certificates. Using these CRDs, you can request a certificate via the Kubernetes API. Once you create the request, the cert-manager controller creates and manages the certificate for you.
The most common scenario I have run into with cert-manager is issuing certificates for Ingress resources. Cluster API, however, does not use cert-manager for this.
Cert-manager and Cluster API
After installing the latest release candidate of cluster API in a fresh management cluster, I looked for certificate requests:
$ kubectl get certificaterequests --all-namespaces
NAMESPACE NAME READY AGE
capi-webhook-system capi-kubeadm-bootstrap-serving-cert-1303918272 True 109m
capi-webhook-system capi-kubeadm-control-plane-serving-cert-1399157480 True 109m
capi-webhook-system capi-serving-cert-3321106290 True 109m
I noticed that all certificate requests live in the
capi-webhook-system namespace. The next thing I wanted to know was the issuer used to create these certificates:
$ kubectl get issuer -n capi-webhook-system
There are three issuers, one for each of the certificates that Cluster API requests during installation. Judging by the name, the issuers create self-signed certificates.
Finally, I listed the Certificates in the
capi-webhook-system to verify that cert-manager had fulfilled the Certificate Requests:
$ kubectl get certificates -n capi-webhook-system
NAME READY SECRET AGE
capi-kubeadm-bootstrap-serving-cert True capi-kubeadm-bootstrap-webhook-service-cert 108m
capi-kubeadm-control-plane-serving-cert True capi-kubeadm-control-plane-webhook-service-cert 108m
capi-serving-cert True capi-webhook-service-cert 108m
After exploring the
certificates created during the Cluster API installation, it is evident that Cluster API leverages cert-manager to create self-signed certificates. But how are these certificates used?
Cluster API Webhooks
Starting with version 0.3, Cluster API configures validating and mutating webhooks in the management cluster. It also uses webhooks for converting custom resources between API versions. These webhooks are admission controllers or plugins that the API server calls when Cluster API resources are created or modified.
The validating webhooks validate
v1alpha3 Cluster API resources. For example, if I try to create a
v1alpha3 Cluster that has an
InfrastructureRef in another namespace, I get a validation error. This validation is implemented in the validating webhook.
$ kubectl apply -f invalid-cluster.yaml
Error from server (Cluster.cluster.x-k8s.io "capi-quickstart" is invalid:
spec.infrastructureRef.namespace: Invalid value: "foo": must match metadata.namespace):
error when creating "sample.yaml": admission webhook "validation.cluster.cluster.x-k8s.io" denied the request:
Cluster.cluster.x-k8s.io "capi-quickstart" is invalid: spec.infrastructureRef.namespace:
Invalid value: "foo": must match metadata.namespace
Cluster API uses the mutating webhooks to set default values in
v1alpha3 custom resources. For example, the
Cluster resource defaults the namespace of the
ControlPlaneRef to the same namespace as the
Cluster. The defaulting of these values improve the UX of these custom resources.
Finally, Cluster API specifies conversion webhooks in the CRD specifications. The conversion webhooks implement the conversion of custom resources from
v1alpha3 and vice-versa. The conversion allows Cluster API users to upgrade their existing management clusters from Cluster API 0.2 to 0.3.
This is great, but what does it have to do with certificates?
When configuring admission webhooks, Kubernetes requires the webhooks to expose an HTTPS service. And this is precisely where the certificates come into play—three certificates for three webhooks. The webhook services use the certificates to expose the webhooks over HTTPS.
# Three certificates
$ kubectl get certificate
NAME READY SECRET AGE
capi-kubeadm-bootstrap-serving-cert True capi-kubeadm-bootstrap-webhook-service-cert 47h
capi-kubeadm-control-plane-serving-cert True capi-kubeadm-control-plane-webhook-service-cert 47h
capi-serving-cert True capi-webhook-service-cert 47h
# Three webhook services
$ kubectl get service
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
capi-kubeadm-bootstrap-webhook-service ClusterIP 10.96.81.253 <none> 443/TCP 47h
capi-kubeadm-control-plane-webhook-service ClusterIP 10.96.18.220 <none> 443/TCP 47h
capi-webhook-service ClusterIP 10.96.11.28 <none> 443/TCP 47h
I hope this post sheds light on why Cluster API v0.3 needs cert-manager in the management cluster.
In brief, Cluster API v0.3 installs admission webhooks in the management cluster to validate, mutate, and convert Cluster API resources. Kubernetes requires these webhooks to be HTTPS services, thus requiring serving certificates. Instead of creating the certificates in a one-shot manner, Cluster API leverages the cert-manager project to create and manage the webhook certificates throughout the lifetime of the management cluster.
If you are looking to test the latest Cluster API release candidates, check out the master version of the Cluster API Quick Start (docs are still under development, so you might run into sharp edges).
Did you find this post useful? Did I get something wrong? I would love to hear from you! Please reach out via @alexbrand.