Kubernetes Baseline

• NGINX Ingress Controller
• cert-manager
• metrics-server
• prometheus and grafana
• keel

Notes

ingress-nginx servicemonitor

For unknown reasons the servicemonitor is not create on helm install and therefore no metrics will be scraped by prometheus. Use helm upgrade -n ingress-nginx ingress-nginx ingress-nginx/ingress-nginx to force an update of t the release.

Linode

PROXY protocol needs to be enabled for ingress-nginx to see the clients IP in ingress log.

Add the PROXY protocol annotation to the ingress-nginx service:

annotations:
    service.beta.kubernetes.io/linode-loadbalancer-proxy-protocol: v2

Update the ingress-nginx ConfigMap to make nginx expect PROXY protocol data:

data:
  use-proxy-protocol: "true"

cert-manager

However, when you have the PROXY protocol enabled, the external load balancer does modify the traffic, prepending the PROXY line before each TCP connection. If you connect directly to the web server internally, bypassing the external load balancer, then it will receive traffic without the PROXY line.

This is particularly a problem when using cert-manager for provisioning SSL certificates.

After enabling the PROXY protocol cert-manager is unable to perform a self check ("propagation check failed", "failed to perform self check GET request").

hairpin-proxy adds PROXY protocol support for internal-to-LoadBalancer traffic for Kubernetes Ingress users, specifically for cert-manager self-checks (no further configuration needed).