k3s Kubernetes + ArgoCD + Baseline

• k3s
• ArgoCD
  • NGINX Ingress Controller
  • cert-manager
    • selfsigned issuer
    • LetsEncrypt issuers (Prod and Staging)
  • zabbix-proxy
  • keel
  • reloader

Run (Deploy k3s + ArgoCD + Baseline)

ansible-playbook k3s_boostrap.yml -i <host|ip>,

Get kubeconfig

cat /etc/rancher/k3s/k3s.yml

Add Agents

Get Agent Token

The secure token format (occasionally referred to as a "full" token) contains the following parts:

<prefix><cluster CA hash>::<credentials>

Get existing server token: cat /var/lib/docker/volumes/baseline_k3s-data/_data/server/token

Create new token: docker compose exec -it k3s k3s token create

Register Agent/Worker

export K3S_URL=https://<cpn.fqdn>:6443
export K3S_NODE_NAME=<node.fqdn>
export K3S_TOKEN=<full-token>
curl -sfL https://get.k3s.io | sh -s -

Notes

ArgoCD

To retrieve the initial admin password use kubectl -n argocd get secret argocd-initial-admin-secret -o jsonpath="{.data.password}" | base64 -d

To change the password follow Argocd account update password.

Sync Applications with Kubectl

Add to application:

operation:
  sync:
    syncStrategy:
      hook: {}

Zabbix Monitoring

See: infrastructure/zabbix-config - Zabbix Kubernetes Monitoring

Cloud Setups

Linode

PROXY protocol needs to be enabled for ingress-nginx to see the clients IP in ingress log.

Add the PROXY protocol annotation to the ingress-nginx service:

annotations:
    service.beta.kubernetes.io/linode-loadbalancer-proxy-protocol: v2

Update the ingress-nginx ConfigMap to make nginx expect PROXY protocol data:

data:
  use-proxy-protocol: "true"

cert-manager

However, when you have the PROXY protocol enabled, the external load balancer does modify the traffic, prepending the PROXY line before each TCP connection. If you connect directly to the web server internally, bypassing the external load balancer, then it will receive traffic without the PROXY line.

This is particularly a problem when using cert-manager for provisioning SSL certificates.

After enabling the PROXY protocol cert-manager is unable to perform a self check ("propagation check failed", "failed to perform self check GET request").

hairpin-proxy adds PROXY protocol support for internal-to-LoadBalancer traffic for Kubernetes Ingress users, specifically for cert-manager self-checks (no further configuration needed).