Kubernetes HA Installation

Steps

Check if swap is disabled: cat /etc/fstab

see https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/install-kubeadm/#letting-iptables-see-bridged-traffic

cat <<EOF | sudo tee /etc/modules-load.d/k8s.conf
br_netfilter
EOF

cat <<EOF | sudo tee /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
EOF

sysctl --system

see https://kubernetes.io/docs/setup/production-environment/container-runtimes/#containerd

cat <<EOF | sudo tee /etc/modules-load.d/containerd.conf
overlay
br_netfilter
EOF

modprobe overlay

modprobe br_netfilter

cat <<EOF | sudo tee /etc/sysctl.d/99-kubernetes-cri.conf
net.bridge.bridge-nf-call-iptables  = 1
net.ipv4.ip_forward                 = 1
net.bridge.bridge-nf-call-ip6tables = 1
EOF

sudo sysctl --system

see https://docs.docker.com/engine/install/debian/

apt-get update

apt-get install -y ca-certificates curl gnupg lsb-release

curl -fsSL https://download.docker.com/linux/debian/gpg | sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg

echo \
"deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/debian \
$(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null

apt-get update

apt install apparmor

apt-get install containerd.io

apt-mark hold containerd.io

mkdir -p /etc/containerd

containerd config default > /etc/containerd/config.toml

systemctl restart containerd
  [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc.options]
    SystemdCgroup = true
  • restart containerd: systemctl restart containerd

see https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/install-kubeadm/#installing-kubeadm-kubelet-and-kubectl

apt-get update

apt-get install -y apt-transport-https ca-certificates curl

curl -fsSLo /usr/share/keyrings/kubernetes-archive-keyring.gpg https://packages.cloud.google.com/apt/doc/apt-key.gpg

echo "deb [signed-by=/usr/share/keyrings/kubernetes-archive-keyring.gpg] https://apt.kubernetes.io/ kubernetes-xenial main" | sudo tee /etc/apt/sources.list.d/kubernetes.list

apt-get update

apt-get install -y kubelet kubeadm kubectl

apt-mark hold kubelet kubeadm kubectl
  • install and start firewall - double check internal network adapter (enp7s0 or ens10)

  • disable ipv6

see

kubeadm init --control-plane-endpoint="<external-ip-of-load-balancer>:6443" --upload-certs --apiserver-advertise-address=<internal-ip> --pod-network-cidr=192.168.0.0/16

# then add more master nodes with (always manualy add --apiserver-advertise-address=<internal-ip>):
  kubeadm join <external-ip-of-load-balancer>:6443 --token <token> \
	--discovery-token-ca-cert-hash sha256:<hash> \
	--control-plane --certificate-key <cert-key> --apiserver-advertise-address=<internal-ip>
mkdir ~/.kube
scp root@<master_ip_or_hostname>:/etc/kubernetes/admin.conf ~/.kube/config
  • untaint master: kubectl taint nodes --all node-role.kubernetes.io/master-

Install Calico

helm repo add projectcalico https://docs.projectcalico.org/charts
helm repo update
helm install calico projectcalico/tigera-operator

Opional: Install metrics server

Install Longhorn

Network Debug

  • see open ports and more:

    • lsof -i -P -n | grep LISTEN

    • lsof -i -P -n | grep kubectl

Next Steps

service:
  externalIPs:
    - <the_external_ip>
helm repo add traefik https://helm.traefik.io/traefik
helm repo update
helm install traefik traefik/traefik -n traefik --create-namespace -f traefik-helm-values.yaml
  • expose the traefik dashboard:

    • k port-forward <traefik_pod> 9000:9000 -n traefik --address 0.0.0.0

    • http://:9000/dashboard/

Join new Nodes to an existing Cluster

  • show token: kubeadm token list

  • create new token: kubeadm token create

  • print join command: kubeadm token create --print-join-command