kubeadm cluster configuration | ArkIT

Kubernetes, often abbreviated as K8s, is a powerful open-source container orchestration platform that automates the deployment, scaling, and management of containerized applications. Originally developed by Google and later donated to the Cloud Native Computing Foundation (CNCF), Kubernetes has emerged as the de facto standard for container orchestration in modern cloud-native environments. kubeadm cluster configuration

Benefits of Kubernetes:

• Portability: Kubernetes abstracts away the underlying infrastructure, making it possible to run applications consistently across various cloud providers or on-premises data centers.
• Scalability: Applications can easily scale up or down based on demand, ensuring optimal resource utilization and performance.
• High Availability: Kubernetes supports the deployment of applications in a highly available manner, minimizing downtime and ensuring continuous service availability.
• Resource Efficiency: The platform optimizes resource utilization by scheduling containers based on available resources, preventing both underutilization and overutilization.
• Automated Rollouts and Rollbacks: Kubernetes facilitates seamless application updates by automating the rollout of new versions and providing easy rollback mechanisms in case of issues.
• Declarative Configuration: Desired state configurations enable users to declare the state they want, allowing Kubernetes to handle the complexities of achieving and maintaining that state.
• Ecosystem Integration: Kubernetes has a rich ecosystem of tools and extensions that enhance its capabilities, covering areas such as monitoring, logging, and security.

Prerequisites

1. 3 Servers (If AWS t2.medium EC2 type)
2. OS Ubuntu 22.04 LTS
3. 2 vCPUs or more
4. Minimum 4GB RAM
5. Familiarity with the Kubernetes Components.
6. On-Primises should have three VMs or Three Physical servers with communication enabled between them (Network)

Controller and Worker Nodes Setup

Configure Network Prerequisites

This step configures IPv4 forwarding and lets iptables see bridged traffic. The overlay module is needed by Docker to work with the overlay network driver. The br_netfilter module is needed by Kubernetes to enable network filtering and NAT (Network Address Translation).

Refer to this Kubernetes documentation for more information.

cat <<EOF | sudo tee /etc/modules-load.d/k8s.conf
overlay
br_netfilter
EOF

Insert the Kernel module 

sudo modprobe overlay
sudo modprobe br_netfilter

Create the k8s configuration file for bridge iptables

cat <<EOF | sudo tee /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-iptables = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.ipv4.ip_forward = 1
EOF

sudo sysctl --system

Check and confirm modules are inserted into the kernel level

lsmod | grep overlay

lsmod | grep br_netfilter

Verify that the net.bridge.bridge-nf-call-iptables, net.bridge.bridge-nf-call-ip6tables, and net.ipv4.ip_forward system variables are set to 1 in your sysctl config by running the following command:

sysctl net.bridge.bridge-nf-call-iptables net.bridge.bridge-nf-call-ip6tables net.ipv4.ip_forward

=== Output ===
net.bridge.bridge-nf-call-iptables = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.ipv4.ip_forward = 1

Ensure to run these steps to configure the required network settings on all nodes.

Configure container runtime

This step configures container runtime for all nodes on the cluster. Kubernetes 1.28 requires that you use a runtime that conforms with the Container Runtime Interface (CRI), which is an API for container run-times to work with kubelet. We will use containerd as the container runtime for this Kubernetes cluster.

Refer to this Kubernetes documentation for more information.

1. Install Docker Engine (as it includes containerd runtime).

   Refer to this Docker documentation for more information.

   1. Update the apt package index, install the required packages, and get the official GPG keys for the Docker package repositories:

sudo apt-get update
sudo apt-get install ca-certificates curl gnupg
sudo install -m 0755 -d /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
sudo chmod a+r /etc/apt/keyrings/docker.gpg

Add the docker repository to the apt sources

echo \
"deb [arch="$(dpkg --print-architecture)" signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu \
"$(. /etc/os-release && echo "$VERSION_CODENAME")" stable" | \
sudo tee /etc/apt/sources.list.d/docker.list > /dev/null

update the apt package cache

sudo apt-get update

Install Docker (Including containerd)

sudo apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin

Make Daemon file for docker to avoid service errors later

sudo mkdir /etc/docker
cat <<EOF | sudo tee /etc/docker/daemon.json
{
"exec-opts": ["native.cgroupdriver=systemd"],
"log-driver": "json-file",
"log-opts": {
"max-size": "100m"
},
"storage-driver": "overlay2"
}
EOF

sudo systemctl enable docker
sudo systemctl daemon-reload
sudo systemctl restart docker

Confirm that Docker Engine installation is sucessful by running the hello-world image

sudo docker run hello-world

Switchoff or Disable the Swap permanently in all the nodes

sudo swapoff -a
sudo sed -i '/ swap / s/^\(.*\)$/#/g' /etc/fstab

Configure the containerd runtime environment

Refer to this Kubernetes documentation for more information.

Login as  a root user

sudo -s

Create a default containerd configuration file using below command

containerd config default > /etc/containerd/config.toml

Open config.toml in a any text editor

vi /etc/containerd/config.toml

Change the value of SystemdCgroup from false to true (it should be visible around line number 125 in config.toml)

SystemdCgroup = true

Restart containerd service

systemctl restart containerd

Exit from root user

exit

Ensure to run these steps to configure the container runtime on all nodes.

Install kubeadm, kubelet, and kubectl

In this step, we install these packages on all nodes:

• kubeadm: The command to bootstrap the cluster
• kubelet: An agent that runs on all nodes in the cluster and does things like starting pods and containers
• kubectl: The command line utility to talk to the cluster

Refer to this Kubernetes documentation for more information.

Since we are using Ubuntu 22.04 LTS, we will use instructions for Debian-based distributions.

Update the apt package index, install the required packages, and get the official GPG keys for the Kubernetes package repositories:

sudo apt-get update
sudo apt-get install -y apt-transport-https ca-certificates curl
curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.28/deb/Release.key | sudo gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg

Add the Kubernetes repository to the apt sources

echo 'deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/v1.28/deb/ /' | sudo tee /etc/apt/sources.list.d/kubernetes.list

Update the apt package cache and index, and install packages

sudo apt-get update
sudo apt install -y kubelet kubeadm kubectl

Avoid automatic update of above packages to it wont be in version mismatch later

sudo apt-mark hold kubelet kubeadm kubectl

Ensure to install kubeadm, kubelet and kubectl on all nodes

Execute Only on Controller Node

Run these commands only on the VM designated as the controller (master) node.

sudo -s

Initialize the API server, Remember to change the <ControllerVM-IP> with the actual IP address (In case Cloud instance use private IP)

kubeadm init --apiserver-advertise-address=<ControllerVM-IP> --pod-network-cidr=10.244.0.0/16 
exit

run as normal user kube config

mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

Deploy Weave network config

kubectl apply -f https://github.com/weaveworks/weave/releases/download/v2.8.1/weave-daemonset-k8s.yaml

Create Cluster join command

sudo -s
kubeadm token create --print-join-command

Worker Nodes ONLY

Copy the output from the above comand cluster join step and run on the worker nodes.

Example:

sudo kubeadm join 192.168.175.100:6443 --token jno9md.v9u1snltrwkv3vix --discovery-token-ca-cert-hash sha256:74b58d0e840e43a7051dcc4d7388b836dbd187c732fbfd3008051d10a14e271a

The Kuberneted cluster is now configured

Now check the status of the nodes and cluster (RUN as NORMAL user)

kubectl get nodes

Thats all explained Kubernetes Cluster setup like production.