K3s NodeDiskPressure

What is K3s NodeDiskPressure

Understanding K3s and Its Purpose

K3s is a lightweight Kubernetes distribution designed for resource-constrained environments and edge computing. It simplifies the deployment and management of Kubernetes clusters by reducing the overhead and complexity typically associated with Kubernetes. K3s is particularly popular for IoT and edge devices due to its minimal resource requirements.

Identifying the Symptom: NodeDiskPressure

When working with K3s, you may encounter the NodeDiskPressure condition. This symptom indicates that a node is experiencing disk pressure, which can lead to pod eviction or scheduling issues. It is crucial to address this issue promptly to maintain the health and performance of your cluster.

Explaining the Issue: What is NodeDiskPressure?

The NodeDiskPressure condition is triggered when the kubelet detects that the available disk space on a node is below a certain threshold. This condition can cause Kubernetes to evict pods from the affected node to free up disk space, which can disrupt workloads and affect application availability. The root cause is typically insufficient disk space or excessive disk usage by pods or system processes.

Common Causes of Disk Pressure

Log files consuming excessive disk space. Large container images stored on the node. Persistent volumes not being cleaned up properly.

Steps to Fix NodeDiskPressure

To resolve the NodeDiskPressure issue, you need to free up disk space on the affected node or increase its disk capacity. Here are the steps you can follow:

Step 1: Identify Disk Usage

First, SSH into the affected node and check the disk usage using the following command:

df -h

This command will display the disk usage of all mounted filesystems. Look for any partitions that are near or at full capacity.

Step 2: Clean Up Unnecessary Files

Remove unnecessary files and logs that may be consuming disk space. For example, you can clear old log files using:

sudo journalctl --vacuum-time=1d

This command will remove journal logs older than one day.

Step 3: Remove Unused Docker Images

Docker images can consume significant disk space. Remove unused images with:

docker image prune -a

This command will remove all unused images, freeing up disk space.

Step 4: Increase Disk Capacity

If disk usage is still high after cleaning up, consider increasing the disk capacity of the node. This may involve resizing the disk in your cloud provider's console or adding additional storage to the node.

Additional Resources

For more detailed information on managing disk pressure in Kubernetes, refer to the official Kubernetes documentation. Additionally, the K3s documentation provides insights into managing K3s clusters effectively.