Kube-probe TCP probe failed: host unreachable

The probe cannot reach the host due to network issues.

Understanding Kube-probe

Kube-probe is a diagnostic tool used in Kubernetes to monitor the health of applications running in a cluster. It helps ensure that applications are running smoothly by checking their readiness and liveness. Probes can be configured to use HTTP, TCP, or command execution to verify the application's state.

Identifying the Symptom

One common issue encountered with Kube-probe is the error message: TCP probe failed: host unreachable. This indicates that the probe is unable to establish a TCP connection to the specified host, which can lead to the application being marked as unhealthy.

Explaining the Issue

The error TCP probe failed: host unreachable typically arises when the network configuration prevents the probe from reaching the target host. This could be due to incorrect IP addresses, network policies, or firewall rules blocking the connection.

Common Causes

  • Incorrect service or pod IP address configuration.
  • Network policies restricting traffic to the host.
  • Firewall rules blocking the TCP connection.

Steps to Fix the Issue

To resolve the TCP probe failed: host unreachable issue, follow these steps:

Step 1: Verify Network Configuration

Ensure that the network configuration allows the probe to reach the host. Check the IP address and port specified in the probe configuration:

kubectl describe pod <pod-name> | grep -A 10 'Liveness:'

Verify that the IP address and port are correct and accessible.

Step 2: Check Network Policies

Review any network policies that might be restricting traffic to the host. Use the following command to list network policies:

kubectl get networkpolicy -n <namespace>

Ensure that the policies allow traffic from the probe to the target host.

Step 3: Inspect Firewall Rules

Check the firewall settings to ensure that they are not blocking the TCP connection. You may need to consult your cloud provider's documentation for specific firewall configurations. For example, see Google Cloud Firewall Rules.

Step 4: Test Connectivity

Manually test the connectivity from the probe's network to the host using tools like telnet or nc:

telnet <host-ip> <port>

If the connection fails, further investigate network configurations and permissions.

Conclusion

By following these steps, you should be able to diagnose and resolve the TCP probe failed: host unreachable issue. Ensuring proper network configurations and permissions is key to maintaining the health of applications in a Kubernetes cluster. For more information on configuring probes, refer to the Kubernetes Probes Documentation.

Master

Kube-probe

in Minutes — Grab the Ultimate Cheatsheet

(Perfect for DevOps & SREs)

Most-used commands
Real-world configs/examples
Handy troubleshooting shortcuts
Your email is safe with us. No spam, ever.

Thankyou for your submission

We have sent the whitepaper on your email!
Oops! Something went wrong while submitting the form.

Kube-probe

Cheatsheet

(Perfect for DevOps & SREs)

Most-used commands
Your email is safe with us. No spam, ever.

Thankyou for your submission

We have sent the whitepaper on your email!
Oops! Something went wrong while submitting the form.

MORE ISSUES

Made with ❤️ in Bangalore & San Francisco 🏢

Doctor Droid