Kube-probe Readiness probe failed: dependency service down

A dependent service required by the application is not available.

Understanding Kube-probe

Kube-probe is a diagnostic tool used in Kubernetes to check the health of containers running within pods. It helps ensure that applications are running smoothly by performing liveness and readiness checks. These probes are crucial for maintaining the desired state of applications and ensuring high availability.

Identifying the Symptom

One common symptom that developers encounter is the readiness probe failure, often indicated by an error message such as: Readiness probe failed: dependency service down. This message suggests that the application is not ready to serve traffic due to an unavailable dependent service.

What You Observe

When this issue occurs, you may notice that the application is not receiving traffic, and the pod status may show as NotReady. This can lead to disruptions in service availability and user experience.

Exploring the Issue

The readiness probe failure indicates that a service required by your application is not accessible. This could be due to network issues, the service not being started, or incorrect service configurations. The readiness probe checks if the application is ready to handle requests, and if it fails, Kubernetes will not route traffic to the pod.

Common Causes

Network connectivity issues between the application and the dependent service.
The dependent service is not running or has crashed.
Misconfiguration in service endpoints or DNS issues.

Steps to Fix the Issue

To resolve the readiness probe failure, follow these steps:

1. Check Dependent Services

Ensure that all dependent services are running. You can use the following command to check the status of services:

kubectl get services

Verify that the services your application depends on are listed and running.

2. Verify Network Connectivity

Use the following command to check network connectivity between your application pod and the dependent service:

kubectl exec <pod-name> -- curl -I http://<service-name>:<port>

This command attempts to connect to the service from within the pod. If it fails, investigate network policies or DNS configurations.

3. Inspect Logs

Check the logs of both your application and the dependent service for any errors or warnings:

kubectl logs <pod-name>

Look for any indications of why the service might be unavailable or why the probe is failing.

4. Update Readiness Probe Configuration

If the issue persists, review and update the readiness probe configuration in your deployment YAML file. Ensure the probe is correctly configured to check the right endpoints and ports.

readinessProbe: httpGet: path: /healthz port: 8080 initialDelaySeconds: 5 periodSeconds: 10

Further Reading

For more information on configuring probes in Kubernetes, refer to the official Kubernetes documentation. Additionally, explore Kubernetes Services to understand how services are managed and accessed.

Master

Kube-probe

in Minutes — Grab the Ultimate Cheatsheet

(Perfect for DevOps & SREs)

Most-used commands

Real-world configs/examples

Handy troubleshooting shortcuts

Thankyou for your submission

We have sent the whitepaper on your email!

Oops! Something went wrong while submitting the form.

Kube-probe

Cheatsheet

(Perfect for DevOps & SREs)

Most-used commands

Thankyou for your submission

We have sent the whitepaper on your email!

Oops! Something went wrong while submitting the form.

MORE ISSUES

Kube-probe Probe failed: invalid probe configuration

The probe configuration contains invalid parameters or syntax errors.

Kube-probe Probe failed: invalid response length

The application returned a response with an unexpected length.

Kube-probe Exec probe failed: file not found

The file required by the exec probe command is not present in the container.

Kube-probe TCP probe failed: address in use

The port specified for the TCP probe is already in use by another process.

Kube-probe Probe failed: invalid character encoding

The application returned a response with an unexpected character encoding.

Kube-probe HTTP probe failed with status code 429

The application is rate limiting requests and the probe exceeded the limit.

Kube-probe Liveness probe failed: application out of memory

The application has run out of memory and cannot respond to the probe.

Kube-probe Readiness probe failed: application misconfiguration

The application is misconfigured and cannot become ready.

Kube-probe HTTP probe failed with status code 405

The application does not allow the HTTP method used by the probe.

Kube-probe Exec probe failed: insufficient memory

The command executed by the exec probe requires more memory than available.

Kube-probe Probe failed: invalid JSON response

The application returned a JSON response that is not well-formed.

Kube-probe HTTP probe failed with status code 408

The application took too long to respond to the probe request.

Kube-probe Liveness probe failed: application deadlock

The application is in a deadlock state and cannot respond to the probe.

Kube-probe TCP probe failed: protocol error

There is a protocol mismatch between the probe and the application.

Kube-probe Probe failed: invalid SSL certificate

The SSL certificate used by the application is invalid or expired.

Kube-probe Probe failed: invalid content type

The application returned a response with an unexpected content type.

Kube-probe Readiness probe failed: application initialization error

The application failed to initialize properly.

Kube-probe Probe failed: invalid response code

The application returned an unexpected HTTP status code.

Kube-probe HTTP probe failed with status code 302

The application is redirecting the probe request temporarily.

Kube-probe Exec probe failed: environment variable missing

The command executed by the exec probe relies on an environment variable that is not set.

Kube-probe HTTP probe failed with status code 400

The application received a bad request from the probe.

Kube-probe TCP probe failed: host unreachable

The probe cannot reach the host due to network issues.

Kube-probe Probe failed: invalid header format

The probe is sending headers in an incorrect format.

Kube-probe HTTP probe failed with status code 404

The HTTP probe is trying to access a path that does not exist on the application.

Kube-probe Probe failed: unsupported protocol

The probe is attempting to use a protocol not supported by the application.

Kube-probe Liveness probe failed: resource starvation

The application is unable to respond to the probe due to lack of resources.

Kube-probe Probe failed: malformed URL

The URL specified in the probe configuration is not correctly formatted.

Kube-probe HTTP probe failed with status code 301

The application is redirecting the probe request to a different URL.

Kube-probe Readiness probe failed: dependency service down

A dependent service required by the application is not available.

Kube-probe Exec probe failed: exit code non-zero

The command executed by the exec probe returned a non-zero exit code, indicating failure.

Kube-probe TCP probe failed: connection timed out

The probe is unable to establish a connection to the application within the timeout period.

Kube-probe Probe failed: invalid HTTP method

The probe is using an HTTP method that the application does not support.

Kube-probe HTTP probe failed with status code 503

The application is temporarily unavailable, possibly due to overload or maintenance.

Kube-probe Probe failed: SSL handshake error

There is an issue with the SSL/TLS configuration between the probe and the application.

Kube-probe Liveness probe failed: application crash

The application has crashed and is unable to respond to the liveness probe.

Kube-probe Readiness probe failed: resource limit exceeded

The application is unable to allocate necessary resources to handle the probe request.

Kube-probe TCP probe failed: network unreachable

The network is not properly configured, preventing the probe from reaching the application.

Kube-probe Probe failed: invalid response format

The application returned a response in an unexpected format that the probe cannot interpret.

Kube-probe Exec probe failed: permission denied

The command specified in the exec probe lacks the necessary permissions to execute.

Kube-probe HTTP probe failed with status code 502

The application is acting as a gateway or proxy and received an invalid response from the upstream server.

Kube-probe HTTP probe failed with status code 403

The application is denying access to the probe due to insufficient permissions.

Kube-probe HTTP probe failed with status code 401

The probe is not authenticated to access the endpoint.

Kube-probe Readiness probe failed: service unavailable

The application is not ready to serve traffic.

Kube-probe Liveness probe failed: container not running

The container has crashed or is not running.

Kube-probe Probe failed: DNS resolution error

The probe is unable to resolve the DNS name of the application.

Kube-probe Probe failed: no route to host

Network connectivity issues between the probe and the application.

Kube-probe Exec probe failed: command not found

The command specified in the exec probe is not available in the container's environment.

Kube-probe Liveness probe failed: HTTP 500

The application is returning an internal server error for the liveness probe request.

Kube-probe TCP probe failed: connection refused

The application is not listening on the specified port, or the service is not running.

Kube-probe Readiness probe failed: timeout

The application is taking too long to respond to the readiness probe.

Backed by

Resources

Contact

Platform

Connect

Made with ❤️ in & 🏢

Doctor Droid