Envoy is an open-source edge and service proxy designed for cloud-native applications. It is used to manage network traffic, providing features such as load balancing, service discovery, and observability. Envoy is often deployed as a sidecar proxy in service mesh architectures, enhancing the communication between microservices.
One common issue encountered when using Envoy is the 'Upstream Connection Timeout'. This occurs when Envoy is unable to establish a connection with an upstream server within a specified timeframe. The error is typically observed in the logs with messages indicating a timeout.
When this issue occurs, you might notice log entries similar to:
[error][upstream] [source/common/upstream/cluster_manager_impl.cc:123] connection to upstream timed out
The primary cause of an upstream connection timeout is that Envoy cannot connect to the upstream server within the configured timeout period. This can be due to various reasons such as network latency, server unavailability, or incorrect timeout settings.
To address this issue, you can follow these steps:
Ensure that Envoy can reach the upstream server. You can use tools like ping or curl to test connectivity:
ping upstream-server-address
curl -I http://upstream-server-address
Review and adjust the timeout settings in your Envoy configuration file. The timeout can be configured in the cluster settings:
clusters:
- name: service_cluster
connect_timeout: 5s
Increase the connect_timeout value if necessary.
Check the performance and load on the upstream server. Ensure it is capable of handling incoming requests promptly. Consider scaling resources if the server is under heavy load.
For more detailed information on configuring Envoy, refer to the official Envoy Configuration Overview. If you are new to Envoy, the Getting Started Guide is a great place to begin.
Let Dr. Droid create custom investigation plans for your infrastructure.
Book Demo
