Envoy is a high-performance open-source edge and service proxy designed for cloud-native applications. It is used to manage network traffic, providing features such as load balancing, service discovery, and observability. Envoy is often deployed as a sidecar in service mesh architectures, enhancing the communication between microservices.
One common issue encountered when using Envoy is the 'Header Size Exceeded' error. This occurs when the HTTP headers in a request or response exceed the configured size limit. This can lead to requests being rejected or responses not being processed correctly.
When this issue occurs, you may notice that requests are failing with HTTP 431 status codes, or you might see log entries indicating that the header size limit has been exceeded. This can disrupt communication between services and degrade the performance of your application.
The 'Header Size Exceeded' error is triggered when the cumulative size of HTTP headers surpasses the maximum size configured in Envoy. This limit is in place to prevent potential denial-of-service attacks and to ensure efficient resource usage. However, in some cases, legitimate requests may exceed this limit due to large cookies, numerous custom headers, or other factors.
The root cause of this issue is typically a misconfiguration of the header size limit in Envoy, or an unexpected increase in the size of headers being sent by clients or services. It is essential to identify whether the increase in header size is justified or if it indicates an underlying problem, such as excessive data being sent in headers.
To resolve this issue, you can either increase the header size limit in Envoy or reduce the size of the headers being sent. Here are the steps to address this problem:
If the increase in header size is legitimate, you can adjust the configuration in Envoy to accommodate larger headers. Modify the max_request_headers_kb parameter in your Envoy configuration file:
static_resources:
listeners:
- name: listener_0
address:
socket_address: { address: 0.0.0.0, port_value: 10000 }
filter_chains:
- filters:
- name: envoy.filters.network.http_connection_manager
typed_config:
"@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
stat_prefix: ingress_http
codec_type: AUTO
route_config:
name: local_route
virtual_hosts:
- name: local_service
domains: ["*"]
routes:
- match: { prefix: "/" }
route: { cluster: some_service }
http_filters:
- name: envoy.filters.http.router
max_request_headers_kb: 128
In this example, the max_request_headers_kb is set to 128 KB. Adjust this value according to your needs.
If increasing the header size limit is not feasible, consider reducing the size of the headers. This can be achieved by:
For more detailed information on configuring Envoy, refer to the Envoy Documentation. Additionally, you can explore the Envoy Threat Model to understand how header size limits contribute to security.
By following these steps, you can effectively manage the 'Header Size Exceeded' issue in Envoy, ensuring smooth and reliable communication between your services.
Let Dr. Droid create custom investigation plans for your infrastructure.
Book Demo
