ElasticSearch is a powerful open-source search and analytics engine designed for scalability and real-time data retrieval. It is widely used for log and event data analysis, full-text search, and more. ElasticSearch operates as a distributed system, which means it can scale horizontally by adding more nodes to the cluster.
When working with ElasticSearch, you might encounter the MasterNotDiscoveredException. This error indicates that the cluster is unable to elect a master node. As a result, the cluster cannot function properly, leading to potential downtime or data unavailability.
When this exception occurs, you might notice that your ElasticSearch cluster is not responding to queries or indexing requests. The logs will typically contain entries similar to:
[WARN ][o.e.c.c.ClusterFormationFailureHelper] [node-1] master not discovered or elected yet, an election requires at least 2 nodes with ids from [node-1, node-2, node-3], have discovered [{node-1}{id}{ip}{port}] which is not a quorum; discovery will continue using [ip:port] from hosts providers and [{node-1}{id}{ip}{port}] from last-known cluster state; node term 0, last-accepted version 0 in term 0
The MasterNotDiscoveredException is primarily due to the cluster's inability to elect a master node. This can happen because of network partitions, incorrect discovery settings, or insufficient nodes to form a quorum.
Network partitions or misconfigurations can prevent nodes from communicating effectively, leading to this exception. Ensure that all nodes can reach each other over the network.
Incorrect settings in the elasticsearch.yml file, particularly those related to discovery and cluster formation, can also cause this issue. It's crucial to verify these settings are correct and consistent across all nodes.
To resolve the MasterNotDiscoveredException, follow these steps:
Ensure that all nodes in the cluster can communicate with each other. You can use tools like ping or telnet to test connectivity:
ping node-2
ping node-3
Check firewall settings and ensure that the necessary ports (default 9200 and 9300) are open.
Review the elasticsearch.yml configuration file on each node. Ensure that the discovery.seed_hosts and cluster.initial_master_nodes settings are correctly configured. For example:
discovery.seed_hosts: ["node-1", "node-2", "node-3"]
cluster.initial_master_nodes: ["node-1", "node-2", "node-3"]
For more details, refer to the ElasticSearch Discovery Settings Documentation.
Make sure that there are enough nodes to form a quorum. In a typical setup, a minimum of three nodes is recommended to avoid split-brain scenarios.
After verifying and correcting the configuration, restart the ElasticSearch nodes to apply the changes:
systemctl restart elasticsearch
By following these steps, you should be able to resolve the MasterNotDiscoveredException and restore your ElasticSearch cluster to a healthy state. For further assistance, consider visiting the ElasticSearch Discuss Forum where the community and experts can provide additional support.
(Perfect for DevOps & SREs)
(Perfect for DevOps & SREs)
