Elastic Search Expert

I have vast experience designing, sizing, and running Elasticsearch in production systems. Recently, I have co-authored the book "Learning Elastic Stack 6.0" with Packt Publishing. Once a node goes down or is deemed to be down by other nodes (due to network partition), it usually takes some time to recover the shards on that node. Aggregation queries also depend on the structure of data and cardinality of the fields on which the aggregation is done. Additionally, it depends on fielddata cache available. The diagnosis of the problem would require further details and access to the real environment - 1) Elasticsearch version 2) Node size & data size 3) Structure of documents and data types 4) Type of aggregation queries I am sure I can be of help in diagnosing and fixing the problem in your cluster.

A proposal has not yet been provided

What ES Version are you running? The 30 min recovery time is not unusual at all. How many shards you have per node, and what is the shard size?

Hello that issue of node recovery, it is totally dependent on the Size of the data and the shards that need to be recovered after the node is back online, there are some ways to improve this recovery speed by editing some settings like node_concurrent_recoveries, indices.recovery.max_bytes_per_sec and some more settings. And they are methods like disabling or delaying allocation when node is offline. as for the 2nd option you will be able to view manage and terminate queries with Task Management API. its in beta state, but it works flawlessly for now,

Hello, I worked in Elasticsearch for a long time. Can I access and view your system? Thanks Relevant Skills and Experience ELK Spark

Please have a look at my profile. I have 12 years of experience developing large scale enterprise applications using java. If you are interested kindly contact me. Technical Expertise: Spring Boot, Hibernate, Spring Data JPA, JMS, Elastic search, Web Service(SOAP/Restful), AWS cloud computing (S3, VPC, EC2, ELB, SQS, RDS, Cloud Watch, Cloud formation script), MQ, ESB, Message Broker, Mongo DB, Multithreading, Micro services. Since last two year I have been working in elastic search and recently I have upgraded to 5.6.4. Regarding to the problem statements 1. If the node is taking 30 minutes to recover, there could be multiple reasons. One reason could be infrastructure issue. How many masters/data nodes are there also does matter. However if one node goes down it should not impact the search because data is replicated across different data nodes. 2. Aggregation queries are always heavy. It is also important on the field type(analyzed/not analyzed) on which aggregation queries are fired. What is the Elastic version currently being used?