Here is a diagram of the basic framework of the ELK, the convenience of this framework is that it is simple to build and easy to use. On the other side, Logstash takes up a lot of CPU and memory to run and it costs a lot of resources. In addition, there is no message queue cache, and there is a high potential for data loss.
This framework is distributed by Logstash on each node to collect relevant logs and data, and after analysis and filtering, it is sent to Elasticsearch on the remote server for storage. Elasticsearch compresses and stores data in shards and provides multiple APIs for users to query and operate. Users can also easily query logs by configuring Kibana Web, and generating reports based on the data.
The ELK also provides the basic functionality of a log management system. However, in ELK, LogStash often causes deadlocks when loading log files from multiple sources to a central server. Therefore, these three systems do not completely solve all the problems of centralized log management systems.
[1] http://eprints.uet.vnu.edu.vn/eprints/id/eprint/1994/1/paper113-van.nam.pdf
Leave a Reply