Logstash

These days a typical deployment consists of at least a couple of servers, behind a load balancer and a database. As your application complexity grows, you continue to add more components. Then you start collecting tons of messages from logs of all these applications. And the problem starts there! It is not easy to keep track of logs from all these components and detect patterns or anomalies which could trigger further action. Logstash helps solve this problem.

What is Logstash?

In simple terms, Logstash is a data processing pipeline, that ingests data from one or more disparate sources. It transforms it as per your configuration and then sends it to the destination.

Gather data from multiple sources and send it to your favorite stash – Logstash

The source could be a web server log, application logs, metrics, time series data or even unstructured data. Logstash can handle it all. The possible transformations are deriving structure from unstructured data using Grok, separate out IP addresses, anonymize personally identifiable information (PII). This is achieved using any of the filters available from this list.

Ecosystem

Logstash forms part of the ELK stack that is promoted by Elastic. ELK is a stack comprising of Elasticsearch, Logstash and Kibana. All three products have their open source versions and are very popular amongst the community.

Reference Links

Related Keywords

Elasticsearch, Kibana, Kubernetes, Prometheus

Leave a Reply

Your email address will not be published.

This site uses Akismet to reduce spam. Learn how your comment data is processed.