Logstash

These days a typical deployment consists of at least a couple of servers, behind a load balancer and a database. As your application complexity grows, you continue to add more components. Then you start collecting tons of messages from logs of all these applications. And the problem starts there! It is not easy to keep track of logs from all these components and detect patterns or anomalies which could trigger further action. Logstash helps solve this problem.

Continue reading “Logstash”

Prometheus

Nowadays, having several servers, load-balancers, API Gateways, and storage devices is a common thing. However, keeping the infrastructure up and running is still a complex task. The administrators need to identify potential problems due to the high availability requirement of several businesses. They strive to have the best tools at their disposal to ensure that they can see the trends and patterns about the health of the infrastructure. Prometheus is an open-source tool that is used to record real-time metrics and making it available through a flexible query model.

Continue reading “Prometheus”

EFS

As your application grows, you can either scale the infrastructure horizontally or vertically. Usually, vertical scaling has a lot of limitations whereas horizontal scaling could potentially have unlimited capacity. In horizontal scaling, one problem that arises quickly is related to common storage, which needs to be accessed by several servers that too efficiently. A typical solution is to have NFS (network file system) configured. If you are on AWS, you could consider taking advantage of EFS i.e. Elastic File System.

Continue reading “EFS”

Spot Instance

Everyone is aware of “cloud” now. The cloud provider sets up huge infrastructure and makes it available to their customers either as bare infrastructure or a managed service. In either case, it is a huge investment for the provider. All the servers, network bandwidth, and other resources are their inventory. As is the case with any business, idle inventory is a problem. And that is where Spot Instance comes into the picture.

Continue reading “Spot Instance”

Munin

You are a system administrator, managing a set of servers. As a part of your job, you need to ensure that all the servers are up and running 24X7, they are resources are utilized optimally. And that no single server is overloaded or consuming huge memory or some similar abnormal behavior over a time period. Obviously, you will be needing a monitoring tool. Munin is one such tool, which helps system administrators to keep track of system resources.

Munin Architecture

Munin derives its name from mythology. It means memory. This monitoring software memorizes what it sees and hence the name.

This monitoring tool uses Master-node architecture. Master periodically pulls data from all the configured nodes and processes it to create graphical dashboards.

Munin Architecture
Munin Architecture – Source: http://guide.munin-monitoring.org/en/latest/architecture/index.html

One of the advantages of Munin is its vast plugin eco-system. There are several hundred plugins available, which make it a powerful monitoring system. Munin itself uses RRDTool, which is written in Perl, whereas plugins could be written in any language. This tool is available on various distros such as Debian, Ubuntu, and CentOS.

Munin-memory-week
Munin graph depicting memory usage over a week
Once data is collected from nodes, it is processed by the master and various graphs are made available on the dashboard. One of the components is – Munin-limits – which monitors the values between configured “ok”, “warn” and “crit” levels and can create a notification for the administrators.

Alternate Server Monitoring Tools

Several alternatives are available.

  • Nagios
  • NewRelic
  • Netdata
  • Zabbix

Considering that most of the deployments are in the cloud these days, you may want to use the monitoring tools provided by your cloud providers:

  • AWS Cloudwatch
  • Azure Monitor
  • Google Cloud – Stackdriver monitoring

Related Links

Related Keywords

Server Monitoring, AWS Cloudwatch, Azure Monitor, Nagios, NewRelic