As your application grows, you can either scale the infrastructure horizontally or vertically. Usually, vertical scaling has a lot of limitations whereas horizontal scaling could potentially have unlimited capacity. In horizontal scaling, one problem that arises quickly is related to common storage, which needs to be accessed by several servers that too efficiently. A typical solution is to have NFS (network file system) configured. If you are on AWS, you could consider taking advantage of EFS i.e. Elastic File System.
Spot Instance
Everyone is aware of “cloud” now. The cloud provider sets up huge infrastructure and makes it available to their customers either as bare infrastructure or a managed service. In either case, it is a huge investment for the provider. All the servers, network bandwidth, and other resources are their inventory. As is the case with any business, idle inventory is a problem. And that is where Spot Instance comes into the picture.
Munin
You are a system administrator, managing a set of servers. As a part of your job, you need to ensure that all the servers are up and running 24X7, they are resources are utilized optimally. And that no single server is overloaded or consuming huge memory or some similar abnormal behavior over a time period. Obviously, you will be needing a monitoring tool. Munin is one such tool, which helps system administrators to keep track of system resources.
Munin Architecture
Munin derives its name from mythology. It means memory. This monitoring software memorizes what it sees and hence the name.
This monitoring tool uses Master-node architecture. Master periodically pulls data from all the configured nodes and processes it to create graphical dashboards.
One of the advantages of Munin is its vast plugin eco-system. There are several hundred plugins available, which make it a powerful monitoring system. Munin itself uses RRDTool, which is written in Perl, whereas plugins could be written in any language. This tool is available on various distros such as Debian, Ubuntu, and CentOS.
Alternate Server Monitoring Tools
Several alternatives are available.
- Nagios
- NewRelic
- Netdata
- Zabbix
- …
Considering that most of the deployments are in the cloud these days, you may want to use the monitoring tools provided by your cloud providers:
- AWS Cloudwatch
- Azure Monitor
- Google Cloud – Stackdriver monitoring
Related Links
Related Keywords
Server Monitoring, AWS Cloudwatch, Azure Monitor, Nagios, NewRelic
Filestack
You are launching a service, where your users need to upload their files to your app or web app. Your userbase is global. You are currently supporting only a handful of sources to pick files from. Also, it is observed that your users are leaving because file upload takes too long or fails due to a variety of issues. Filestack comes to your rescue.
But what is Filestack?
Filestack is a developer service that allows app developers to build apps which can connect to several sources and also upload the files reliably. It solves 2 problems:
- Making uploads reliable and faster
- Enabling users to upload files from several sources (25+)
The Filestack team has used an interesting model to make the uploads faster. It is a two-step process. The process uses an S3 bucket as an intermediate hop. S3 is a file storage solution by AWS, which has a presence across the globe. Using this network, Filestack uploads the user-provided file to the nearest S3 bucket. This turns out to be faster due to the elimination of network latency. As soon as the upload is completed, it returns URL to the application and application can start processing the file as per the need. In the background, the file from Filestack S3 bucket is moved to cloud storage specified by the app. The URL is updated to point to new location and app can continue to use that URL for rest of the lifecycle. So, even if it includes additional hop, it turns out to be faster from a user perspective.
Filestack not only solves above mentioned two problems, but it also provides several additional functionalities for image files.
- Image transformation such as adding borders, flipping the image, applying filters, image rotation
- Resizing of images – on the fly
- Image cropping
- Collage
You can check out the live examples here.
It also provides some advanced features based on ML (Machine Learning)
- Object recognition
- Object detection
- OCR (Optical Character Recognition)
- Copyright detection
Alternatives to Filestack:
Obviously, you can build your own connectors for various sources, but probably you would end up spending more time to implement all the functionality that Filestack is already providing. There have been some alternatives available:
- UploadCare
- Kloudless
- Filespin.io
Related Links
Related Keywords
Machine Learning, Google Vision, AWS S3
Vagrant
You are part of a development team and you want to try out the latest version of the VM that your application uses. Or you want to make some configuration changes and test them out in your development environment. However, you are not sure if your changes would be useful or successful. And hence you want an ability to roll back the changes or potentially version control them. How do you do it? Vagrant comes to the rescue!!
What is Vagrant?
Vagrant is an open source tool for building and maintaining portable development environments. With the rise of complex architectures involving several different servers and technology stacks, Vagrant simplifies the task of creating and maintaining the required stack of software/libraries etc.
Vagrant stores the configuration in the form of a text file(s) and these files could be put under your favorite version control system such as git. If changes don’t work out, you can easily roll back and go to the earlier working stage. This capability improves the development productivity a lot and hence Vagrant has become a darling of several development teams.
This tool uses a concept of “provisioners” and “providers”. Provisioners are tools that allow to customize/modify the environments – examples are Chef and/or Puppet. Whereas Providers are services which provide virtual machines such as AWS, Docker, VMWare etc.
One might be tempted to compare Vagrant with other configuration management tools such as Chef/Puppet/Ansible. However, Vagrant is commonly used along with one of these tools as they serve different purposes.
Related Links
Related Keywords
Configuration Mangement, Docker, Chef, Puppet, Ansible, VMWare, AWS, DevOps