Everyone is aware of “cloud” now. The cloud provider sets up huge infrastructure and makes it available to their customers either as bare infrastructure or a managed service. In either case, it is a huge investment for the provider. All the servers, network bandwidth, and other resources are their inventory. As is the case with any business, idle inventory is a problem. And that is where Spot Instance comes into the picture.
Munin
You are a system administrator, managing a set of servers. As a part of your job, you need to ensure that all the servers are up and running 24X7, they are resources are utilized optimally. And that no single server is overloaded or consuming huge memory or some similar abnormal behavior over a time period. Obviously, you will be needing a monitoring tool. Munin is one such tool, which helps system administrators to keep track of system resources.
Munin Architecture
Munin derives its name from mythology. It means memory. This monitoring software memorizes what it sees and hence the name.
This monitoring tool uses Master-node architecture. Master periodically pulls data from all the configured nodes and processes it to create graphical dashboards.
One of the advantages of Munin is its vast plugin eco-system. There are several hundred plugins available, which make it a powerful monitoring system. Munin itself uses RRDTool, which is written in Perl, whereas plugins could be written in any language. This tool is available on various distros such as Debian, Ubuntu, and CentOS.
Alternate Server Monitoring Tools
Several alternatives are available.
- Nagios
- NewRelic
- Netdata
- Zabbix
- …
Considering that most of the deployments are in the cloud these days, you may want to use the monitoring tools provided by your cloud providers:
- AWS Cloudwatch
- Azure Monitor
- Google Cloud – Stackdriver monitoring
Related Links
Related Keywords
Server Monitoring, AWS Cloudwatch, Azure Monitor, Nagios, NewRelic
Filestack
You are launching a service, where your users need to upload their files to your app or web app. Your userbase is global. You are currently supporting only a handful of sources to pick files from. Also, it is observed that your users are leaving because file upload takes too long or fails due to a variety of issues. Filestack comes to your rescue.
But what is Filestack?
Filestack is a developer service that allows app developers to build apps which can connect to several sources and also upload the files reliably. It solves 2 problems:
- Making uploads reliable and faster
- Enabling users to upload files from several sources (25+)
The Filestack team has used an interesting model to make the uploads faster. It is a two-step process. The process uses an S3 bucket as an intermediate hop. S3 is a file storage solution by AWS, which has a presence across the globe. Using this network, Filestack uploads the user-provided file to the nearest S3 bucket. This turns out to be faster due to the elimination of network latency. As soon as the upload is completed, it returns URL to the application and application can start processing the file as per the need. In the background, the file from Filestack S3 bucket is moved to cloud storage specified by the app. The URL is updated to point to new location and app can continue to use that URL for rest of the lifecycle. So, even if it includes additional hop, it turns out to be faster from a user perspective.
Filestack not only solves above mentioned two problems, but it also provides several additional functionalities for image files.
- Image transformation such as adding borders, flipping the image, applying filters, image rotation
- Resizing of images – on the fly
- Image cropping
- Collage
You can check out the live examples here.
It also provides some advanced features based on ML (Machine Learning)
- Object recognition
- Object detection
- OCR (Optical Character Recognition)
- Copyright detection
Alternatives to Filestack:
Obviously, you can build your own connectors for various sources, but probably you would end up spending more time to implement all the functionality that Filestack is already providing. There have been some alternatives available:
- UploadCare
- Kloudless
- Filespin.io
Related Links
Related Keywords
Machine Learning, Google Vision, AWS S3
Arduino
Arduino is an open source hardware and software platform, which can be used for building electronics projects. It consists of an electronics board and a software to work with that board. It also comes with an IDE (integrated development environment) which runs on your PC.
What all things Arduino can do?
Arduino can interact with buttons, LEDs, sensors, motors and several other electronic devices. You will need separate connector and cables though. To state some sample projects – 3D Printer, Robots, Drones, Home Automation etc.
Arduino is a brand name, however, their hardware and software are available under open-source. Even the circuit diagrams are available, in case someone wants to create the board themselves! As a result, there are other clones also available for Arduino hardware.
Each board can work independently after loading the program initially. Or it can work in tandem with other Arduino boards to provide additional functionality. Or it can also remain connected with a computer to carry out tasks which need access to data from sensors. A board contains following components:
- USB connector – power as well as connectivity with computer
- Various pins such a Ground, 3.3V, 5V, analog and digital
- RESET button
- Power Indicator
- Transmitter and Receiver
- Main IC (Integrated Circuit)
Arduino family contains multiple boards:
- Uno
- LilyPad
- RedBoard
- Leonardo
Some of the projects that you can build using this board:
- Smart blind stick – one can use sensor to detect the obstacles and give feedback using a buzzer
- Metal Detector
- Measure Sound Level
- Club it with Thermistor and you can control temperature in your room
You can find a huge list on the internet for such projects and also quite a few videos on youtube.
Related Links
Related Keywords
Hardware, Electronics, DIY
ESB
Ever wondered how heterogeneous applications communicate with each other? e.g. A Java application that needs to communicate with a .Net application. And not just one-to-one communication. Consider use cases where there are several applications talking to more than one applications. How does it happen? Does each application need to implement an interface for every other application? Enterprise Service Bus i.e. ESB comes to your rescue.
What is Enterprise Service Bus?
ESB implements a communication system between various applications which need to communicate with each other. The primary goal of ESB is to enable faster integration between Enterprise applications commonly referred as EAI.
Consider a set of applications which need to communicate with each other. This is a heterogeneous set and each one talks separate standard, protocol. And many of them need to talk to more than one applications. ESB removes the need that each application needs to implement communication with more than one other applications. If one tries to implement point-to-point communication between apps, it is going to get messy. Changing one part would result in breaking something else. ESB handles the communication transparently and hence eases up the process of communication.
Key capabilities of ESB
- Transport protocol conversion – JMS to file
- Message transformation
- Routing e.g. content based routing
- Message enhancement e.g. Order data that only has a client ID can be enhanced to pass Order Data and client details together to next level. In this case, client details are pulled from a different source and the incoming message is “enhanced” with additional client data before passing it to next application.
- Security / Authorization – incoming messages are checked for required authorization before processing them further.
- Mediation
- Monitoring
Key providers of Enterprise Service Bus software
- Microsoft BizTalk Server
- MuleSoft
- Oracle Enterprise Service Bus
- Tibco Software
- Apache ServiceMix
Related Links
Related Keywords
Software Architecture, Service Oriented Architecture