How many nodes should the cluster have? It’s a difficult question. Ultimately, it will boil down to questions like the following:

1. How much data are you working with?
2. How many searches will you be processing?
3. How complex are your searches?
4. How much resources will each node have to work with?
5. How many indexes/applications will you be working with?

The answer to that question depends on a lot of factors, like expected load, data size, hardware, etc. In this tutorial post we discuss how to avoid the split brain problem.

We recently announced Qbox hosted ElastAlert — the superb open-source alerting tool built by the team at Yelp Engineering — now available on all new Elasticsearch clusters on AWS.

Most organizations use the ELK Stack for managing their ever increasing amount of data and logs. Kibana is great for visualizing and querying data, but it needs a companion tool like ElastAlert for alerting on inconsistencies, anomalies, spikes, or other patterns of interest from data in Elasticsearch.

Redis, the popular open source in-memory data store, has been used as a persistent on-disk database that supports a variety of data structures such as lists, sets, sorted sets (with range queries), strings, geospatial indexes (with radius queries), bitmaps, hashes, and Hyper Logs. The in-memory store is used to solve various problems in areas such as real-time messaging, caching, and statistic calculation.

Provisioning an Elasticsearch cluster in Qbox is easy. In this article, we walk you through the initial steps to start and configure your cluster. We then setup and configure logstash to ship the logs to elasticsearch in order to monitor Redis performance. Redis performance logs shipped to elasticsearch can then be visualized and analyzed via Kibana dashboards.

A comprehensive log management and analysis strategy is vital, enabling organizations to understand the relationship between operational, security, and change management events and maintain a comprehensive understanding of their infrastructure. Log files from web servers, applications, and operating systems also provide valuable data, though in different formats, and in a random and distributed fashion.

No real-world web application can exist without a data storage backend, and most applications today use relational database management systems (RDBMS) for storing and managing data. The most commonly used database is MySQL, which is an open-source RDBMS that is the ‘M’ in the open-source enterprise LAMP Stack (Linux, Apache, MySQL and PHP).

Middle and large-sized applications send multiple database queries per second, and slow queries are often the cause of slow page loading and even crashes. The task of analyzing query performance is critical to determine the root cause of these bottlenecks, and most databases come with built-in profiling tools to help us.

Provisioning an Elasticsearch cluster in Qbox is easy. In this article, we walk you through the initial steps and show you how simple it is to start and configure your cluster. We then install and configure logstash to ship our MySQL or MariaDB/Galera logs to Elasticsearch. MySQL logs shipped to elasticsearch can then be visualized and analyzed via Kibana dashboards.

A comprehensive log management and analysis strategy is mission critical, enabling organisations to understand the relationship between operational, security, and change management events and maintain a comprehensive understanding of their infrastructure. Log files from web servers, applications, and operating systems also provide valuable data, though in different formats, and in a random and distributed fashion.

Why is Apache Web Server so popular? It’s free and open source, and open source is becoming vastly more popular than proprietary software. It’s maintained by dedicated developers, it provides security, is well suited for small and large websites alike, can be easily set up on all major operating systems, as well as being extremely powerful and flexible. Does that sound about right?

Provisioning an Elasticsearch cluster in Qbox is easy. In this article, we walk you through the initial steps and show you how simple it is to start and configure your cluster. We then install and configure logstash to ship our apache logs to elasticsearch. Apache logs shipped to elasticsearch can then be visualized and analyzed via Kibana dashboards.

A comprehensive log management and analysis strategy is mission critical, enabling organizations to understand the relationship between operational, security, and change management events and to maintain a comprehensive understanding of their infrastructure. Log files from web servers, applications, and operating systems also provide valuable data, although in different formats, and in a random and distributed fashion.

As with any web server, the task of logging NGINX is somewhat of a challenge. NGINX access and error logs can produce thousands of log lines every second, and this data, if monitored properly, can provide you with valuable information not only on what has already transpired but also on what is about to happen. But how do you extract actionable insights from this information? How do you effectively monitor such a large amount of data?

NGINX access logs contain a wealth of information including client requests and currently active client connections that, if monitored efficiently, can provide a clear picture of how the web server and the application that it serves is behaving. This tutorial describes how Qbox can be used to overcome this challenge by monitoring NGINX access logs with Qbox provisioned Elasticsearch Stack.