In this article, we’ll break down the ELK stack’s real total cost of ownership. Keep reading as the TCO analysis will provide insight into a platform’s hidden costs.
Let’s begin!
What Is the ELK Stack?
Before we analyze the TCO, let’s understand what is the ELK stack?
Stack refers to the technological layers of applications in a program. Meaning, the E, L, and K represent different applications that stack together to create the platform. ELK is an acronym for the three open-source applications: Elasticsearch, Logstash, and Kibana.
Elasticsearch is a RESTful search and analytic engine built on Apache Lucene, a free Java open-source search engine. It supports many languages, and supplies log analytics and full-text search. It uses APIs to search and retrieve documents that it automatically stores and adds searchable references to.
Elasticsearch also offers security intelligence, business analytics, and operational intelligence.
Logstash acts as the ingestion tool for the ELK Stack. It’s a server-side data processing unit that can collect data from a variety of sources. With over 200 open-source plug-ins to index data, it can transform and store data wherever needed.
Kibana aids in the visualization and exploration of data. It reviews the logs and events, creating interactive charts. It delivers operational intelligence and application monitoring.
Simplified: Elasticsearch analyzes the data, Logstash stores it, and Kibana monitors it. Together these three tools create a trifecta of intelligent programming that helps businesses attain actionable decision-making and yield perceptive analysis.
Why Is It Popular?
The ELK Stack is a popular choice amongst businesses for numerous reasons.
The biggest being it’s free, open-source software that effectively analyzes, logs, and monitors data. It’s also easily accessible on most of the cloud service providers.
With the ELK Stack, there’s the option to set-up the software on-premise, where you can hold data in-house or store data in the cloud where you rent space.
Open-Source
Open-source means the software is publically accessible. Anyone can edit the code as they see fit, making it a popular choice among organizations to edit the software to match their current infrastructure.
However, accessing the software is only a small fraction of the work you’ll be doing with the platform when it comes to business intelligence tools. The software requires constant upkeep and updating to keep it aligned with your business goals.
TCO Analysis for ELK Stack
Now it’s time to dive into the TCO of Elk Stack. Before we do, here’s the layout of the scenario we’ll use as our evaluation basis.
Our calculations are based on a 3-year scenario of moderate business growth. We’ve broken the analysis down into three sections, infrastructure acquisition, infrastructure maintenance, and future operations.
Infrastructure Acquisition
First, an organization will need to figure out how much infrastructure they need running the ELK Stack and how much machine data they’ll use. It depends on how much data the organization expects or intends to generate.
If they undershoot the amount they need, they lose visibility, inconvenience users, and violate SLAs.
If they overshoot, it’s a waste of money and leaves features underused.
To estimate how much infrastructure you’ll need, think of things like “how many gigabytes will be running a day” or “how long will it be stored or duplicated.” Consider how many users will interact with the data. All of these factors impact infrastructure costs.
For the first year, we’ve estimated a company runs about 150 GB a day. At that rate, the company will pay around $222,000 for the server and $47,000 for storage for the year. With the yearly data and business growth, by year 3, the company can expect to run 640GB a day. That increases the server cost to around $476,000 a year and storage to nearly $200,000 a year.
In three years, the company will have spent a little over a million dollars in storage and sever alone.
Infrastructure Maintenance
Now that the initial server and storage cost is figured out, the company has to calculate how much it’ll cost to maintain the platform. A routine analysis must be completed to ensure the platform operates at optimal levels. Because it’s a stack of applications, each separate application has a chance of individual issues and complications arising.
These issues, if untreated, can affect the communication between applications and security.
It’s important to consider how often will correction, detection, and cause analysis to be run. What recovery plans will be in place, and who will implement them. And when will data be encrypted, and who will validate this security.
Following the same scenario above, the company will pay on average $80,000 a year in security, data replication, and backup costs. Equalling out to $240,000 after three years.
Further Operations
With the software installation and maintenance complete, the organization will have a commercial-grade machine data analyst tool. However, it’s now on the company to sustain this commercial-grade tool.
The construction and upkeep of this level of software need engineering talent to enhance core capabilities. This is an administrative and operational task that must take place. When discussing cost, the company will need to examine the price of professional services.
Alternatively, if their staying in-house, how many full-time staff members are needed to manage, update, and set up the stack. Think about training costs for the employees working with the tool.
Support is also another expenditure. AWS does not provide free support, and what is available comes in a tiered system. So you’ll only get the benefits you pay for.
With the same scenario in mind, the first year will round out at $445,000 for employees and ELK Stack support. By year three, the total will jump to $965,000.
Over the three years, the organization will have paid over 2 million dollars in support and operations alone.
These prices increase because as the tool becomes more complex, the staff’s training is more complicated. As their role becomes more advanced, the staff will need to be compensated appropriately. And as the machine data tool becomes more advanced, it’ll cost more to maintain and support.
The Results Are In…
Based on the provided scenario, approximately $3,800,000 will be spent in only three years of using the ELK Stack program.
This is a significant deviation from the free software we originally started with.
While these may not be the exact numbers for your company, you can expect to pay around this amount. For this price, a platform should have cutting edge technology and features, yet ELK Stack struggles in this aspect.
A few of the missing components include real-time functionality, security, and scalability.
Lack of Real-Time Capabilities
ELK Stack does not possess real-time data visibility or logging.
That means as the program processes the data, there’s no access to the analysis taking place. That creates delays in obtaining information on security or operational issues. There’s also a delay in the companies ability to access the logs and obtain the data.
Real-time analysis and functions are innovative processes that take place in most modern business intelligence tools. The lack of this function minimizes this program.
ELK also does not process cold data, only hot, which leads to inaccurate historical analysis.
Data Protection and Security
There’s a massive gap in the security of this program.
When the data is at rest, there is no encryption creating a massive exposure to data theft and other incidents. There’s also no security attestation, so there’s a chance of non-compliance.
Scalability
Any business that creates a platform with customers in mind offers scalability. It helps create a seamless integration process and optimal work environments.
While you decide how much infrastructure you need in the beginning, that amount changes continuously. ELK Stack does not support on-demand scaling, so the amount you choose is concrete. This can lead to a lot of wasted money on unused features, or worse, not enough infrastructure to support your growing business.
ELK Stack TCO: The Real Deal
It’s easy to see a recognizable name and affordable software and think it’s the perfect deal. But if the ELK Stack TCO has taught anything, it’s that to have a proper understanding of cost, it’s essential to view the whole picture.
All tools will cost money. It’s about making sure that money goes towards a tool that can maximize your business with advanced features and innovative technology.
We hope you found this article helpful. Here at Apica, we want to help professionals make informed decisions. If you have any questions, don’t hesitate to contact us. In the meantime, check out our blog for more industry insight!
