(I have no rights to use any official Splunk logos or images as stated in Splunk’s policy on use of their image library, hence I depend completely on my design skills in Canva 😉 )
As we briefly mentioned in our Big Data article Splunk is one of the many pieces of software that can help us deal with the many challenges we face since big data has come along.
In this article we will cover some of those challenges and how Splunk can help us solve these. 🙂 )
Schemaless or schema on the fly
I have seen both terms used in commercial and training documents but … what does this mean exactly? The problem most of the traditional data storage technologies have is that data is supposed to follow the exact same schema per data type. While in theory this looks great, in practice we see that there a lot of variations in one datatype. Splunk really does not care about this when the data is ingested (the indexing phase), it is really flexible when it comes to searching your data. If your data does not use indexed extractions, all fields are extracted at search time.
Search time?
Yes, Splunk Search Heads (the server role that does most of the search coordination) only do the field extractions when a search is launched. This has several advantages but the most important one is speed. When you launch a search in Splunk you will most likely be searching a limited set of data, meaning data from a specific time, day, week or month. This means that these extractions will only be done on that specific set and NOT on the whole data set. This, in and of itself, has another advantage, where classic technologies fail, that you do not need to store the field extractions on disk, which can save you a considerable amount of diskspace
Data Models
Splunk comes with a great set of tools for normalization. As soon as your data is normalized, which is basically a mapping of a technology specific field to a field that is known in a Splunk data model, you will be able to have searches prepopulate your datasets for even faster searching.
While the above is one big advantage of data models let’s not forget how easy it gets to correlate data across different devices on the network. There is no burden anymore of different log format across different devices. That hurdle has been taken out of the equation already by the normalisation
SIEM integration & IT service management
While Splunk already has many advantages they also have a couple of premium addons which even extend the functionalities. Splunk Enterprise Security gives you the possibility to centrally handle security incidents and Splunk ITSI gives you the possibility to monitor all your services at a glance.
In our next article we will actually install splunk on a server and show you what Splunk can do in a so called all in one installation.
Stay tuned!
