What are metadata in Splunk?

Meta what?? This does really sound more complicated than it really is. meta is in face ‘data about data’.

You remember from our article about getting data into Splunk, that I told you you should not any time trying to understand sourcetype, source, sourcetype and index. Now let’s take a look at these for a moment.

Sourcetype

The type of source you are getting in. Let’s that you got 4 different types of logs from 4 different type of files. You might opt to choose for different sourcetypes. A different time stamp notation, might be a good reason too. Performance would be another benefit when you look at field extractions (we will talk about field extractions later). If you need to try extracting fields with 10 regexes or you only use one, the impact on performance will be considerable.

Source

Source is as the name implies the source where the data came from. This can be a file name, a protocol and portnumber, …

Host

The host metadata field by default contains the value of the forwarder the data was received from. There are some other options to change this behavior as well. We can set a fixed value for example OR even take a specific segment of the source path. This last one can be useful if you gather the data of different hosts on a shared location.

Index

An index is an entity in which we store data (events) For every data stream (already chopped up in events, or just a big chunk of data) an index metadata needs to be specified. An index is also the only way to limit access to data, by granting or denying a user access to an index, that person will or will not be able to see data pop up in search results, dashboard or reports.