Enterprise Visibility with Splunk. The Building Blocks for Incident Detection.

Mark Ernest
6 min readSep 28, 2020

Visibility into unique organizational machine data is a core capability for cybersecurity teams and provides the foundation to building incident detection capabilities. The collection of a unique data source into a centralized platform, often referred to as log aggregation, allows cybersecurity teams to build detection signatures and facilitates security information and event management (SIEM) capabilities.

The diagram below illustrates this process.

Enterprise Visibility Diagram

A cybersecurity team needs to:

  • Identify data sources critical to visibility objectives.
  • Configure the systems to send data sources to a centralized log aggregation platform.
  • Use predefined correlation rules or create unique detection signatures to identify malicious activity.
  • Consume alerts into incident response workflows.

Another way to evaluate the need for visibility following this model is to identify how your organization would be the requirements in the NIST Cybersecurity Framework. Specifically, the DETECT (DE) function includes the following requirements:

  • Anomalies and Events (DE.AE): Anomalous activity is detected and the potential impact of events is understood.
  • Security Continuous Monitoring (DE.CM): The information system and assets are monitored to identify cybersecurity events and verify the effectiveness of protective measures.
  • Detection Processes (DE.DP): Detection processes and procedures are maintained and tested to ensure awareness of anomalous events.

Similar objectives would be identified in other regulatory requirements that may be more pertinent to an organization’s business sector, however, a cybersecurity team can evaluate the NIST Cybersecurity Framework requirements and associated controls as a baseline to understand how to meet the objectives.

While there are multiple vendors that can support the visibility objectives and requirements outlined above, Splunk offers a free trail of its software that can be used to evaluate the value of the capability.

Splunk Hands-On Lab.

With any testing, having a dedicated lab environment that can be quickly created, easily managed, and destroyed once finished is critical to completing the learning objectives.

If you do not have a lab environment, DigitalOcean provides an environment that meets these objectives: Disclaimer! this is an affiliate link: https://m.do.co/c/5c28b1b39512

If you have a lab environment skip ahead to Installing Splunk Enterprise.

Create the Splunk Project

Create a new project called splunk-lab as a Class project / Education purposes.

DigitalOcean create new Splunk project

When the Splunk project is created, click on the Get Started with a Droplet.

Add droplet

Choose an Ubuntu 20.04 LTS x64 image.

Ubuntu 20.04 image

In a lab scenario, Splunk can be memory intensive based on overall usage. While the minimum plan for most virtual private server providers will be likely be sufficient, in this guide, a 2 GB / 1 CPU system was provisioned.

Basic plan selection.

Select the datacenter region most applicable to your geo graphic location.

Datacenter region

For the lab, select Password for remote access and enter a password. In a product deployment, additional remote access security controls would likely be implemented.

Remote access setup

Enter splunk-lab as the hostname and create the instance.

Once finalized, the IP address of the instance will be shown.

IP address

On macOS open the terminal app and ssh as root to the IP address. On Windows, utilities like putty can be leveraged to connect to the IP address.

ssh root@$IP_ADDRESS

Installing Splunk Enterprise

With any application, it’s important to review the documentation. Splunk’s docs can be found here:

Create an Account and Download Splunk

Splunk is a commercially licensed product and would need to be purchased if deployed in an organization. Splunk does offer free trials of its products. To evaluate Splunk, create an account and click the Free Splunk button.

Free Splunk

For the lab, select Splunk Enterprise and download a free 60-day trial.

Splunk Enterprise Trial

Select the Linux package and click the .deb download option. At the time of this post, Splunk Enterprise was on version 8.0.6.

Linux .deb package

Review the license agreement and start the download.

Splunk Software License Agreement

Click the Download via Command Line (wget) and copy the entire command.

wget Splunk Enterprise

Install Splunk Enterprise

With the wget command copied, open the splunk-lab terminal session, paste the command, and press return to download the install package.

To verify the install packaged downloaded successfully click the Download MD5 to verify your bits from your Splunk account and then compare the hash by running the md5sum command against the downloaded install package.

md5sum splunk-8.0.6-linux-2.6-amd64.deb

Run the dpkg command with the “-i” flag to install the Splunk Enterprise package.

dpkg -i splunk-8.0.6–linux-2.6-amd64.deb

By default, Splunk Enterprise installs into /opt/splunk. The Splunk application resides in the bin directory.

cd /opt/splunk/bin/

To start Splunk Enterprise issue a splunk start command.

./splunk start

Review and agree with the license.

Splunk Enterprise license

During the initial install process, a username and password are required. These credentials will be required to access the administration panel.

Splunk Enterprise Credentials

Once the install process is completed, Splunk Enterprise will be accessible via HTTP on port 8000. This lab does not cover setting up a unique domain or SSL certificate with Splunk and can be found in the docs. These are steps that should be taken in a production deployment.

Splunk Enterprise Access

Open a web browser and type in the IP address of the splunk-lab instance via port 8000. Enter the credentials from the previous steps.

Splunk Login

The Splunk Enterprise environment is now accessible.

Splunk Enterprise

Unique data sources can now be setup to forward machine data to Splunk Enterprise and detection signatures can be created to provide cybersecurity teams with insights to malicious activity.

Closing Remarks.

Cybersecurity leaders and analysts should be familiar with log aggregation and SIEM capabilities and the value of having enterprise visibility. In subsequent posts, onboarding of unique data sources, detection signature creation strategies, and capability maturity will be covered.

If you made it this far, thanks for reading! Any feedback is always appreciated.

--

--

Mark Ernest

Dad, husband, cybersecurity researcher & practitioner, developer.