The following article exclusively pertains to a Graylog Enterprise feature or functionality. To learn more about obtaining an Enterprise license, please contact the Graylog Sales team.

The Graylog correlation engine is the central component of Graylog that allows you to analyze complex sequences of events to identify meaningful incidents.

Objectives

There are a variety of different approaches you can take when it comes to event correlation and analysis. The Graylog correlation engine was built with a few central methods in mind:

  • Per Field Correlation
    • This allows you to create rules that are grouped by a specific field, paving the way for unique field alerting (i.e. by user name).
  • Cross Event Source Correlation
    • This is a monitor for a pattern of logs across multiple sources so that, for example, you can see a login happening immediately after the IDS log from the same IP address.
  • Look for the Negative Event
    • This is the approach to correlation analysis that allows you to monitor for logs that are not being input into Graylog. An example of this would be when a web server that has shut down and should be starting back up is not doing so.
  • Complex Correlation
    • This means that you can build correlation alerts on top of another alert, leading to more complex and nuanced event alerts for your environment. So, if you detect a brute force attack, for example, you can also receive an alert if someone creates user credentials immediately after.

Procedure

Event Definitions

The first step to creating event correlation rules is to create a new event definition for this correlation. Remember that event correlation is all about building a series of specific event definitions in some kind of designated sequence that, when all conditions are met, an event notification is triggered. So, if you want to create a correlation that searches for criteria that may indicate a brute force attack is occuring, for example, you will want to create a brute force attack event definition.

For specific instructions on how to create an event definition, see the related documentation on Defining an Event.

Hint: Events, event definitions, and alerts are all covered more extensively in the Alerts page. This is highly recommended reading to be completed first as this article builds on these existing principles.

Key Fields

Building event correlations is specifically centered around key fields within event definitions. Designating a specific field as a key field is simply a way to arrange events into groups based on this key field, and Graylog will generate as many events as unique keys are found. So, if the user_name field is designated as key, for instance, then an event will be generated for each username that triggers an event notification.

For this example, we’re going to build our event correlation around one field, which is the user_name field.

  1. Navigate to the Event Definitions page from your Alerts dashboard and select the specific correlation event.

  1. Select “Edit” to edit the event.
  2. In the horizontal menu, select “Fields” to show the event fields that have been previously used to create the event. Then, select “Edit” from the “Actions” menu next to the field name.
  3. In this menu, you will want to ensure that the field around which your correlation will function is designated as key. Make sure that the box “Use Field as Event Key” is selected as shown.

Event Correlation

Now that you have selected your desired field(s) as key, we need to build the actual correlation sequence that designates the event.

  1. Navigate to the event definition you’ve created for this correlation event and select “Edit.”
  2. In the horizontal menu, select “Filter and Aggregation.”
  3. You will now change the condition type from an aggregation event to a correlation event. In the “Event Condition” widget, select “Event Correlation” as your condition type.
  4. This change will trigger the corresponding correlation menu allowing you to designate the rules that constitute your event.

This menu is how you will set up your correlation rules to trigger the event notification.

Event Correlation Rules

  • “The following sequence of events must be satisfied within”
    • Allows you to determine the length of time within which all correlated events must be completed to satisfy the requirements of the event correlation.
  • “The correlation should execute”
    • Determines how often the correlation query should run on your instance.

Events

In these menus, you will select the individual event definitions that will collectively determine the event correlation. If we take our previous example of a brute force attack event, then you would likely select failed login attempts and successful login attempts as your specific events requiring some kind of correlation logic; recall that these are separate event definitions that would need to be created beforehand following the instructions for Defining an Event.

Event # 1

  • “Event from”
    • Designates the event definition that has been created to determine what constitutes your desired event.
  • “Should occur at least”
    • Defines the minimum number of times that this event should occur within the correlation event period.

Event # 2

Along with the previous fields, the second and all subsequent events will require you to determine whether or not this is an event that should:

  • “Occur at least”
    • Meaning that the occurrence of this event happening a prescribed amount of times will trigger the event correlation.
  • “Not occur, and it is followed by another event.”
    • Meaning that this is an event that is typically expected but the lack of its occurrence could indicate some event or issue.
  • “Not occur in the next”
    • Meaning that this is an event that is typically expected but the lack of its occurrence could indicate some event or issue if it does not happen in a prescribed amount of time.

You can add multiple events to the sequence by selecting “Add Event.”

Correlation Preview

The flowchart for the event correlation will be generated to the right of the selection menu, allowing you to view the actual sequence of events that need to occur in order for the overall correlation event to be determined to occur.

Alerts

You will likely want to set up some method of notification alerting you to any occurrences of your event correlation. For detailed instructions on creating various types of event alerts, see the Notifications article.

Use Case Scenario

The Graylog correlation engine can be configured to allow for a variety of different correlation events in a variety of different parameters and sequences. For a more detailed use case approach, we recommend the Correlation Engine demo available on the Graylog website.