The following article exclusively pertains to a Graylog Enterprise feature or functionality. To learn more about obtaining an Enterprise license, please contact the Graylog Sales team.
The Graylog correlation engine is the central component of Graylog that allows you to analyze complex sequences of events to identify meaningful incidents.
Objectives
There are a variety of different approaches you can take when it comes to event correlation and analysis. The Graylog correlation engine was built with a few central methods in mind:
- Per Field Correlation
- This allows you to create rules that are grouped by a specific field, paving the way for unique field alerting (i.e. by user name).
- Cross Event Source Correlation
- This is a monitor for a pattern of logs across multiple sources so that, for example, you can see a login happening immediately after the IDS log from the same IP address.
- Look for the Negative Event
- This is the approach to correlation analysis that allows you to monitor for logs that are not being input into Graylog. An example of this would be when a web server that has shut down and should be starting back up is not doing so.
- Complex Correlation
- This means that you can build correlation alerts on top of another alert, leading to more complex and nuanced event alerts for your environment. So, if you detect a brute force attack, for example, you can also receive an alert if someone creates user credentials immediately after.
Create an Event Definition
The first step to creating event correlation rules is to create a new event definition for this correlation. Remember that event correlation is all about building a series of specific event definitions in some kind of designated sequence that, when all conditions are met, an alert is triggered. So, if you want to create a correlation that searches for criteria that may indicate a brute force attack is occurring, for example, you will want to create a brute force attack event definition.
Designate Key Fields
Building event correlations is specifically centered around key fields within event
definitions. Designating a specific field as a key field is simply a way to arrange events into groups based on this
key field. Graylog will generate as many events as unique keys are found. So, if the user_name
field is designated as key, for instance, then an event will be generated for each username that triggers an alert.
For this example, we’re going to build our event correlation around one field, which is the
user_name field.
- Navigate to Alerts> Event Definitions and select the specific correlation event.
- Click Edit at the end of the index set row.
- Click on Fields to display the event fields that have been previously used to create the event.
- Select Edit from the Actions menu next to the field name.
- In this menu, you will want to ensure that the field around which your correlation will function is designated as key. Make sure that Use Field as Event Key is selected as shown.
Build Event Correlation
Now that you have selected your desired field(s) as key, we need to build the actual correlation sequence that designates the event.
- Navigate to the event definition you have created for this correlation event and click Edit.
- Navigate to Filter and Aggregation in the wizard.
- Select Event Correlation as the condition type.
This change will trigger the corresponding correlation menu allowing you to designate the rules that constitute your event.
This menu is how you will set up your correlation rules to trigger the alert.
Event Correlation Rules
- “The following sequence of events must be satisfied within”
- Allows you to determine the length of time within which all correlated events must be completed to satisfy the requirements of the event correlation.
- “The correlation should execute”
- Determines how often the correlation query should run on your instance.
Select Events
In these menus, you will select the individual event definitions that will collectively determine the event correlation. If we take our previous example of a brute force attack event, then you would likely select failed login attempts and successful login attempts as your specific events requiring some kind of correlation logic; recall that these are separate event definitions that would need to be created beforehand following the instructions for Create an Event Definition.
Event # 1
- Event from
- Designates the event definition that has been created to determine what constitutes your desired event.
- Should occur at least
- Defines the minimum number of times that this event should occur within the correlation event period.
Event # 2
Along with the previous fields, the second and all subsequent events will require you to determine whether or not this is an event that should:
- Occur at least
- Meaning that the occurrence of this event happening a prescribed amount of times will trigger the event correlation.
- Not occur, and it is followed by another event.
- Meaning that this is an event that is typically expected but the lack of its occurrence could indicate some event or issue.
- Not occur in the next
- Meaning that this is an event that is typically expected but the lack of its occurrence could indicate some event or issue if it does not happen in a prescribed amount of time.
You can add multiple events to the sequence by selecting Add Event.
Preview the Correlation
The flowchart for the event correlation will be generated to the right of the selection menu, allowing you to view the actual sequence of events that need to occur in order for the overall correlation event to be determined to occur.
Set Up an Alert
You will likely want to set up some method of alerting to any occurrences of your event correlation. For detailed instructions on creating various types of event alerts, see the Alerts article.
Use Case Scenario
The Graylog correlation engine can be configured to allow for a variety of different correlation events in a variety of different parameters and sequences. For a more detailed use case approach, we recommend the Correlation Engine demo available on the Graylog website.
