After defining the Events that are needed to trigger an Alert, it is possible to attach a Notification. By attaching a Notification to an Event or group of Events, we can determine how and when information will flow out of Graylog. Notifications can be created by selecting the Notifications button under the Alerts tab, or by defining them in the Event workflow.

In this section we explain what the default alert notifications included in Graylog do and how to configure them. Alert notifications are meant to be extensible through Plugins. You can find more types in the Graylog Marketplace or even create your own.

Data Available to Notifications

Graylog makes the following data available when it runs a notification.

Different notification types will expose the data differently, the details are listed with the description of the specific notifications below.

Event Definition Metadata Information about the event definition that created the alert.

• event_definition_id (String) - The database ID of the event definition.
• event_definition_type (String) - The internal name of the event definition type (aggregation-v1 or correlation-v1).
• event_definition_title (String) - The title set in the UI.
• event_definition_description (String) - The description set in the UI.
• job_definition_id (String) - The internal job definition ID associated with a scheduled event definition.
• job_trigger_id (String) - The internal ID associated with the current execution of the job.

Event Data

• event The event as it is stored in Graylog
  • id (String) - The message ID of the stored event.
  • event_definition_id (String) - Same as event_definition_id in the metadata section.
  • event_definition_type (String) - Same as event_definition_type in the metadata section.
  • origin_context (String) - URN of the message or event creating this event (either event or message). Can be empty.
  • timestamp (DateTime) - The timestamp this event is describing can be set to the underlying event or message (see origin_context).
  • timestamp_processing (DateTime) - The timestamp for when the event has been created by Graylog.
  • timerange_start (DateTime) - The start of the window of data Graylog used to create this event. Can be empty.
  • timerange_end (DateTime) - The end of the window of data Graylog used to create this event. Can be empty.
  • streams - (Strings) - The list of stream IDs the event is stored in.
  • source_streams (Strings) - The list of stream IDs the event pulled data from.
  • alert (bool) - Whether this event is considered to be an alert. Always true for event definitions that have notifications.
  • message (String) - A human-friendly message describing this event.
  • source (String) - The host name of the Graylog server that created this event.
  • key_tuple (Strings) - The list of values making up the event’s key.
  • key (String) - The event’s key as a single string.
  • priority (long) - The event’s priority value.
  • fields (Map<String, String>) - The custom fields attached to the event.

Backlog

• backlog (List of Message summaries) - The list of messages or events which lead to this alert being generated
  • id (String) - The message ID.
  • index (String) - The name of the index the message is stored in. Use together with id to uniquely identify a message in Graylog.
  • source (String) - The source field of the message.
  • message (String) - The message field of the message.
  • timestamp (DateTime) - The timestamp field of the message.
  • stream_ids (Strings) - The stream IDs of the message.
  • fields (Map<String, Object>) - The remaining fields of the message, can be iterated over.

Email Alert Notification

The email alert notification can be used to send an email to the configured alert receivers when the conditions are triggered.

Make sure to check the email-related configuration settings in the Graylog configuration file.

Three configuration options are available for the alert notification to customize the email that will be sent. The email body and email subject are JMTE templates. JMTE is a minimal template engine that supports variables, loops and conditions. See the JMTE documentation for a language reference.

All of the data described above is available in the JMTE templates.
The default body template shows some advanced examples of accessing the information listed above:

--- [Event Definition] ---------------------------
Title:       ${event_definition_title}Description: ${event_definition_description}
Type:        ${event_definition_type}
--- [Event] --------------------------------------
Timestamp:            ${event.timestamp}
Message:              ${event.message}
Source:               ${event.source}
Key:                  ${event.key}
Priority:             ${event.priority}
Alert:                ${event.alert}
Timestamp Processing: ${event.timestamp}
Timerange Start:      ${event.timerange_start}
Timerange End:        ${event.timerange_end}
Fields:
${foreach event.fields field}${field.key}: ${field.value}
${end}
${if backlog}
--- [Backlog] ------------------------------------
Last messages accounting for this alert:
${foreach backlog message}
${message}
${end}
${end}

HTTP Alert Notification

The HTTP alert notification lets you configure an endpoint that will be called when the alert is triggered.

Graylog will send a POST request to the notification URL including information about the alert. The body of the request is the JSON encoded data described above.

Here is an example of the payload included in a notification:

{
  "event_definition_id": "this-is-a-test-notification",
  "event_definition_type": "test-dummy-v1",
  "event_definition_title": "Event Definition Test Title",
  "event_definition_description": "Event Definition Test Description",
  "job_definition_id": "<unknown>",
  "job_trigger_id": "<unknown>",
  "event": {
    "id": "NotificationTestId",
    "event_definition_type": "notification-test-v1",
    "event_definition_id": "EventDefinitionTestId",
    "origin_context": "urn:graylog:message:es:testIndex_42:b5e53442-12bb-4374-90ed-0deadbeefbaz",
    "timestamp": "2020-05-20T11:35:11.117Z",
    "timestamp_processing": "2020-05-20T11:35:11.117Z",
    "timerange_start": null,
    "timerange_end": null,
    "streams": [
      "000000000000000000000002"
    ],
    "source_streams": [],
    "message": "Notification test message triggered from user <admin>",
    "source": "000000000000000000000001",
    "key_tuple": [
      "testkey"
    ],
    "key": "testkey",
    "priority": 2,
    "alert": true,
    "fields": {
      "field1": "value1",
      "field2": "value2"
    }
  },
  "backlog": []
}

PagerDuty Alert Notification

The PagerDuty alert notification allows you to create new incidents in PagerDuty in response to Events in your Graylog server.

These are the supported configuration options:

Routing Key
Your PagerDuty integration routing key.

Use Custom Incident Key
If enabled, an incident key will be generated using the provided Incident Key Prefix. This will prevent PagerDuty from creating multiple incidents for a single Event. If not checked, no incident key will be generated and each event notification will create a new incident in PagerDuty.

Incident Key Prefix
If a Custom Incident Key is enabled, this will be used as a prefix for the incident key.

Client Name
The name of the Graylog system that triggered the PagerDuty incident.

Client URL
The URL for your Graylog web interface. If provided, this will be used to construct links which will be embedded in your PagerDuty incident.

Slack Alert Notification

The Slack alert notification allows you to send notifications to your slack workspace in response to events in your Graylog server.

Webhook URL
The unique URL used to send messages to your Slack instance.

Channel
A channel to send a message to.

Configuration Color
Highlight the custom message with this color.

Custom Message
The message that will be sent to Slack. The data described above can be used in this template.

Message Backlog Limit (optional)
Limit the number of backlog messages sent as part of the Slack notification. If set to 0, no limit will be enforced.

User Name (optional)
User name of the sender in Slack.

Icon URL
Image to use as the icon for this message.

Icon Emoji
Emoji to use as the icon for this message (overrides Icon URL).

Discord Notification

The Discord alert notification allows you to send Graylog alerts to Discord in response to events in your Graylog server. This is done via a webhook, which is the same method used for Slack notifications.

These are the supported configuration options:

Webhook URL
The unique URL used to send messages to your Discord instance. Add /slack to the end of the webhook to form the link that marks this as a Slack notification to Discord.

Channel
A channel to send message to.

Configuration Color
Highlight the custom message with this color.

Custom Message
The message that will be sent to Discord. The data described above can be used in this template.

Message Backlog Limit (optional)
Limit the number of backlog messages sent as part of the Slack notification. If set to 0, no limit will be enforced.

User Name (optional)
User name of the sender in Discord.

Icon URL
Image to use as the icon for this message.

Icon Emoji
Emoji to use as the icon for this message (overrides Icon URL).

Microsoft Teams Notification

The Microsoft Teams notification allows you to send messages to a Teams channel when specific events occur in your Graylog setup. This is done via a webhook, which is the same method used for Slack notifications.

If you're already using Microsoft Teams and would like to start receiving Graylog alerts in your team's channel, here are the steps to configure.

1. Start by creating an Incoming Webhook to share content in your Teams channels.

2. Then, in Graylog, navigate to the Alerts page and select Notifications > Create Notifications.
3. Fill out the required fields and select Microsoft Teams in the "Notification Type" field.

Here are the currently supported fields that can be configured:

Title
This is to identify the notification type being configured, e.g. File Permission Change.

Description
Describe what this notification is for or what it will do, like, "Notify Security Team’s Channel whenever there is a file permission change."

Notification Type
Select "Microsoft Teams Notification."

Configuration Color
Highlight the custom message with your preferred color.

Webhook URL
This is the unique URL generated whilst setting up your webhook. Copy the URL that was generated when creating your webhook and paste it into this field.

Custom Message
This is the message that will be sent to your MS Teams channel.

Message Backlog Limit (optional)
Limit the number of backlog messages sent as part of the MS Teams notification. If set to 0, no limit will be enforced.

Icon URL
This is an image to use as the icon for this message.

4. Execute a test notification by clicking on the "Execute Test Notification" button to ensure the configuration works as expected.

5. Click the update button to save your Microsoft Teams notification. You're all set! You should now start receiving notifications in your Microsoft Teams channel.
   

Script Alert Notification [Operations]

The Script Alert Notification lets you configure a script that will be executed when the alert is triggered.

These are the supported configuration options.

Script Path
The path to where the script is located. Must be within the permitted script path (which is customizable).

Script Timeout
The maximum time (in milliseconds) the script will be allowed to execute before being forcefully terminated.

Script Arguments
Space-delimited string of parameters. Any of the data described above can be used.

Send Alert Data Through STDIN
Sends the JSON encoded data described above through standard in. You can use a JSON parser in your script.

Script Alert Notification success is determined by its exit value; success equals zero. Any non-zero exit value will cause it to fail. Returning any error text through STDERR will also cause the alarm callback to fail.

Here is a sample Python script that shows all supported Script Alert Notification functionalities (argument parsing, STDIN JSON parsing, STDOUT, exit values, and returning an exit value).

#!/usr/bin/env python3
import json
import sys

# Function that prints text to standard error
def print_stderr(*args, **kwargs):
    print(*args, file=sys.stderr, **kwargs)

# Main function
if __name__ == "__main__":

    # Print out all input arguments.
    sys.stdout.write("All Arguments Passed In: " + ' '.join(sys.argv[1:]) + "\n")

    # Turn stdin.readlines() array into a string
    std_in_string = ''.join(sys.stdin.readlines())

    # Load JSON
    event_data = json.loads(std_in_string)

    # Extract some values from the JSON.
    sys.stdout.write("Values from JSON: \n")
    sys.stdout.write("Event Definition ID: " + event_data["event_definition_id"] + "\n")
    sys.stdout.write("Event Definition Title: " + event_data["event_definition_title"] + "\n")
    sys.stdout.write("Event Timestamp: " + event_data["event"]["timestamp"] + "\n")

    # Extract Message Backlog field from JSON.
    sys.stdout.write("\nBacklog:\n")
    for message in event_data["backlog"]:
        for field in message.keys():
            sys.stdout.write("Field: " + field + "\t")
            sys.stdout.write("Value: " + str(message[field]) + "\n")

    # Write to stderr if desired
    # print_stderr("Test return through standard error")

    # Return an exit value. Zero is success, non-zero indicates failure.
    exit(0) 

Legacy Script Alert Callback

The Legacy Script Alert Callback lets you configure a script that will be executed when the alert is triggered.

These are the supported configuration options.

Script Path
The path to where the script is located. Must be within the permitted script path (which is customizable).

Script Timeout
The maximum time (in milliseconds) the script will be allowed to execute before being forcefully terminated.

Script Arguments
String of parameters in which the delimiters are either space-delimited or a new-line. The following argument variables may be used:

Stream
The stream this alert belongs to.

• stream_id ID of the stream
• stream_name title of the stream
• stream_description stream description
• stream_url a string that contains the URL to the view the relevant messages for the alert. Make sure to set the HTTP URL configuration parameter, as there is no default.

Alert

The check result object for this stream.

• alert_description text that describes the check result
• alert_triggered_at date when this condition was triggered

Condition

The available conditions to request are

• condition_id ID of the condition
• condition_description description of the condition
• condition_title title of the condition
• condition_type type of condition
• condition_grace grace period for the condition
• condition_repeat_notification repeat notification of the script

Send Alert Data Through STDIN

Sends JSON alert data through standard in. You can use a JSON parser in your script.

Legacy Script Alert Callback success is determined by its exit value; success equals zero. Any non-zero exit value will cause it to fail. Returning any error text through STDERR will also cause the alarm callback to fail.

Event Summary

When all of the components have been defined the Event Summary will be displayed to the user. At this time, the user may select a previous point in the Workflow to change a parameter. The user may also cancel out of the workflow, select done. The Event may be viewed under Alerts > Event Definitions.