As detailed in Alerts, alerts are messages that you can configure to inform you about an event. During the process of creating an event definition, you can attach an alert to the event, and you can choose how these alerts are sent (email, Slack or Discord alert, etc.).

Graylog supports alerts through the following methods:

• PagerDuty (Enterprise Only)

• Slack

• Microsoft Teams

• Script (Enterprise Only)

• Email

• HTTP

• Custom HTTP

• Legacy Alarm Callbacks (depreciated)

PagerDuty Alerts (Enterprise Only)

The following section exclusively pertains to a Graylog Enterprise feature. To learn more about obtaining an Enterprise license, please contact the Graylog Sales team.

The PagerDuty alert allows you to create new incidents in PagerDuty in response to events in your Graylog server. These are the supported configuration options to be defined in the New Notification menu for PagerDuty:

• Routing Key
  Your PagerDuty integration routing key.

• Use a Custom Incident Key
  If enabled, an incident key will be generated using the provided Incident Key Prefix. This will prevent PagerDuty from creating multiple incidents for a single event. If not selected, an incident key will NOT be generated, and each event notification will create a new incident in PagerDuty.

• Incident Key Prefix
  If a custom incident key is enabled, this will be used as a prefix for the incident key.

• Client Name
  The name of the Graylog system that triggered the PagerDuty incident.

• Client URL
  The URL for your Graylog web interface. If provided, this will be used to construct links that will be embedded in your PagerDuty incident.

Slack Alerts

Slack allows you to send alerts to your Slack workspace in response to events in your Graylog server. These are the supported configuration options to be defined in the New Notification menu for Slack:

• Configuration Color
  Highlight the custom message with a color you prefer.

• Webhook URL
  The unique URL used to send messages to your Slack instance.

• Channel
  The channel to which you will send a message or you can select @user or @team to receive the alert.

• Custom Message
  The message that will be sent to Slack. The data described above can be used in this template.

• Time Zone for Date/Time Values

  Select your preferred time zone used for timestamps in the alert. This selection will default to UTC.

• Message Backlog Limit (optional)
  Limit the number of backlog messages sent as part of the Slack alert. If set to 0, no limit will be enforced.

• User Name (optional)
  User name of the sender in Slack.

• Selections (you may choose to enable the following options):

  • Include Title: If enabled, this will include the full event definition title and description in the alert.

  • Notify Channel: If enabled, this will notify all the users in a channel with @channel.

  • Notify Here: If enabled, this will notify only active users in the channel with @here.

  • Link Names: If enabled, this will find and link channel names and user names in the alert.

• Icon URL(optional)
  Image to use as the icon for this message.

• Icon Emoji(optional) 
  Emoji to use as the icon for this message (this overrides any icon URL).

Microsoft Teams V2 Alerts

Released in Graylog 6.0.5, the new Microsoft Teams notification allows you to send messages to a Teams channel when specific events occur in your Graylog setup. This is done via a workflow that is created in Microsoft Teams. The Microsoft Teams Notification V2 supports sending an adaptive card to the Post to a channel when a webhook request is received workflow.

Create a Workflow in Microsoft Teams

Follow the steps below to create a workflow in Microsoft Teams:

1. Navigate to Apps > Workflows > Manage Workflows > Create.

2. Search for Post to a channel when a webhook request is received under Notifications. Select the template with the most uses in Workflows.

3. Give your workflow a unique title and click Add Workflow.

4. Save the webhook URL which is displayed in a pop up window. This is required for configuring a Graylog notification later on.

In order to edit the adaptive card:

1. Click Manage your workflow in the next pop up window.

2. Click Edit on the following page.

3. Expand the Send each adaptive card action.

4. Replace body.attachments with attachments, which can be found in the dynamic content options.

5. Click the Post card in a chat or channel.

6. Select Post as User.

7. Make your selections for Post In, Team and Channel.

8. Save the workflow.

Once you have the webhook URL, navigate to Graylog to create a Microsoft Teams Notification V2.

Create a Microsoft Teams Notification in Graylog

1. Navigate to Alerts > Notifications and click Create notification.

2. Give the notification a unique title and description.

3. Select Microsoft Teams Notification V2 from the Notification Type drop-down menu.

4. Enter the URL that you saved previously into the Webhook URL box.

5. The default Adaptive Card Template is similar to your original adaptive card. Use the Microsoft Adaptive Card Designer if you want to customize it.

6. Choose a relevant time zone under Time zone for date/time values. This selection will default to UTC.

7. Select a backlog limit. This is optional.

8. Click Execute Test Notification to validate that your notification works.

9. Click Create Notification.

Script Alerts (Enterprise Only)

The following section exclusively pertains to a Graylog Enterprise feature. To learn more about obtaining an Enterprise license, please contact the Graylog Sales team.

The script alert option lets you configure a script that will be executed when the alert is triggered.

These are the supported configuration options to be defined in the New Notification menu for script alerts:

• Script Path
  The path to where the script is located. Must be within the permitted script path (which is customizable).

• Script Timeout
  The maximum time (in milliseconds) the script will be allowed to execute before being forcefully terminated.

• Script Arguments
  Space-delimited string of parameters. Any of the data described above can be used.

• Send Alert Data Through STDIN
  Sends the JSON encoded data described above through standard in. You can use a JSON parser in your script.

Script Alert success is determined by its exit value; success equals zero. Any non-zero exit value will cause it to fail. Returning any error text through STDERR will also cause the alarm callback to fail.

Here is a sample Python script that shows all supported Script Alert functionality (argument parsing, STDIN JSON parsing, STDOUT, exit values, and returning an exit value).

#!/usr/bin/env python3
import json
import sys

# Function that prints text to standard error
def print_stderr(*args, **kwargs):
    print(*args, file=sys.stderr, **kwargs)

# Main function
if __name__ == "__main__":

    # Print out all input arguments.
    sys.stdout.write("All Arguments Passed In: " + ' '.join(sys.argv[1:]) + "\n")

    # Turn stdin.readlines() array into a string
    std_in_string = ''.join(sys.stdin.readlines())

    # Load JSON
    event_data = json.loads(std_in_string)

    # Extract some values from the JSON.
    sys.stdout.write("Values from JSON: \n")
    sys.stdout.write("Event Definition ID: " + event_data["event_definition_id"] + "\n")
    sys.stdout.write("Event Definition Title: " + event_data["event_definition_title"] + "\n")
    sys.stdout.write("Event Timestamp: " + event_data["event"]["timestamp"] + "\n")

    # Extract Message Backlog field from JSON.
    sys.stdout.write("\nBacklog:\n")
    for message in event_data["backlog"]:
        for field in message.keys():
            sys.stdout.write("Field: " + field + "\t")
            sys.stdout.write("Value: " + str(message[field]) + "\n")

    # Write to stderr if desired
    # print_stderr("Test return through standard error")

    # Return an exit value. Zero is success, non-zero indicates failure.
    exit(0) 

Email Alerts

Email alerts can be used to send an email to the configured alert receivers when conditions are triggered. Make sure to check the email-related configuration settings in the Graylog configuration file.

Several configuration options are available for the alert to customize the email that will be sent. Note that the email body and email subject are JMTE templates. JMTE is a minimal template engine that supports variables, loops and conditions. See the JMTE documentation for a language reference.
The default body template shows some advanced examples of accessing the available information in the template:

--- [Event Definition] ---------------------------
Title:       ${event_definition_title}Description: ${event_definition_description}
Type:        ${event_definition_type}
--- [Event] --------------------------------------
Timestamp:            ${event.timestamp}
Message:              ${event.message}
Source:               ${event.source}
Key:                  ${event.key}
Priority:             ${event.priority}
Alert:                ${event.alert}
Timestamp Processing: ${event.timestamp}
Timerange Start:      ${event.timerange_start}
Timerange End:        ${event.timerange_end}
Fields:
${foreach event.fields field}${field.key}: ${field.value}
${end}
${if backlog}
--- [Backlog] ------------------------------------
Last messages accounting for this alert:
${foreach backlog message}
${message}
${end}
${end}

These are the supported configuration options to be defined in the New Notification menu for email alerts:

• Subject

  The subject used for the email alert. As noted above, the email subject is a JMTE template.

• Reply-To

  The email address recipients should reply to if necessary.

• Sender

  The email address that will be used as the sending address. If the field is left empty, the default address will be used.

• User Recipient(s) (optional)

  Select all Graylog users that will receive the alert.

• Email Recipient(s) (optional)

  Add any additional email addresses to receive the alert.

• Time Zone for Date/Time Values (optional)