Linux System Content Pack

The following content pack is available for use with a Graylog Illuminate license and Graylog Enterprise or Graylog Security. Contact sales to learn more about obtaining Illuminate.

Linux is a ubiquitous, open-source operating system that powers everything from servers and cloud infrastructure to desktop systems and embedded devices.

This technology pack provides common log parsing, normalization, and enrichment for Debian/Ubuntu Linux distributions.

Supported Distributions

• Ubuntu 20.04 LTS (Focal Fossa) and 22.04 LTS (Jammy Jellyfish)

• Debian 11 (bullseye) and 12 (bookworm)

Requirements

• Graylog 6.0.1+ with a valid Enterprise license

Stream Configuration

This technology pack includes 1 stream:

• Illuminate:Linux System Messages

Hint: If this stream does not exist prior to the activation of this pack then it is created and configured to route messages to this stream and the associated index set. There should not be any stream rules configured for this stream.

Index Set Configuration

This technology pack includes 1 index set definition:

• Linux System Logs

Hint: If this index set is already defined, then nothing is changed. If this index set does not exist, then it is created with retention settings of a daily rotation and 90 days of retention. These settings can be adjusted as required after installation.

Log Collection

This technology pack primarily parses logs written to syslog and auth.log under the /var/log directory.

The following log delivery methods are recommended:

• Syslog daemons such as rsyslog and syslog-ng

• Filebeat (with Graylog Sidecar)

Rsyslog Configuration

1. Create a matching Syslog input in Graylog.

2. Configure rsyslog to send logs under /var/log to Graylog.

3. You can specifically forward messages from syslog and auth.log as the pack primarily targets these log sources.

4. Create a custom template that identifies the logs as Linux logs for the parser to recognize.

5. The simplest method is to set the field event_source_product as linux in the structured data portion of the syslog message. See the following example template configuration:

   Copy

   template(
                       name="custom_linux_syslog_5424"
                       type="string"
                       string="<%PRI%>1 %TIMESTAMP:::date-rfc3339% %HOSTNAME% %APP-NAME% %PROCID% %MSGID%
                       [exampleSDID@12345 event_source_product=\"linux\"]%msg:::sp-if-no-1st-sp%%msg:::drop-last-lf%\n"
                       )
   
                       *.* action(type="omfwd" target="YOUR GRAYLOG SERVER" port="514" protocol="tcp"
                       template="custom_linux_syslog_5424")

Hint: It is recommended to send RFC 5424-compliant syslog messages.

Syslog-ng Configuration

1. Create a matching Syslog input in Graylog.

2. The field event_source_product must be added to each message by the syslog-ng configuration for Illuminate to identify and process the logs as Linux logs.

   1. Modify the configuration to set the custom field event_source_product as linux using a rewrite function. See the following example under the Destinations section of the configuration:

   Copy

   #########################
                       # Destinations
                       #########################
   
                       # Set custom field using rewrite.
                       rewrite custom_field {
                       set("linux" value(".SDATA.meta.event_source_product"));
                       };
   
                       # Define TCP syslog destination.
                       destination d_graylog {
                       syslog("YOUR GRAYLOG SERVER" port(514));
                       };

3. Under Log paths define the source, rewrite, and destination. This example refers to the configurations set in the previous step:

   Copy

   #########################
                       # Log paths
                       #########################
   
                       # Send from the default source s_src to the custom destination d_graylog
                       # using the intermediary rewrite as configured above.
                       log {
                       source(s_src);
                       rewrite(custom_field);
                       destination(d_graylog);
                       };

Filebeat Configuration

1. Use the official Graylog Sidecar documentation to set up Graylog Sidecar for Filebeat.

2. Create a matching Beats input in Graylog.

3. Ensure that the option Do not add Beats type as prefix is disabled.

4. Create an API access token and custom Linux Filebeat collector.

5. Configure the collector to ship messages in syslog and auth.log to Graylog. The Filebeat input must add the field event_source_product: linux for the parser to identify the log source as Linux.

6. In addition, the option fields_under_root must be set to true for message identification to work. See the following example:

   Copy

   filebeat.inputs:
                       - input_type: log
                       paths:
                       - /var/log/auth.log
                       - /var/log/syslog
                       type: filestream
                       fields_under_root: true
                       fields:
                       event_source_product: linux

7. Install Graylog Sidecar on the client host.

8. Edit the Graylog Sidecar client configuration with your Graylog server API URL and API access token.

Hint: Note that this pack has partial support for messages delivered using GELF. Parsing works for most event types, but some such as variants of IAM events do not get processed as intended when in GELF.

What is Provided

• Rules to parse, normalize, and enrich Linux system log messages.

• A dashboard displaying events and statistics of interest.

Events Processed by This Technology Pack

The Linux System Logs content pack supports the following log types. Generic processing is provided for log types not listed.

• SSH Logs

• Login Logs

• Sudo Logs

• Su Logs

• PAM Logs

• Systemd Logs

• Cron Logs

• UFW Logs

• Iptables Logs

• User Activity Logs

• Group Activity Logs

GIM Categorization

GIM categorization is provided for the following messages:

Message Fields Included in This Pack

General Parsing

SSH

Login

Sudo

Su

PAM

Systemd

Cron

User and Group Activity

UFW

Iptables

Hint: User names are case-sensitive in some Linux distributions. While case is retained in the original message and in extracted fields like user_name and source_user_name, searches treat the values as case-insensitive. For example, a search for user_name:user1 would return results containing both cases such as user1 and USER1.

Linux System Logs Spotlight Content Pack

This spotlight offers a dashboard with 4 tabs:

Overview

SSH

Sudo

User and Group Activity