The following content pack is available for use with a Graylog Illuminate license and Graylog Enterprise or Graylog Security. Contact sales to learn more about obtaining Illuminate.

Linux is a ubiquitous, open-source operating system that powers everything from servers and cloud infrastructure to desktop systems and embedded devices.

This technology pack provides common log parsing, normalization, and enrichment for Debian/Ubuntu Linux distributions.

Supported Distributions

• Ubuntu 20.04 LTS (Focal Fossa) and 22.04 LTS (Jammy Jellyfish)

• Debian 11 (bullseye) and 12 (bookworm)

Hint: While the initial release of this pack was primarily developed and tested on Ubuntu, it also supports Debian as Ubuntu is based on Debian. Both distributions share similar log formats for common system outputs like syslog and auth.log. Minor differences in service naming or security frameworks may exist, but the core log structures are compatible across both platforms.

Requirements

• Graylog 6.0.1+ with a valid Enterprise license

Stream Configuration

This technology pack includes one stream:

• “Illuminate:Linux System Messages”

Hint: If this stream does not exist prior to the activation of this pack then it will be created and configured to route messages to this stream and the associated index set. There should not be any stream rules configured for this stream.

Index Set Configuration

This technology pack includes one index set definition:

• "Linux System Logs"

Hint: If this index set is already defined, then nothing will be changed. If this index set does not exist, then it will be created with retention settings of a daily rotation and 90 days of retention. These settings can be adjusted as required after installation.

Log Collection

This pack primarily parses logs written to syslog and auth.log under the /var/log directory.

The following log forwarders are recommended:

• Rsyslog

• Filebeat (with Graylog Sidecar)

Rsyslog Configuration

1. Create a matching syslog input in Graylog.

2. Configure rsyslog to send logs under /var/log to Graylog.

3. You can specifically forward messages from syslog and auth.log as the pack primarily targets these log sources.

4. Create a custom template that identifies the logs as Linux logs for the parser to recognize.

5. The simplest method is to set the field event_source_product as linux in the structured data portion of the syslog message. See the following example template configuration:

   Copy

   template(name="Custom_Linux_SyslogProtocol23Format" type="string" string="<%PRI%>1 %TIMESTAMP:::date-rfc3339% %HOSTNAME% %APP-NAME% %PROCID% %MSGID% [origin event_source_product=\"linux\"] %MSG%\n")
   
   *.* action(type="omfwd" target="YOUR GRAYLOG SERVER" port="514" protocol="udp" template="Custom_Linux_SyslogProtocol23Format")

Hint: Only RFC 5424-compliant syslog messages are accepted.

Filebeat Configuration

1. Please refer to the official documentation to set up Graylog Sidecar for Filebeat.

2. Create a matching Beats input in Graylog.

3. Ensure that the option Do not add Beats type as prefix is disabled.

4. Create an API access token and custom Linux Filebeat collector.

5. Configure the collector to ship messages in syslog and auth.log to Graylog. The Filebeat input must add the field event_source_product: linux for the parser to identify the log source as Linux.

6. In addition, the option fields_under_root must be set to true for message identification to work. See the following example:

   Copy

   - input_type: log
     paths:
       - /var/log/auth.log
       - /var/log/syslog
     type: filestream
     fields_under_root: true
     fields:
       event_source_product: linux

7. Install Graylog Sidecar on the client host.

8. Edit the Graylog Sidecar client configuration with your Graylog server API URL and API access token.

What is Provided

• Rules to parse, normalize, and enrich Linux system log messages

• A dashboard displaying events and statistics of interest

Events Processed by This Technology Pack

The Linux System Logs content pack supports the following log types. Generic processing will be provided for log types not listed.

• SSH Logs

• Sudo Logs

• Su Logs

• PAM Logs

• Systemd Logs

• Cron Logs

• User Activity Logs

• Group Activity Logs

GIM Categorization

GIM categorization is provided for the following messages:

Log Type	Vendor Event Description	GIM Category	GIM Subcategory	GIM Event Type Code
pam	Authentication failure	authentication	authentication.default	109999
ssh	Postponed keyboard-interactive	authentication	authentication.default	109999
ssh	Accepted password	authentication	authentication.credential validaion	100500
ssh	Accepted publickey	authentication	authentication.credential validaion	100500
ssh	Failed password (for invalid/illegal user)	authentication	authentication.credential validaion	100500
ssh	Failed publickey (for invalid/illegal user)	authentication	authentication.credential validaion	100500
systemd	Service status message	service	service.default	219999
systemd	Unit service entered <STATE> state	service	service.default	219999
systemd	Failed with result	service	service.default	219999
systemd	Main process exited	service	service.default	219999
systemd	Control process exited	service	service.default	219999
users/groups	Changed password expiry for user	iam	iam.default	119999
users/groups	Changed user information	iam	iam.default	119999
users/groups	Changed user shell	iam	iam.default	119999
users/groups	User added to group	iam	iam.default	119999
users/groups	User removed from group	iam	iam.default	119999
users/groups	User created	iam	iam.default	119999
users/groups	User deleted	iam	iam.default	119999
users/groups	User username changed	iam	iam.default	119999
users/groups	User password changed	iam	iam.default	119999
users/groups	User ID changed	iam	iam.default	119999
users/groups	Group removed	iam	iam.default	119999
users/groups	Group ID changed	iam	iam.default	119999
users/groups	Group password changed by user	iam	iam.default	119999
users/groups	Group password removed by user	iam	iam.default	119999

Hint: GIM categorization for IAM events currently lacks support for events referencing group entities.

Message Fields Included in This Pack

Field Name	Example Value	Field Type	Description
General Parsing
application_name	sshd	keyword	This field is generated by the syslog input or parsed from the syslog header in Filebeat messages
facility	security/authorization	keyword	This field is generated by the syslog input
process_id	5222	keyword/loweronly	This field is generated by the syslog input
source	prodlxv01	keyword	Hostname or IP of the source system that generated the event
SSH
event_outcome	success	keyword	The outcome of the event
event_repeat_count	3	long	Count of times a message has been repeated
host_ip	10.1.2.3	keyword	IP address that the SSH server binds to
host_port	22	long	Port that the SSH server listens on
source_ip	10.1.2.3	keyword	IP address of the client
source_hostname	workstation.corp.example.com	keyword	Hostname of the client
source_port	62681	long	Port from which the client connects
user_name	webadmin	keyword	The user used to authenticate or associated with the session
vendor_credential_type	publickey	keyword	The type of credentials used for authentication
vendor_event_action	refused	keyword	The action taken in the event
vendor_event_description	Accepted password	keyword	The description of the event action
vendor_event_outcome	not allowed	keyword	The vendor-defined outcome of the action described in the message
vendor_event_outcome_reason	not listed in AllowUsers	keyword	The vendor-provided text detailing the reason for the event outcome
vendor_ssh_signature	ED25519 SHA256:YbexxF5dVcxxmEh9xx/DuXoUX6xxLvKI2u3xx7LjYuw	keyword	The fingerprint of the key or certificate used for authentication
Sudo
event_outcome	success	keyword	The outcome of the event
process_command_line	/bin/bash	keyword	The full command line that the user attempted to execute
process_working_directory	/home/user1	keyword	The current working directory that the process was called from
source_user_name	user1	keyword	The user that initiated the request to run a command with elevated privileges
user_name	root	keyword	The user that the command is run as
vendor_event_description	User executed a command with elevated privileges	keyword	The description of the event action
vendor_event_outcome	success	keyword	The outcome describing whether the sudo attempt was successful or not
vendor_sudo_error	user NOT in sudoers	keyword	The error that occurred, often indicating a failed sudo attempt
vendor_tty	pts/1	keyword	The terminal from which the sudo command was executed
Su
event_outcome	success	keyword	The outcome of the event
source_user_name	user1	keyword	The user that initiated the action to switch to another user
user_name	user2	keyword	The target user that the original user attempted to switch to
vendor_event_description	Failed user attempt to switch to another user	keyword	The description of the event action
vendor_event_outcome	FAILED	keyword	The outcome describing whether the su attempt was successful or not
vendor_tty	pts/0	keyword	The terminal from which the su command was executed
PAM
event_outcome	failure	keyword	The outcome of the event
source_ip	10.1.2.3	keyword	Remote IP address from which the authentication attempt originated
source_hostname	workstation.corp.example.com	keyword	Remote hostname from which the authentication attempt originated
source_user_id	0	keyword	ID of the user or process that initiated the session for another user
source_user_name	user1	keyword	The remote user that made the authentication attempt or user that initiated the session
user_id	1000	keyword	ID of the user for which a session was opened/closed
user_name	user2	keyword	The user account being authenticated or the user for which a session was opened/closed
vendor_event_action	opened	keyword	The action taken in the event
vendor_event_description	Session opened for user	keyword	The description of the event action
vendor_pam_euid	0	keyword	The effective user ID used to execute the process, which may differ from the real user ID
vendor_pam_function	auth	keyword	The specific function being invoked by PAM
vendor_pam_logname	user1	keyword	The login name of the user
vendor_pam_module	pam_unix	keyword	The name of the PAM module handling the authentication
vendor_pam_service_name	sshd	keyword	The service or application using PAM for authentication
vendor_pam_uid	0	keyword	ID of the user being authenticated or of the process facilitating the authentication
vendor_tty	/dev/pts/0	keyword	The terminal over which the authentication attempt was made
Systemd
event_outcome	failure	keyword	The outcome of the event
service_name	logrotate	keyword	The name of the service
service_state	active	keyword	The state of the service
vendor_event_action	Finished	keyword	The action taken in the event
vendor_event_description	Main process exited	keyword	The description of the event action
vendor_event_outcome	Failed	keyword	The vendor-defined outcome of the action described in the message
vendor_systemd_code	exited	keyword	The exit code returned by the process when it terminates
vendor_systemd_service_status	Deactivated successfully.	keyword	The general status of the service
vendor_systemd_status_code	1	keyword	The status code that indicates the result of the process
vendor_systemd_status_description	FAILURE	keyword	The description of the status of the service after its process has terminated
vendor_systemd_unit_description	Rotate log files	keyword	The description of the systemd unit, often from when it changes state
Cron
process_command_line	cd / && run-parts --report /etc/cron.hourly	keyword	The full command line scheduled to run
source_user_name	root	keyword	The user that modified the crontab of itself or another user
user_name	user1	keyword	The user that the command is run under or whose crontab is modified
vendor_cron_error	grandchild #2176 failed with exit status 1	keyword	The error that occurred during a job or when processing crontab
vendor_cron_info	Running @reboot jobs	keyword	The general information output from a job
vendor_event_description	Cron job initiated	keyword	The description of the event action
User and Group Activity
source_user_id	1001	keyword	ID of the user account impacting another user account, or the ID before a change
source_user_name	user1	keyword	Name of the user account impacting another user account, or the name before a change
user_id	1002	keyword	ID of the user account
user_name	user2	keyword	Name of the user account
vendor_event_action	created	keyword	The action taken in the event
vendor_event_description	User created	keyword	The description of the event action
vendor_source_group_id	998	keyword	ID of the group before a change
vendor_source_group_name	group1	keyword	Name of the group before a change
vendor_group_id	999	keyword	ID of the group
vendor_group_name	group2	keyword	Name of the group
vendor_tty	/dev/pts/0	keyword	The terminal from which an action was taken
vendor_user_home	/home/user1	keyword	User home directory
vendor_user_shell	/bin/bash	keyword	User login shell

Hint: User names are case-sensitive in some Linux distributions. While case is retained in the original message and in extracted fields like user_name and source_user_name, searches will treat the values as case-insensitive. For example, a search for user_name:user1 would return results containing both cases.

Linux System Logs Spotlight Content Pack

Linux System Overview

SSH

Sudo

Users and Groups