NGINX [Engine x] is an HTTP Server that runs on Linux systems. It was originally written by Igor Sysoev and publicly released in 2004.

This pack parses and configures NGINX access logs and error logs. The default log location is: /var/log/nginx/.

Requirement(s)

• A Graylog server with a valid Enterprise license that is running Graylog version 5.1.10 or 5.2.3+.

• This pack will work with all versions of NGINX, as long as the combined log format is used.

• Log folders must be named: access.log.

• Either Filebeat (with Sidecar) or rsyslog are required for delivering logs.

What is Provided

This pack includes parsing rules that convert Nginx logs into Graylog schema-compatible fields. Nginx access logs get the GIM code “180200”.

Input via Filebeat Together with Graylog Sidecar

Please follow the official documentation. You will need to configure your Graylog server and your client(s).

To do this:

1. Create an input and an API key. Then set up Graylog sidecar.

2. Add your clients, e.g. web server.

Graylog Server Settings

1. Create a global Beats input in Graylog.

2. Create a Graylog REST API Access Token and save it.

3. Create a (Linux) filebeat configuration under Sidecar > Configuration with a filebeat on Linux collector.

4. After configuring the file:

• Add the Graylog server IP under hosts.

• Configure the log source to the desired value and configure the field event_source_product with the value NGINX-web.

filebeat.inputs:
- input_type: log
  paths:
    - /var/log/nginx/access.log
    - /var/log/nginx/error.log
  type: filestream
  fields_under_root: true
  fields:
    event_source_product: nginx-web

• Save the configuration

Configure a Client with Filebeat and Graylog Sidecar

1. Install Sidecar on the remote machine. See here for instructions.

2. Edit the/etc/graylog/sidecar/sidecar.yml file and configure:

   • server_url as: GraylogServerIP

   • server_api_token as: your API token

3. Install the Sidecar service.

4. Enable and start the Sidecar service.

5. Check the Sidecar status.

Here are some sample commands for Ubuntu, please use the official documentation.

wget https://packages.graylog2.org/repo/packages/graylog-sidecar-repository_1-5_all.deb
sudo dpkg -i graylog-sidecar-repository_1-5_all.deb
sudo apt-get update && sudo apt-get install graylog-sidecar

sudo gedit /etc/graylog/sidecar/sidecar.yml
server_url: "http://192.168.122.52:9000/api/"
server_api_token: "65ol7edseo24mub8o7pu86h2rsr8j9fjjpimtrm9nrpbjso7cnv"

sudo graylog-sidecar -service install
sudo systemctl enable graylog-sidecar
sudo systemctl start graylog-sidecar
sudo systemctl status graylog-sidecar

6. Install Filebeat.

7. Download the link for the OSS version.

• If you choose to install it manually, install it under /etc/filebeat.

• If you choose to install it via apt-get, no further action is required.

Sample commands for Ubuntu, please use the official documentation.

wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add -
sudo apt-get install apt-transport-https
echo "deb https://artifacts.elastic.co/packages/oss-8.x/apt stable main" | sudo tee -a /etc/apt/sources.list.d/elastic-8.x.list
sudo apt-get update && sudo apt-get install filebeat
sudo systemctl enable filebeat
sudo systemctl start filebeat
sudo systemctl status filebeat

You may edit thefilebeat.yml file via the Sidecar configuration in Graylog.

8. Start the daemon. Any permission issues may be solved with sudo chown root filebeat.yml.

9. Configure filebeat to start automatically after reboot (recommended).

10. Assign a configuration to your machine in Graylog.

Input via rsyslog

• You will need a configured UDP or TCP_syslog input on the Graylog server side.

• Install rsyslog via the official documentation.

• Modify the configured rsyslog.conf file.

Example for /etc/nginx.conf with a UDP input at 1544 on IP 192.168.122.40:

sudo gedit /etc/rsyslog.conf

• Under MODULES add or modify:

# provides UDP syslog reception
$ModLoad imudp
$UDPServerRun 1544
*.* @192.168.122.40:1544;RSYSLOG_SyslogProtocol23Format

The next code block can be set in a different configuration file or at the end of thersyslog.conffile.

module(load="imfile" PollingInterval="10" statefile.directory="/var/spool/rsyslog")
input(type="imfile"
      File="/var/log/nginx/access.log"
      Tag="nginx_web_http_access"
      Severity="info"
      Facility="local6")
input(type="imfile"
      File="/var/log/nginx/error.log"
      Tag="nginx_web_http_error"
      Severity="info"
      Facility="local6")
local6.access        @192.168.122.40:1544

The command to restart the service on Ubuntu:

sudo systemctl restart rsyslog

You can check the status with:

sudo systemctl status rsyslog

Red lines may indicate problems. Sometimes a full reboot of the system is needed.

Tested rsyslog version

8.2212.0

Stream Configuration

This technology pack includes one stream:

"Illuminate:Nginx_web Messages"

If this stream name is already defined, then nothing will be changed. If this stream name does not exist, then it will be created.

Index Set Configuration:

This technology pack includes one index set definition:

NGINX_web Logs

If this index set is already defined, nothing will be changed. If this index set does not exist, it will be created with retention settings of a daily rotation and 90 days of retention. These settings can be adjusted as required after installation.

Log Format Examples:

Access Logs version 1.18
127.0.0.1 - - [04/Mar/2023:19:25:07 -0600] "GET / HTTP/1.1" 200 3543 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/109.0"

Access Logs version 1.24
127.0.0.1 - - [13/Sep/2023:08:34:54 -0500] "GET / HTTP/1.1" 200 615 "-" "Mozilla/5.0 (X11; Linux x86_64; rv:103.0) Gecko/20100101 Firefox/103.0" "-"

Error Logs:
2023/03/05 09:34:46 [emerg] 2032#2032: no "ssl_certificate" is defined for the "listen ... ssl" directive in /etc/nginx/sites-enabled/default:21

2023/01/05 11:50:44 [error] 22053#0: *3 open() "/usr/local/stefan/nginx/1.10.2_1/html/stest" failed

NGINX Spotlight Content Pack

The NGINX content pack provides an Overview Dashboard.