Graylog Illuminate is available for use with Graylog Enterprise and Graylog Security. Contact sales to learn more about obtaining Illuminate.

The following article describes how to install Illuminate using an externally provided bundle obtained from your sales representative. For information on how to download Illuminate via the Graylog interface, which is the preferred method of installation, see the related documentation.

It is recommended that you use this article for installation if your Graylog server cannot access the public internet or if you are operating Graylog with a free Enterprise license.

Hint: If your Graylog server cannot access the public internet, then you may also wish to turn off notifications for new Illuminate versions as this may generate error messages. Adjust the following setting in your server.conf file to turn off Illuminate version notifications: illuminate_hub_new_version_check_interval=0s.

Prerequisites

The following is required prior to installing Graylog Illuminate:

  • A Graylog server running at least version 5.0.0.

  • A valid Enterprise or Security license.

  • Administrator access to the Graylog server.

Download the Illuminate Bundle

Once you have obtained the Illuminate bundle, download the file to a system that can be used to access the Graylog server web interface where Illuminate will be installed.

Illuminate Installation With The Graylog Web Interface

  1. Log into your Graylog web interface using an account that has administrative privileges.

  2. Click on the Enterprise menu and select Illuminate.

Installing Graylog Illuminate

When installing Illuminate for the first time, you will be presented with the Illuminate Install page after navigating to the Illuminate menu. It will contain a box on the right side of the page with the text, "Drag & Drop your Illuminate package here, or click to choose manually."

Upload the Illuminate Bundle

Upload the bundle using one of two methods:

  • Manual

    • You can click on the box to open a file browser window on your local system, browse to the directory where the Illuminate bundle zip file was downloaded, select the Illuminate bundle zip file, and then click Open.

  • Drag and Drop

    • You can drag and drop the bundle zip file from a file explorer window on your operating system to the drag & drop section of the installer page.

After the bundle has uploaded, you will see a message informing you that the bundle was uploaded successfully but that the previous version is still active until you activate the upgraded bundle.

Click on the Continue to Packs Manager link. This will return you to the Illuminate Processing Packs page.

Illuminate Pack Selection

After the Illuminate installation is complete, navigate back to Enterprise > Illuminate for a list of Illuminate packs that can be activated on your Graylog system. You can enable the following Illuminate packs from this menu:

Activating Illuminate Packs

1. Browse through the list of packs provided by Illuminate using the controls near the bottom of the page, selecting any packs you wish to activate.

2. When you have selected all of your chosen packs, click Enable Selected on the upper right of the Illuminate packs list window.

Illuminate Core Extension Packs

There are some Illuminate packs that are optional add-on extensions to the functionality of Illuminate core. The optional packs are:

About the Anomaly Detection Add-on Pack

Graylog Security includes an anomaly detection feature, and Graylog Illuminate provides an anomaly detection content pack containing pre-defined rules that work with Illuminate. This add-on provides:

  • An index set and stream definition for events generated by the anomaly detection functionality in Graylog Security.

  • Rules to enrich events required by Graylog anomaly detection rules pack to analyze events processed by Graylog Illuminate.

This pack must be enabled to utilize the Graylog Security anomaly detection functionality with the anomaly detection definitions included in the Graylog Illuminate anomaly detection rules content pack.

About the GIM Enforcement Pack

What is GIM?

GIM, short for Graylog Information Model, is how we ensure known types of messages that have been properly categorized will have the necessary fields required for processing.

Why Enable GIM Enforcement?

GIM Enforcement, when enabled, will ensure that all events that have been categorized and intended to be available for search and aggregation, even if the message has been parsed incorrectly. The GIM Enforcement rules will identify categorized messages that are missing required fields; mark those fields and assign default values for the missing fields. Missing fields can be due to log format changes between versions of a product or unexpected data in the message that the parsing logic did not account for.

When the GIM Enforcement rules identify a categorized message that is missing a required field, they will add a field named gim_error with a value that identifies the categorization assignment that failed, and then they will assign a placeholder value to the fields missing values. The placeholder values assigned depend upon the field type:

  • Text fields will be assigned the value _undefined_.

  • Numeric fields will be assigned the value 0.

  • IP fields will be assigned the value 0.0.0.0.

For example, all logon events should have the field user_name. With GIM Enforcement enabled, any message that has been categorized but is missing one of these required fields will have a default value assigned, and the field gim_error will be added indicating that the message is incomplete. This will ensure that searches, which look for logon messages by user_name, will include these messages in related search results and aggregations.

Without GIM enforcement messages may not be included in search results or aggregations if they have been improperly parsed or if they are malformed in some way.

We recommend enabling GIM enforcement at least occasionally when troubleshooting field extraction issues or performing a test or review of data quality.

Warning: Be aware that enabling GIM Enforcement will incur additional slight computational costs per categorized message.

About the Geolocation and Autonomous System (AS) Packs

Two technology packs support geolocation and ASN enrichment: one supporting MaxMind city and AS databases and another supporting IPinfo city and AS databases.

Warning: We recommend that you activate only one of these packs. As they both provide similar functionality, enabling both will increase the computational cost of processing messages with no benefit.

Illuminate Geolocation and AS Deprecation Notice

The Graylog Illuminate Geolocation and AS enrichment processing packs are deprecated and will be removed from a future version of Graylog Illuminate. The functionality of these packs is replaced by the Geolocation Processor. Instead of using the Illuminate Geolocation Processor packs, please configure the Geolocation Processor, making sure the Enforce default Graylog schema option is selected.

Enabling MaxMind Geolocation and ASN Enrichment

The “Geolocation and AS Enrichment Add-on for MaxMind Databases Geolocation and ASN Enrichment” requires that two files be installed on every Graylog Enterprise node in your cluster:

  • The MaxMind City database in MMDB format with the file name GeoLite2-City.mmdb.

  • The MaxMind AS database in MMDB format with the file name GeoLite2-ASN.mmdb.

These files must be placed in the directory /etc/graylog/server on all Graylog nodes in your cluster for the enrichment to function properly.

Enabling IPinfo Geolocation and ASN Enrichment

The “Geolocation and AS Enrichment Add-on for IPinfo Databases” requires that two files be installed on every Graylog Enterprise node in your cluster:

  • The IPinfo City database in MMDB format with the file name standard_location.mmdb.

  • The IPinfo ASN database in MMDB format with the file name asn.mmdb.

These files must be placed in the directory /etc/graylog/server on all Graylog nodes in your cluster for the enrichment to function properly.

Illuminate Spotlights

The Illuminate "Spotlight" content packs are a component of Illuminate that contain Graylog web interface content such as dashboards and saved searches.

Most of the Spotlight content packs are product focused and are a companion to the Illuminate packs included in the Illuminate bundle, but there are additional content packs included that provide other content.

Installation of the Spotlight content packs is optional and does not affect the operation of the Illuminate processing packs.

Additional Spotlight Content

In addition to the product Spotlight content packs, there are some additional content packs included with Illuminate:

  • The Message Summaries content pack (for Graylog Security 5.0.0+): summarizes messages in the message view that have been categorized according to the GIM model, called "message summaries."

  • Event Definition content packs: contains pre-defined event definitions.