The following content pack is available for use with a Graylog Illuminate license and Graylog Enterprise or Graylog Security. Contact sales to learn more about obtaining Illuminate.

Illuminate functionality includes enriching events with additional data that can help contextualize events. This is done throughout Illuminate, both in the core functionality and in many of the Illuminate processing packs. This article focuses on the enrichments added and used by Illuminate core.

Some of the lookups provided in Illuminate can be customized by the user, as detailed explained in the descriptions of the lookups where it applies.

Severity Mapping Enrichments

Event logs and alerts will often have a severity assigned. Sometimes this event is provided as a plain text severity, and sometimes it is provided as a numeric value. Severity mapping lookups are used to ensure that, where either the text or numeric value is the only value defined, that corresponding value will be defined.

The severity mapping lookups included with Illuminate core:

  • illuminate-mapping-alert-severity-to-level: When a message includes an alert_severity value but not an alert_severity_level field, this mapping is used to define the level value.
  • illuminate-mapping-level-to-alert-severity: When a message includes an alert_severity_level numeric value but not an alert_severity value, this mapping is used to define the severity value.
  • illuminate-mapping-event-severity-to-level: When a message includes an event_severity value but not an event_severity_level field, this mapping is used to define the level value.
  • illuminate-mapping-event-level-to-severity: When a message includes an event_severity_level numeric value but not an event_severity value, this mapping is used to define the severity value.

Warning: These lookups should not be customized.

GIM Data Enrichment

This lookup takes the gim_event_type_code value and defines the following fields:

  • gim_event_class
  • gim_event_category
  • gim_event_subcategory
  • gim_event_type

Warning: The GIM event data mapping lookup is core_gim_data_lookup. This lookup should not be customized.

Network Range Enrichment Lookup

Illuminate provides the ability to define category fields for source, destination, and host systems defined in logs. This enrichment will examine the fields source_ip, destination_ip, and host_ip. If the IP address in those fields is in a range that has been added to the network ranges lookup, then any category values defined in that lookup will be assigned to the fields source_category, destination_category, or host_category, respectively.

After installing Graylog Illuminate and enabling any content pack, there will be a lookup added with a lookup adapter titled core_networks_adapter. You can add entries to this lookup adapter and provide one or more category values that will be added to incoming logs.

Defining Categories for Hosts Based On Network Ranges

Warning: Use caution when adding lookup table customization entries for which you are modifying the intended lookup table adapter.

  1. Navigate to Enterprise > Illuminate in the Graylog interface.
  2. Click on the Customization tab located on the top left-hand side of the Illuminate page. Here you will see a list of Illuminate lookup adapters.
  3. Locate the lookup adapter named core_networks_lookup and select the Edit button for the adapter, located to the right of the adapter entry. You will see an entry box with the key column titled ip range and a value column titled ip category data.
  4. Add a CIDR-notation formatted IP range (both IPv4 and IPv6 are supported) in the ip range column.
  5. Add one or more category values, separating each category value with a pipe character (|), in the ip category data column.

We recommend using only alphanumeric characters and replacing spaces with a replacement character, such as an underscore (_), to simplify searching for these category values.

HTTP Data Enrichment Adapters

These lookups will modify some common HTTP fields with useful data. There are two lookup adapters in this group:

  1. lookup_http_class
  2. lookup_http_response

Illuminate will detect when the http_response_code field is defined and enrich these events with two fields:

  • http_response_class: The response class is based upon the first digit of the HTTP response (RFC2616#6.1.1).
  • http_response: The response phrase corresponds with the response code (RFC2616#6.1.1 and RFC6585).

Network Protocol Enrichments

These lookups will enrich events based on the presence of fields associated with network-related event logs.

Network Protocols

Some network event sources will provide a numeric value representing the IANA-registered protocol number (which should be assigned to the field network_iana_number) but not the protocol name. This enrichment, using a lookup with the lookup adapter named core_iana_to_network_transport, will identify when only network_iana_number is defined and use it to determine the protocol name, which will be assigned to the field network_transport.

Reserved/Non-Routable Address Identification

This lookup will inspect the key fields:

  • source_ip
  • destination_ip
  • host_ip

When any of these fields exist, a lookup with the adapter titled core_ip_processing_reserved_ip_ranges_adapter will be used to identify the use of any reserved IP address ranges. The reserved ranges used are:

Range(s) Tag Suffixes
192.168.0.0/16, 172.16.0.0/12, 10.0.0.0/8, RFC1918 reserved_ip, rfc1918, is_internal
127.0.0.0/8, RFC1122: See section 3.2.1.3 regarding loopback addressing reserved_ip, is_internal, is_loopback, rfc1122
169.254.0.0/16, RFC3927: IPv4 link-local addressing (A.K.A. APIPA addressing) reserved_ip, is_internal, rfc3297
224.0.0.0/4, RFC3171: IPv4 Multicast addressing multicast, reserved_ip, rfc3171
fe80::/10, ff00::/8, ::1/128, ::/128, ::FFFF:0:0/96, RFC4291: See section 2.4, IPv6 Address Architecture reserved_ip, is_internal, rfc4291
fc00::/7, RFC4193: See section 3.1, IPv6 unique local unicast addressing reserved_ip, is_internal, rfc4193
2002::/16, RFC3056: IPv6 encapsulation of IPv4 reserved_ip, 6_to_4, rfc3056
2001::/32, RFC4380: Teredo reserved_ip, teredo, rfc4380
192.0.2.0/24, 198.51.100.0/24, 203.0.113.0/24, RFC5737: IPv4 reserved address block for documentation reserved_ip, is_illegal, rfc5737
198.18.0.0/15, RFC2544: IPv4 reserved addresses for benchmark testing reserved_ip, is_internal, rfc2544
2001:db8::/32, RFC3849: IPv6 reserved address block for documentation reserved_ip, is_illegal, rfc3849
2001:10::/28, RFC4843: Orchid routing reserved_ip, rfc4843

Illuminate will detect when an IP in one of the key fields is in scope for any of the identified ranges and add values to the gim_tags field. The values added have some common values and some that are specific to the ranges, and these will be prefixed with the context of the key field for which Illuminate has identified. For example, if the value source_ip value is the IP address 192.0.2.10 , Illuminate will add the following values to the gim_tags field:

  • source_reserved_ip
  • source_is_illegal
  • source_rfc5737