Breaking Notes

• Graylog 5 is Java 17 only. We no longer support earlier Java versions.

• Support for Elasticsearch 6.X has been removed! Please use either Elasticsearch 7.10.2 or, preferably, latest OpenSearch.

• Graylog 5 needs at least MongoDB 5.0. Our recommended upgrade path is to first bring your MongoDB to 5.0 and then perform the Graylog upgrade. Hint: Graylog 4.3.x does support MongoDB 5.0, which allows for a seamless upgrade path.

• The flatten_json pipeline function now preserves the original type of the extracted values, instead of converting them to string. An optional flag is provided so existing rules can continue using the legacy behavior.

Disallowing Embedding the Frontend by Default

To prevent click-jacking, we are now preventing the frontend from being embedded in <frame>/<iframe>/etc. elements by sending the X-Frame-Options header with all HTTP responses. The header value depends on the new configuration setting http_allow_embedding. The different combinations are:

If you want to be able to embed the Graylog frontend in another HTML page, you most likely want to set http_allow_embedding to true. Only do this if you are aware of the implications!

For further information about the meanings of the different header values and how they are interpreted by browsers, please read this documentation.

New Default Message Processing Order

The new default Message Processing order will run the Message Filter Chain before the Pipeline Processor.

This applies only to new Graylog installations. Existing setups keep the former default order for backwards compatibility.

Stream-Aware Field Types

So far, when listing fields for a query, Graylog has been showing fields for all streams. For some systems, this list may be extremely long and contain many fields that are not present in the query results. It is now possible to change this behavior. When configuration property stream_aware_field_types is set to true, Graylog will periodically collect information on the stream-field relation from Elasticsearch/OpenSearch and use it to provide only those fields that are present in the streams used in the query.

If all of your streams go to dedicated, separate index sets, it is advised to keep the default value of stream_aware_field_types property (false). It will decrease the load on ES/OS and stream separation across index sets, which already helps with showing proper fields for a query. On the other hand, if multiple streams go to the same index sets, and you want precise field types and suggestions, you should set it to true. Consider monitoring your ES/OS load after that change, especially when using huge numbers of fields and streams.

API Endpoint Deprecations

The following API endpoints are deprecated beginning with 5.0.

API Endpoint Removals

The following API endpoints have been removed in 5.0.

Legacy Alert API

5.0 eliminates all of the previously deprecated legacy alert APIs. Content packs that include legacy alerts can still be installed, but the alerts will be silently ignored.

API Endpoint Changes

Java Code API Deprecations

The following Java Code API deprecations have been made in 5.0.

• The org.graylog2.plugin.PluginModule.addNotificationType(name, notificationClass, handlerClass, factoryClass) method has been deprecated in favor of a new/preferred version, which also properly registers the notification configuration content pack entity so that instances the corresponding content pack entity can be installed successfully: org.graylog2.plugin.PluginModule.addNotificationType(name, notificationClass, handlerClass, factoryClass, contentPackEntityName, contentPackEntityClass). See for more info.

Java Code API Changes

The following Java Code API changes have been made in 5.0.

Configuration File Changes

Behavior Changes

• The Prometheus metrics for Graylog inputs were previously only exposed for inputs of type GELFHttpInput. They are now exposed for all configured inputs and labeled accordingly. To support this, the default Prometheus mappings for the following metrics have been changed:

  • input_empty_messages
  • input_incoming_messages
  • input_open_connections
  • input_raw_size
  • input_read_bytes_one_sec
  • input_read_bytes_total
  • input_total_connections
  • input_written_bytes_one_sec
  • input_written_bytes_total

• The system_messages collection in MongoDB will be created as a 50MB capped collection going forward. This happens at creation, so existing system_messages collections remain unconstrained.

• You can manually convert your existing collection to a capped collection by following these instructions.

• Introducing new archive configuration parameter retentionTime in days. Archives exceeding the specified retention time are automatically deleted. By default the behavior is unchanged: archives are retained indefinitely.

• Introducing new input configuration option encoding, enabling users to override the default UTF-8 encoding.

• Note that this encoding is applied to all messages received by the input. A single input cannot handle multiple log sources with different encodings.

• The permissions for which options are populated in the System dropdown menu were updated to more closely match the page that they link to. See graylog2-server#13188 for details. The Page permissions remain unchanged but this could affect the workflow for users with legacy permissions.

• Newly created aggregation widgets will now have rollup disabled by default. Existing widgets are unchanged.

Changed Archived Default Path

On new Graylog installations, the default archiving configuration will now store archives under the data_dir instead of /tmp/graylog-archives. (The data_dir is configured in graylog.conf and defaults to /var/lib/graylog-server)

Configuring Archive Retention Time and Max Value

It is now possible to configure default archive retention time and a limit via configuration flags: default_archive_retention_time and max_archive_retention_time using a duration in days. e.g. 365d.

Microsoft Teams Notification Template Changes

Microsoft Teams notification template parsing no longer parses each line in the template and tries to form a key-value pair using a colon delimiter. This will result in Teams notifications with a templated custom message lacking any formatting. Existing custom templates should be updated to use HTML or Markdown in order to display properly. If using the old default template, it can be replaced with the one found when creating a new Teams notification. It can also be found in this pull request.

Operating Systems

Graylog 5.0 removes official support for the following Linux distributions:

• Debian 8, 9

• Ubuntu 16.04, 18.04

• RHEL/CentOS 6

Operating System Packages

JVM Dependency

The Graylog 5.0 operating system packages bundle version 17 of the JVM, so it's no longer required to install any JVM/Java packages to run Graylog. You can configure Graylog to use an external JVM if required.

Removed Packages

The following operating system packages are no longer available in Graylog 5.0.

• graylog-integrations-plugins
• graylog-enterprise-plugins
• graylog-enterprise-integrations-plugins

Use the graylog-enterprise or graylog-server package instead.

RPM Digest Changes

The RPM packages switched from the legacy SHA1 digest to SHA256 for package signatures. The checksums of the files inside the package switched from the legacy MD5 to SHA256.

Package Upgrade

RPM

Install the latest graylog-5.0-repository RPM package to update the repository metadata.

When using Graylog Operations

sudo rpm -Uvh https://downloads.graylog.org/repo/packages/graylog-5.0-repository_latest.rpm
sudo yum clean all
sudo yum install graylog-enterprise

When using Graylog Open

sudo rpm -Uvh https://downloads.graylog.org/repo/packages/graylog-5.0-repository_latest.rpm
sudo yum clean all
sudo yum install graylog-server

If you are using the plugin packages, you have to remove them before upgrading.

DEB

Install the latest graylog-5.0-repository DEB package to update the repository metadata.

When using Graylog Operations

wget https://downloads.graylog.org/repo/packages/graylog-5.0-repository_latest.deb
sudo dpkg -i graylog-5.0-repository_latest.deb
sudo apt-get update
sudo apt-get install graylog-enterprise

When using Graylog Open

wget https://downloads.graylog.org/repo/packages/graylog-5.0-repository_latest.deb
sudo dpkg -i graylog-5.0-repository_latest.deb
sudo apt-get update
sudo apt-get install graylog-server

New Functionality

Sidecar

The Sidecar Administration UI now allows the assignment of multiple configurations for a single collector. Please note that this feature requires a Sidecar with version 1.3 or greater. Older versions will only run a single (random) configuration per collector.