Upgrade to Graylog 7.2

This article contains important information about upgrading to Graylog 7.2. For detailed steps, see our upgrade guides for supported operating systems and services:

Breaking Changes

Entity Suggestion Search: Regex No Longer Supported in Query Parameter

The query parameter of the entity suggestion endpoint is now treated as a plain-text substring rather than a regular expression. Previously, passing a regex pattern (for example, foo.*bar) would be evaluated by MongoDB, which allowed ReDoS attacks through crafted inputs.

After upgrading, queries containing regex metacharacters (., *, +, ?, (, ), and so on) are matched literally instead of being interpreted as a pattern.

Paginated REST APIs: Case-Insensitive Matching and Sorting

Paginated entity endpoints (for example, streams, event definitions, notifications, lookup tables, dashboards, Sigma rules, and investigations) now share a case-insensitive, numeric-aware collation for both sorting and filtering. Previously, only sort order was affected when individual endpoints opted in; matching was always case-sensitive.

After upgrading:

Sorting by string fields such as title or name interleaves upper- and lower-case entries, and strings containing numbers sort naturally (Stream 2 before Stream 10).
Filter expressions on string fields match case-insensitively. For example, a query that previously matched only test now also matches Test and TEST. API clients relying on exact-case matching through paginated endpoints see additional results.

`gl2_accounted_message_size` Can Now Be `0` for Restored Data Lake Messages

When messages are restored from the data lake, those that do not count against your license traffic now have their gl2_accounted_message_size field set to 0. Previously the field always held the message's accounted size, regardless of whether the restore counted against the license.

This field is informational and is not used to compute license usage, so your license consumption is unaffected by the change.

Changed Parsing for Okta Log Events `securityContext.userBehaviors` Field

Due to a bug in the Okta SDK, a workaround was introduced in 7.1 to stringify the objects in the securityContext.userBehaviors array in logs pulled in from the Okta Log Events input. The updated SDK now properly serializes that field as an array of objects, and the workaround has been removed. Custom parsing on Okta Log Events messages that expects the securityContext.userBehaviors field to be an array of strings must be modified to expect an array of objects, per the Okta API.

In 7.1, the field serialized as:

{ "securityContext": { "userBehaviors": [ "{\"name\":\"New City\",\"id\":\"bbbbbbbbbbbbbbbbbbbb\",\"result\":\"NEGATIVE\"}", "{\"name\":\"New Country\",\"id\":\"aaaaaaaaaaaaaaaaaaaa\",\"result\":\"NEGATIVE\"}" ] } }

In 7.2, the same field serializes as:

{ "securityContext": { "userBehaviors": [ { "name": "New City", "id": "bbbbbbbbbbbbbbbbbbbb", "result": "NEGATIVE" }, { "name": "New Country", "id": "aaaaaaaaaaaaaaaaaaaa", "result": "NEGATIVE" } ] } }

CEF Inputs No Longer Prefix the `source` Field with a Slash

When a CEF message carries no deviceAddress/dvc extension and no syslog hostname, the CEF input falls back to the sender's socket address for the source field. That fallback formatted the address with InetAddress.toString(), which renders hostname/1.2.3.4. Because the CEF input never resolves the hostname, the hostname half was always empty and source was left with a leading slash (/128.66.23.42). IPv6 senders were additionally recorded in fully expanded form (0:0:0:0:0:0:0:1).

source now holds the bare address, compressed for IPv6 (128.66.23.42, ::1), which is what every other input already does.

Check this if you built anything around the old value. Pipeline rules that strip the leading slash off source, saved searches or lookup tables keyed on the slash-prefixed form, and stream rules matching it no longer match and should be updated. Messages ingested before the upgrade keep their original source, so both forms can coexist in existing indices.

Scripting API Default Fields on Message Export

Graylog now exports all fields in a message by default. Before this change, exports defaulted to a limited list of fields with no option to export all fields, so you had to know through the web interface which fields actually existed. You can now export with all fields and limit the results by specifying the fields you want.

System CPU and Memory Metrics Now Reflect Container Limits When Containerized

When Graylog server or Data Node runs in a container with cgroup CPU and memory limits configured (for example, Docker --memory/--cpus or Kubernetes resources.limits), the org.graylog2.system.cpu.percent metric and the Data Node metrics mem_total, mem_free, mem_total_used_bytes, and mem_total_used now reflect the container's cgroup-scoped limits and usage instead of the underlying host's.

Previously, these metrics always reported host-level values, so a container with a memory limit well below the host's total RAM showed a misleadingly small used percentage. After upgrading, the same metrics scale to the container's actual limit, so used-percentage values can jump significantly even though nothing about the node's real memory or CPU pressure has changed. Review and, if necessary, adjust any dashboards or alert thresholds built against the old host-scaled values.

`stream_aware_field_types` Is Now Enabled by Default

The stream_aware_field_types option now defaults to true. Graylog then tracks which fields are used in which streams, so field lists and field types match the streams you are searching in. This option is enabled because more features now rely on accurate, stream-specific field types.

If your graylog.conf still contains stream_aware_field_types=false from a previous version, the option stays disabled. If you do not configure it (for example, Docker setups using only environment variables), it is enabled after the upgrade.

Enabling this option can decrease performance on systems with many streams and fields. To disable it, set stream_aware_field_types = false (or GRAYLOG_STREAM_AWARE_FIELD_TYPES=false).

Web Interface Changes

Event Definition Fields Step Renamed to Additional Details

The Fields step on the event definition wizard is renamed to Additional Details to better reflect its content, which now covers more than event fields (for example, tags). Along with the visible label, the step's step query parameter changed from fields to additional-details, so the URL for that step is now .../edit?step=additional-details. Existing bookmarked links using the old ?step=fields value continue to work, because they are mapped to the renamed step. Update them to the new value, because support for the old value may be removed in a future version.

Java API Changes

File/Method	Description
`org.graylog2.contentpacks.facades.EntityWithExcerptFacade#resolveGrants`	Removed

Plugin Builds: New `requireUpperBoundDeps` Maven Enforcer Rule

Plugin builds inheriting from the graylog-plugin-parent or graylog-plugin-web-parent Maven parent POM now run the requireUpperBoundDeps enforcer rule. It fails the build when a transitive dependency resolves to a lower version than another of the plugin's dependencies requires.

You can usually fix such conflicts by updating the outdated dependency, or by adding a <dependencyManagement> entry for the flagged artifact using the highest required version shown in the error message. Plugin authors who are unable to align their dependencies can override the enforce-versions execution of the maven-enforcer-plugin in their own POM.

Sigma Rules Folded into Event Definitions

Before 7.2, Sigma rules were first-order entities that you could manage directly. Each rule was also backed by an event definition that controlled the execution scheduling and could have some of its configuration managed directly. Sigma rules are now folded into event definitions, and the first-order Sigma rule entity no longer exists. You can create Sigma rule event definitions by either a file upload or a configured Git repository import. Manual modification of Sigma rule source YAML is no longer supported. After a Sigma rule is imported as an event definition, all management is handled directly on the event definition. In support of this change, the Security > Sigma Rules menu option and its associated interface are removed. Management of Sigma rule Git repositories has moved to Alerts > Sigma Repos.

Sigma correlation rules can no longer be directly imported or uploaded. For event_count and value_count types, you can import the correlated rules and then modify the resulting event definitions to add aggregation information without importing another Sigma rule. For temporal_ordered types, you can create an Event Correlation event definition after the rules are imported to create the same temporal correlation.

All previously imported Sigma rules, including correlation rules, are migrated to the new event definition pattern on upgrade and work as they did before.

Sigma Level to Event Definition Priority Correction

Event definitions created from Sigma rules previously mapped to only a select set of the supported priority values. They now map to the full range, and all existing rules have the correct priority applied during the 7.2 upgrade.

Sigma Level	Before	After
`informational`	1 (Low)	0 (Informational)
`low`	1 (Low)	1 (Low)
`medium`	2 (Medium)	2 (Medium)
`high`	3 (High)	3 (High)
`critical`	3 (High)	4 (Critical)

Changes to Sigma Events

The sigma_rule_tag_* fields are no longer added to fired events. Previously, fired events included sigma_rule_tag_1, sigma_rule_tag_2, and so on in their additional fields. MITRE information recognized by Graylog is now stored on the event definition as tactics_techniques, and any other tag values are moved to a dedicated tags field. Both tactics_techniques and tags are multi-valued (array) fields, unlike the individual sigma_rule_tag_N fields they replace. If you rely on the sigma_rule_tag_* fields in summary templates, notification bodies, or downstream processing, update those references.
The Sigma: prefix is no longer added to fired event titles. Events already stored in the index keep their original title, but newly fired events do not include the prefix.

For previously imported Sigma rules, the upgrade migration applies these changes to the event definitions automatically: each source Sigma rule's tag values are written onto its event definition (MITRE references into tactics_techniques, all other tags into tags), and the Sigma: prefix is removed from the event definition title. You do not need to re-import or reconfigure your rules. Events already stored in the index are not rewritten: they keep the original sigma_rule_tag_* additional fields and titles they were fired with. The changes above apply to events fired after the upgrade.

Sigma API Changes

Endpoint	Description
`POST /plugins/org.graylog.plugins.securityapp.sigma/sigma/rules/validate_zip`	Moved to `POST /plugins/org.graylog.plugins.securityapp.sigma/sigma/import/validate_zip`
`POST /plugins/org.graylog.plugins.securityapp.sigma/sigma/rules/import`	Moved to `POST /plugins/org.graylog.plugins.securityapp.sigma/sigma/import/bulk/import`
`POST /plugins/org.graylog.plugins.securityapp.sigma/sigma/rules/upload`	Moved to `POST /plugins/org.graylog.plugins.securityapp.sigma/sigma/import/bulk/upload`
All other `/plugins/org.graylog.plugins.securityapp.sigma/sigma/rules/...`	Deleted (use the event definition API to manage)

Threat Coverage Percentages May Change After Upgrade

Due to the migration of Sigma rules to event definitions in 7.2, the percentages displayed in the Threat Coverage widget may differ from what they were in 7.1. Coverage is now computed directly from event definitions rather than from the previous Sigma rules. Every event definition with MITRE tactics or techniques assigned is now included, and coverage reflects how many of them are enabled versus disabled, with no log source check that was previously present for Sigma rules. A tactic may therefore show a higher or lower percentage than it did in 7.1, without any change to the actual installed event definitions.

AWS Kinesis/CloudWatch Input: Required DynamoDB Permissions

In Graylog 7.2, the AWS Kinesis/CloudWatch input is upgraded to Kinesis Client Library (KCL) 3.5, which calls DynamoDB actions that KCL 2.x did not. This applies to every Kinesis/CloudWatch input, whether it is new or existed before 7.2, so a policy written for an earlier Graylog release can look correct and still deny the input. A policy that grants only the actions KCL 2.x needed leaves the input logging an authorization error on a fixed schedule while consuming no records. In 7.2, Graylog watches the two calls the consumer cannot work without, DynamoDB lease discovery (Query) and the Kinesis record fetch (GetRecords), and fails the input naming the denied action once one has been denied continuously for two minutes. A denial KCL works around, such as one affecting only lease rebalancing, still leaves the input running.

In addition to the actions the lease table already needed (CreateTable, DescribeTable, GetItem, PutItem, Scan, UpdateItem, DeleteItem), KCL 3.5 requires:

dynamodb:Query on arn:aws:dynamodb:<region>:<account>:table/graylog-aws-plugin-*/index/*. KCL 3.5 discovers leases through a global secondary index on the lease table. An index is a separate IAM resource from its table, so granting Query on the table alone still denies this call.
dynamodb:UpdateTable on arn:aws:dynamodb:<region>:<account>:table/graylog-aws-plugin-*. KCL creates that index on first start. If this is denied, KCL retries the call throughout startup and then aborts, so the input fails with a generic initialization error rather than starting. That is outside the steady-state detection described above, which watches denials of Query and GetRecords on a running input, but the input still fails visibly instead of consuming nothing.

The two legacy tables, <application-name>-CoordinatorState and <application-name>-WorkerMetricStats, also need access, and how much depends on the input:

An input created on 7.2 or later needs only dynamodb:DescribeTable on both. KCL checks whether they exist on every start and treats anything other than "table not found" as a failure, so a policy scoped to the lease table alone prevents the input from starting even though it never had these tables.
An input that existed before 7.2 keeps using both tables until you complete the single-table migration described in the next section. Until then they need the same item-level actions as the lease table (GetItem, PutItem, UpdateItem, DeleteItem, Scan) in addition to DescribeTable: KCL holds its leader lock in -CoordinatorState and writes worker metrics to -WorkerMetricStats every 30 seconds.

A policy whose resource is arn:aws:dynamodb:<region>:<account>:table/graylog-aws-plugin-* covers the lease table and both legacy tables, because their names share that prefix.

If you enable the single-table migration described in the next section, it moves state entities in a DynamoDB transaction, which additionally requires dynamodb:ConditionCheckItem on the -CoordinatorState table alongside the item-level actions above. ConditionCheckItem is used only by transactions, so policies written for non-transactional access usually omit it, and without it the migration never completes while reporting nothing in the Graylog interface.

AWS Kinesis/CloudWatch Input: Single DynamoDB Table State Tracking

KCL stores its coordination state in DynamoDB. Before KCL 3.5, this used three tables per input: the lease table, plus separate <application-name>-CoordinatorState and <application-name>-WorkerMetricStats tables. KCL 3.5 introduced a single-table format that consolidates all of this into the lease table alone, where each item is tagged with an entityType attribute. This reduces the number of DynamoDB tables and helps you stay under account-level table limits.

There are two things to know about how this applies to your inputs:

New inputs use the single-table format automatically. This is the AWS KCL 3.5 default for a newly created consumer. A fresh input always uses a single lease table.
Existing inputs keep their three-table layout until you deliberately migrate them using the new Migrate to single DynamoDB table for state tracking input option.

To let you migrate existing inputs on your own schedule, the input's Edit input dialog exposes a Migrate to single DynamoDB table for state tracking option. This option is only relevant for inputs created before Graylog 7.2. Enabling it on such an input starts a one-way migration that consolidates the -CoordinatorState and -WorkerMetricStats entities into the input's lease table. Stream checkpoints are preserved, so ingestion should continue without replay or gaps.

Migration Steps for a Kinesis Input That Existed Before Graylog 7.2

Upgrade to Graylog 7.2 and allow the input to start and run for one hour with the Migrate to single DynamoDB table for state tracking option off. KCL 3.5 requires the input to run steadily for a whole hour before it accepts the migration. This readiness period is fixed by KCL 3.5 and cannot be shortened.
Before enabling the option, confirm the input is ready. In DynamoDB, open the input's coordinator-state table (graylog-aws-plugin-<stream-name>-CoordinatorState), find the TableMigration3.5 item, and check its tm attribute. It must read TABLE_MIGRATION_STATUS_DEPLOYED. If it still reads TABLE_MIGRATION_STATUS_INIT, the readiness period has not elapsed yet; wait and check again. Enabling the option before this point causes the input to fail to start.
Enable the Migrate to single DynamoDB table for state tracking option in the input's Edit input dialog and save. The migration begins.
Verify completion. KCL 3.5 bakes for 24 hours by default before finalizing. After that period, check the same TableMigration3.5 item again. Its tm attribute should read TABLE_MIGRATION_STATUS_COMPLETE. Once complete, the legacy -CoordinatorState and -WorkerMetricStats tables are no longer used and can be deleted.

Warning: The migration is one-way and cannot be reverted once complete. It is also not instantaneous, and AWS recommends monitoring the migration until it reaches completion.

For how to monitor the migration, along with the migration steps and how to remove the now-unused legacy tables afterward, see the AWS documentation. Use the KCL 3.x to 3.5 guide for inputs that were already running KCL 3.x, and the KCL 2.x to 3.x guide for inputs that were running KCL 2.x: