Set a Data Retention Policy

You can set a data retention policy for your Data Lake so that data is held in storage for only the amount of time you determine is required. This ability can be a key part of compliance with legal and regulatory requirements. In fact, you can set a global retention policy for the Data Lake but also include separate retention settings for individual streams, where required.

This article explains how to establish a global retention policy as well as how to set up override policies for individual streams.

Prerequisites

Before proceeding, ensure that the following prerequisites are met:

• You must be a Graylog administrator to set up and manage a Data Lake.

• You should understand your organization's retention and compliance needs.

Set Global Retention Policy

To set a global data retention policy for the Data Lake:

1. Navigate to the Configuration tab of the Data Lake > Setup page.

2. Under Retention settings, set the value for Maximum number of days in the Data Lake.

   Hint: You set this value using the conventions for durations from the ISO 8601 standard.

3. Click Update configuration.

Set Stream Override Retention Policy

To set a data retention policy for an individual stream:

1. Navigate to the Overview tab of the Data Lake > Setup page and locate the appropriate stream.

2. Click Data Routing.

   Hint: You can also navigate to the Streams page, locate the stream, click Data Routing, then proceed to step 3, Destinations.

3. In the Data Lake section, click Data Retention.

4. In the dialog box, set the value for Maximum number of days in the Data Lake.

   Hint: You set this value using the conventions for durations from the ISO 8601 standard.

5. Click Update.

Further Reading

Explore the following additional resources and recommended readings to expand your knowledge on related topics: