AWS CloudTrail

The AWS CloudTrail input enables Graylog to ingest log messages directly from the AWS CloudTrail service.

AWS generates CloudTrail logs for every action performed within your account. These events are written to an S3 bucket, and an SQS notification is sent when a new log file is available. The CloudTrail input reads the notifications from SQS and retrieves the corresponding log data from S3.

These logs are valuable for auditing purposes, including tracking user activity, monitoring API usage, and detecting changes to your AWS resources.

Prerequisites

Before proceeding, ensure that the following prerequisites are met:

You must have a valid AWS account with Amazon CloudTrail enabled.

Supported Log Types

This input supports collecting the following log types:

See AWS CloudTrail User Guide for all supported log types.

Required Third-Party Setup

To enable integration, complete the following required setup with your third-party service:

Create a trail with AWS CloudTrail.
Configure SNS notifications for CloudTrail.
Create a new SNS topic. Specify a name for the topic (e.g. cloudtrail-log-write) or choose from existing topics. This name is needed to configure the Graylog input.
Specify a queue name (e.g. cloudtrail-notifications). This name is needed to configure the Graylog input. The recommended default value is cloudtrail-notifications. CloudTrail writes notifications with S3 file name references to this queue.
To connect Graylog to AWS S3, create and use an IAM role with permissions to read the target SQS queue and S3 access. Enable the following permissions:

sqs:ReceiveMessage

sqs:DeleteMessage

sqs:GetQueueAttributes
Subscribe the SQS queue to your CloudTrail SNS topic.
Enabling Cloudwatch logs and adding tags are optional.

Required Configuration Values

In your third-party configuration, make note of the following values that are required when configuring the input in Graylog:

SNS topic
SQS Queue Name

Input Type

This input is a pull input type. See Inputs to learn about input types.

Input Configuration

Follow the input setup instructions. During setup of this input, you can configure the following options:

Configuration Option	Description
Title	Provide a unique name for your new input.
AWS SQS Region	Select the AWS region the queue is in.
AWS S3 Region	Select the AWS region where the S3 bucket storing CloudTrail logs resides.
SQS Queue Name	Provide the SQS queue name you created, where SNS is writing CloudTrail notifications to.
Enable Throttling	Enables Graylog to stop reading new data for this input whenever the system falls behind on message processing and needs to catch up.
AWS access key (optional)	The unique identifier created for the AWS Identity and Access Management (IAM) user. The Credential settings retrieval order documentation provides more information.
AWS secret key (optional)	The access key ID for the IAM user with permission to the subscriber and the SQS queue.
AWS assume role (ARN) (optional)	This setting is often used for cross-account access.
Override source (optional)	By default, the source is a hostname derived from the received packet. You can override the default value with a custom string. This option allows you to optimize the source for your specific needs.
Encoding (optional)	All messages need to support the encoding configured for the input. For example, UTF-8 encoded messages should not be sent to an input configured to support UTF-16.

Hint: If the CloudTrail input is starting and the debug log messages show that messages are being received but no messages are visible when searching in Graylog, then make sure the SQS subscription is not set to deliver the messages in raw format.

Next Steps

After you complete input setup, visit Input Diagnosis for testing and validation of the new input. Use this functionality to help troubleshoot any connection issues.