AWS CloudTrail

The AWS CloudTrail input enables Graylog to ingest log messages directly from the AWS CloudTrail service.

AWS generates CloudTrail logs for every action performed within your account. These events are written to an S3 bucket, and an SQS notification is sent when a new log file is available. The CloudTrail input reads the notifications from SQS and retrieves the corresponding log data from S3.

These logs are valuable for auditing purposes, including tracking user activity, monitoring API usage, and detecting changes to your AWS resources.

Prerequisites

Before proceeding, ensure that the following prerequisites are met:

You must have a valid AWS account with CloudTrail enabled.

Supported Log Types

This input supports collecting the following log types:

See AWS CloudTrail User Guide for all supported log types.

Required Third-Party Setup

To enable integration, complete the following required setup with your third-party service:

Create a trail with AWS CloudTrail and its S3 destination.
Configure the trail to deliver log files to an S3 bucket (create a new bucket or select an existing one). Note the bucket name and region, you will need them for event notifications and IAM.
Create an SQS queue in the AWS region where Graylog will poll (for example, cloudtrail-notifications), ensure the queue is available. Graylog uses it to identify which S3 objects to retrieve, and record the SQS queue name and AWS region for use during the Graylog input setup.
Grant the S3 bucket permission to send messages to the SQS queue. The queue needs an access policy allowing sqs:SendMessage from the CloudTrail S3 bucket; without it, S3 event notifications will silently fail to enqueue. See the AWS documentation for detailed instructions on granting permissions to publish event notification messages to a destination.
Configure S3 event notifications on the CloudTrail bucket. Add a notification for s3:ObjectCreated:* and select one of the following targets:
- Send notifications directly to the SQS queue for a single consumer setup.
- Send notifications to an SNS topic that the SQS queue subscribes to when multiple consumers require the same notifications.
Note that the SNS topic name is used only within AWS. Do not enter it in the Graylog input.
To connect Graylog to AWS S3, create and use an IAM role with permissions to read the target SQS queue and S3 access. Enable the following permissions:
- sqs:ReceiveMessage
- sqs:DeleteMessage
- s3:GetObject
Hint: AWS recommends using Identity and Access Management (IAM) roles with temporary credentials instead of long-term static access keys. When configuring the Graylog input, use the AWS Assume Role (ARN) option whenever possible.
Subscribe the SQS queue to your CloudTrail SNS topic.

Hint: Note that enabling CloudWatch logs and adding tags are optional.

Required Configuration Values

In your third-party configuration, make note of the following values that are required when configuring the input in Graylog:

SQS Queue Name

Input Type

This input is a pull input type. See Inputs to learn about input types.

Input Configuration

Follow the input setup instructions. During setup of this input, you can configure the following options:

Configuration Option	Description
Input Name	Provide a unique name for your new input.
AWS Authentication Type	Select either to allow the system automatically look for credentials using the AWS default credential provider chain or provide AWS Access and Secret Keys.
AWS Assume Role (ARN)	The ARN of the Identity and Access Management (IAM) role that Graylog will assume to access the CloudTrail logs. AWS recommends using IAM roles with temporary credentials instead of long-term static access keys. This option is preferred and supports cross-account access.
AWS SQS Region	Select the AWS region where the SQS queue is hosted.
AWS S3 Region	Select the AWS region where the S3 bucket storing CloudTrail logs resides.
SQS Queue Name	Provide the name of the SQS queue that receives CloudTrail notifications from SNS.
Polling Interval	Determines how often (in minutes) Graylog checks for new data in AWS CloudTrail. The default value is 1 minute.
Enable Throttling	Enables Graylog to stop reading new data for this input whenever the system falls behind on message processing and needs to catch up.
Override source (optional)	By default, the source is a hostname derived from the received packet. This option allows overriding the default aws-cloudtrail source value with a custom value.
SQS Message Batch Size	Sets the maximum number of messages retrieved from SQS per polling cycle; the default is 5, and the maximum is 10.
Include full_message_json	The Include full_message_json option stores the complete raw CloudTrail event as a JSON object in the message.

Hint: If the CloudTrail input is starting and the debug log messages show that messages are being received but no messages are visible when searching in Graylog, then make sure the SQS subscription is not set to deliver the messages in raw format.

Next Steps

After you complete input setup, visit Input Diagnosis for testing and validation of the new input. Use this functionality to help troubleshoot any connection issues.