Cloudflare Content Pack

Cloudflare is a comprehensive cloud web-based security and performance platform that works by filtering out malicious traffic, preventing cyberattacks like DDoS, and optimizing content delivery to ensure faster loading times for users.

This technology pack will process Cloudflare logs, providing normalization and enrichment of those events.

Supported Version(s)

Cloudflare 2024.10.2

Stream Configuration

This technology pack includes one stream:

"Illuminate:Cloudflare Messages"

Hint: If this stream does not exist prior to the activation of this pack, then it will be created and configured to route messages to this stream and the associated index set. There should not be any stream rules configured for this stream.

Index Set Configuration

This technology pack includes one index set definition:

"Cloudflare Event Log Messages"

Hint: If this index set is already defined, then nothing will be changed. If this index set does not exist, then it will be created with retention settings of a daily rotation and 90 days of retention. These settings can be adjusted as required after installation.

Log Collection

You can leverage Graylog's Raw HTTP input to receive logs directly from Cloudflare via its Logpush functionality. By configuring Logpush, users can define which types of logs to send (e.g. HTTP requests, firewall events) and specify Graylog as the destination endpoint. This allows seamless log forwarding from Cloudflare to Graylog for centralized log analysis. For detailed instructions, refer to the Graylog documentation on setting up Logpush with the Raw HTTP input.

Graylog Server Configuration

Create a new Raw HTTP input for receiving Cloudflare logs, unless one already exists. This input must be used for Cloudflare logs only.
Once created (or if the Raw HTTP input has already been created), click Show received messages to obtain the input ID (this will pull up a search window with the All Time timeframe). If there are a large number of logs, then you may want to adjust the timeframe to speed up the process.
Copy the gl2_source_input value.
Navigate to Enterprise >Illuminate and select the Customization tab.
Locate the lookup_adapter_input_routing title and click Edit on the right. For the content_name key, enter the gl2_source_input ID copied earlier. For the input_id value, enter cloudflare. (Note that the column names are reversed; this will be corrected in a future release).
Select Configure value to confirm.
If receiving Cloudflare logs on multiple inputs, repeat this process for each input.

Log Format Example

{"Action":"log","ClientASN":209,"ClientASNDescription":"CENTURYLINK-US-LEGACY-QWEST","ClientCountry":"us","ClientIP":"205.186.39.189","ClientIPClass":"noRecord","ClientRefererHost":"","ClientRefererPath":"","ClientRefererQuery":"","ClientRefererScheme":"","ClientRequestHost":"cloudtest.life","ClientRequestMethod":"GET","ClientRequestPath":"/","ClientRequestProtocol":"HTTP/1.1","ClientRequestQuery":"","ClientRequestScheme":"http","ClientRequestUserAgent":"Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.61 Safari/537.36","ContentScanObjResults":[],"ContentScanObjSizes":[],"ContentScanObjTypes":[],"Datetime":"2024-10-16T14:03:41Z","Description":"Test_WAF","EdgeColoCode":"DFW","EdgeResponseStatus":404,"Kind":"firewall","LeakedCredentialCheckResult":"none","MatchIndex":0,"Metadata":{"ruleset_version":"2","version":"1","type":"customer"},"OriginResponseStatus":404,"OriginatorRayID":"00","RayID":"8d389800b7c76c57","Ref":"","RuleID":"af14fd3aedda46898b13c9d033ad0a5f","Source":"firewallcustom"}

What is Provided

We provide categorization for the following log types:
- HTTP Requests
- Firewall Events
- DNS logs
We provide parsing rules to normalize and enrich Cloudflare log messages.

Events Processed by This Technology Pack

The Cloudflare content pack supports parsing and GIM categorization for the HTTP Request, Firewall Events, and DNS Logs events.