CEF Inputs

Common Event Format (CEF) is an extensible, text-based format designed to support multiple device types. It defines a syntax for log records consisting of a standardized header and a variable extension section, formatted as key-value pairs.

Most network and security systems support either Syslog or CEF as a means for sending data. Graylog provides the option to ingest CEF-formatted messages over UDP and TCP transport protocols or Kafka and AMQP as a queuing system.

Prerequisites

Before proceeding, ensure that the following prerequisites are met:

Input Type

This Input is a listener input type. See Inputs to learn about Input types.

Input Configuration

Follow the input setup instructions. During setup of this input, you can configure the following options:

The CEF TCP Input allows Graylog to receive Common Event Format (CEF) messages over a reliable, connection-oriented TCP stream.

Configuration Option Description

Global

Select this check box to enable the input on all Graylog nodes, or keep it unchecked to enable the input on a specific node.
Node Select the node on which the input should start.
Title Assign a title to the input. Example: CEF TCP Input for XYZ Source.
Bind address Enter an IP address for this input to listen on. The source system/data sends logs to this IP/input.
Port

Enter a port to use in conjunction with the IP address.
Timezone Select the timestamp configured on the system that is sending CEF messages. If the sender does not include the timezone information, you can configure the timezone applied to the messages on arrival. That configuration does not overwrite the timezone included in the timestamp; however, it is the assumed timezone for messages that do not include timezone information.
Receive Buffer Size (optional) Depending on the amount of traffic being ingested by the input, this value should be large enough to ensure proper flow of data but small enough to prevent the system from spending resources trying to process the buffered data.
No. of worker threads

This setting controls how many concurrent threads are used to process incoming data. Increasing the number of threads can enhance data processing speed, resulting in improved throughput. The ideal number of threads to configure depends on the available CPU cores on your Graylog server. A common starting point is to align the number of worker threads with the number of CPU cores. However, it is crucial to strike a balance with other server demands.

TLS cert file (optional)

The certificate file that is stored on a Graylog system. The value of this field is a path (/path/to/file) that Graylog should have access to, etc/graylog/tls (recommended). Ensure this path is readable by the user running the Graylog service.

TLS private key file (optional)

The certificate private key file that is stored on a Graylog system. The value of this field is a path (/path/to/file) that Graylog should have access to, etc/graylog/tls (recommended). Ensure this path is readable by the user running the Graylog service.

Enable TLS (Checkbox)

Select this option if this input should use TLS.

TLS key password (optional)

The private key password.

TLS client authentication (optional)

If you want to require authentication, set this value to optional or required.

TLS Client Auth Trusted Certs (optional)

The path where client (source) certificates are located on a Graylog system. The value of this field is a path (/path/to/file) to a file or a directory containing client certificates that Graylog should have access to, etc/graylog/tls (recommended). Ensure this is readable by the user running the Graylog service.

TCP keepalive

Enable this option if you want the input to support TCP keep-alive packets to prevent idle connections.

Null frame delimeter

This option is typically left unchecked. New line is the delimiter for each message.

Maximum message size (optional)

The maximum message size of the message. The default value should suffice but can be modified depending on message length. Each input type usually has specifications that note the maximum length of a message.

Locale (optional)

This setting is used to determine the language of the message.

Use full field names

The CEF key name is usually used as the field name. Select this option if the full field name should be used.

The CEF UDP Input allows Graylog to receive Common Event Format (CEF) messages over the User Datagram Protocol (UDP).

Configuration Option Description

Global

Select this check box to enable the input on all Graylog nodes, or keep it unchecked to enable the input on a specific node.
Node Select the node on which the input should start.
Title Assign a title to the input. Example: CEF UDP Input for XYZ Source.
Bind address Enter an IP address for this input to listen on. The source system/data sends logs to this IP/input.
Port

Enter a port to use in conjunction with the IP address.
Timezone Select the timestamp configured on the system that is sending CEF messages. If the sender does not include the timezone information, you can configure the timezone applied to the messages on arrival. That configuration does not overwrite the timezone included in the timestamp; however, it is the assumed timezone for messages that do not include timezone information.
Receive Buffer Size (optional) Depending on the amount of traffic being ingested by the input, this value should be large enough to ensure proper flow of data but small enough to prevent the system from spending resources trying to process the buffered data.
No. of worker threads

This setting controls how many concurrent threads are used to process incoming data. Increasing the number of threads can enhance data processing speed, resulting in improved throughput. The ideal number of threads to configure depends on the available CPU cores on your Graylog server. A common starting point is to align the number of worker threads with the number of CPU cores. However, it is crucial to strike a balance with other server demands.

Locale (optional)

This setting is used to determine the language of the message.

Use full field names

The CEF key name is usually used as the field name. Select this option if the full field name should be used.

The CEF AMQP Input allows Graylog to ingest Common Event Format (CEF) messages from AMQP-compatible message brokers such as RabbitMQ. This input is designed for environments where reliable, broker-based message delivery is essential.

Configuration Option Description

Global

Select this check box to enable the input on all Graylog nodes, or keep it unchecked to enable the input on a specific node.
Node Select the node on which the input should start.
Title Assign a title to the input. Example: CEF AMQP Input for XYZ Source.

Broker hostname

The hostname or IP address of the message broker (e.g., RabbitMQ server).

Broker virtual host

The logical namespace within the broker used to separate applications or tenants.

Prefetch count

The maximum number of messages a consumer can receive before acknowledging them.

Queue

The name of the queue from which messages will be consumed.

Exchange

The exchange to which the input should bind in order to receive messages.

Routing key

The key used by the exchange to route messages into the specified queue.

Number of consumers/queues

Defines how many consumers or queues should be created for message processing.
Timezone Select the timestamp configured on the system that is sending CEF messages. If the sender does not include the timezone information, you can configure the timezone applied to the messages on arrival. That configuration does not overwrite the timezone included in the timestamp; however, it is the assumed timezone for messages that do not include timezone information.

Allow throttling this input (Checkbox)

Enables message rate limiting to control ingestion speed and prevent system overload.

Broker port (optional)

The port number used by the message broker.

Username (optional)

The username for authenticating against the message broker, if required.

Password (optional)

The password associated with the broker username for authentication.

Passive queue declaration

Declares the queue in passive mode, ensuring it exists without modifying or creating it.

Bind to exchange

Binds the input to a specific exchange in the broker to receive messages from it.

Heartbeat timeout (optional)

Interval (in seconds) for client–broker heartbeat checks to detect lost connections.

Enable TLS

Select this option if this input should use TLS.

Re-queue invalid messages

Determines whether invalid/unprocessable messages are placed back into the queue.

Connection recovery interval (optional)

Time interval (in seconds) to wait before retrying a broker connection after failure.

Locale (optional)

This setting is used to determine the language of the message.

Use full field names

The CEF key name is usually used as the field name. Select this option if the full field name should be used.

The CEF Kafka Input allows Graylog to ingest Common Event Format (CEF) messages from a Kafka topic.

Configuration Option Description

Global

Select this check box to enable the input on all Graylog nodes, or keep it unchecked to enable the input on a specific node.
Node Select the node on which the input should start.
Title Assign a title to the input. Example: CEF Kafka Input for XYZ Source.

Legacy mode

Select this option to use the old ZooKeeper-based consumer API.

Bootstrap Servers (optional)

Enter the IP address and port on which the Kafka server is running.

ZooKeeper address (legacy mode only) (optional)

The IP address and port on which the Zookeeper server is running.

Topic filter regex

Enter the topic name filter that is configured in the filebeats.yml file.

Fetch minimum bytes

Minimum byte size a message batch should reach before fetching.

Fetch maximum wait time (ms)

Enter the maximum time (in milliseconds) to wait before fetching.

Processor threads

Enter the number of threads to process. This setting is based on the number of partitions available for the topic.
Timezone Select the timestamp configured on the system that is sending CEF messages. If the sender does not include the timezone information, you can configure the timezone applied to the messages on arrival. That configuration does not overwrite the timezone included in the timestamp; however, it is the assumed timezone for messages that do not include timezone information.

Allow throttling this input

Enables message rate limiting to control ingestion speed and prevent system overload.

Auto offset reset (optional)

Choose the appropriate selection from the drop-down menu if there is no initial offset in Kafka or if an offset is out of range.

Consumer group id (optional)

The name of the consumer group the Kafka input belongs to.

Locale (optional)

This setting is used to determine the language of the message.

Use full field names

The CEF key name is usually used as the field name. Select this option if the full field name should be used.
Custom Kafka properties (optional) Custom Kafka properties (optional).

Next Steps

After you complete input setup, visit Input Diagnosis for testing and validation of the new input. Use this functionality to help troubleshoot any connection issues.

Further Reading

Explore the following additional resources and recommended readings to expand your knowledge on related topics: