NetFlow Input

NetFlow, a network protocol developed by Cisco, provides IP traffic data that allows for monitoring and analysis. With Graylog, you can collect IP flow data to include source, destination, service data, and other associated data points.

NetFlow sends data using User Datagram Protocol (UDP), which is a connectionless protocol. Support for NetFlow export is device-dependent.

Prerequisites

Before proceeding, ensure that the following prerequisites are met:

If the input is set up behind a load balancer, configure UDP persistence or use a single non-global input to ensure consistent delivery.

Input Type

This Input is a listener input type. See Inputs to learn about Input types.

Associated Illuminate Content Pack

This log source has associated Illuminate content:

NetFlow Content Pack

Hint: If an Illuminate pack is available for your log source, enable it before configuring an input to avoid creating duplicate entities.

Input Configuration

Follow the input setup instructions. During setup of this input, you can configure the following options:

Configuration Option	Description
Global	Select this check box to enable the input on all Graylog nodes, or keep it unchecked to enable the input on a specific node.
Node	Select the Graylog node this input will be associated with.
Title	Provide a unique name for your new input. Example: `NetFlow Input for XYZ Source`.
Bind Address	Enter an IP address that this input will listen on. The source system/data will send logs to this IP/Input.
Port	Enter a port to use in conjunction with the IP. The default port of 2055 is the standard for most devices. However, if you need multiple inputs, you need to refer to vendor documentation on other port options (9555, 9995, 9025, and 9026 are common options).
Receive Buffer Size (optional)	This setting determines the size of the buffer that stores incoming data before it is processed. A larger buffer can accommodate more data, reducing the chance of data loss during high traffic periods. Depending on the amount of traffic being ingested by the input, this value should be large enough to ensure proper flow of data but small enough to prevent the system from spending resources trying to process the buffered data. The optimal size depends on your network traffic volume. Graylog's default setting is somewhat conservative at 256 KB for testing and small deployments, so if you are dealing with high volumes of NetFlow data, increasing this value is advised. A practical recommendation is to start with a buffer size of at least 1 MB (1024 KB) and adjust based on observed performance.
No. of worker threads	This setting controls how many concurrent threads are used to process incoming data. Increasing the number of threads can enhance data processing speed, resulting in improved throughput. The ideal number of threads to configure depends on the available CPU cores on your Graylog server. A common starting point is to align the number of worker threads with the number of CPU cores. However, it is crucial to strike a balance with other server demands.
Override source (optional)	By default, messages parse the source field as the provided hostname in the log message. However, if you want to override this setting for devices that output non-standard or unconfigurable hostnames, you can set an alternate source name here.
Encoding (optional)	All messages need to support the encoding configured for the input. For example, UTF-8 encoded messages should not be sent to an input configured to support UTF-16.
NetFlow 9 field definitions (optional)	NetFlow v9 field definitions specify how each data type is interpreted. It is crucial to define fields accurately to ensure that the collected NetFlow data is correctly parsed and understood. You should customize field definitions to match the specific types of data your network devices export. Below is a sample .yml file structure for defining NetFlow v9 fields. This example includes commonly exported fields by a Juniper Networks EX series switch. Please note that the actual fields and their IDs may vary depending on the switch configuration and NetFlow version. Adjust these definitions based on the specific NetFlow data exported by your Juniper Networks EX series switch. netflow_definitions: # Basic flow fields - id: 1 name: IN_BYTES type: UNSIGNED64 description: Incoming counter with length N x 8 bits for the number of bytes associated with an IP Flow. - id: 2 name: IN_PKTS type: UNSIGNED64 description: Incoming counter with length N x 8 bits for the number of packets associated with an IP Flow. - id: 10 name: INPUT_SNMP type: UNSIGNED32 description: Input interface index. Use this value to query the SNMP IF-MIB. - id: 14 name: OUT_BYTES type: UNSIGNED64 description: Outgoing counter with length N x 8 bits for the number of bytes associated with an IP Flow. - id: 15 name: OUT_PKTS type: UNSIGNED64 description: Outgoing counter with length N x 8 bits for the number of packets associated with an IP Flow. - id: 16 name: OUTPUT_SNMP type: UNSIGNED32 description: Output interface index. Use this value to query the SNMP IF-MIB.

Device Sampling Rate

The following table includes recommended sampling rates for your devices based on average traffic volume.

Data Volume (95th percentile)	Recommended Sampling Rate
< 25 Mb/s	1 in 1
< 100 Mb/s	1 in 128
< 400 Mb/s	1 in 256
< 1 Gb/s	1 in 512
< 5 Gb/s	1 in 1024
< 25 Gb/s	1 in 2048

Next Steps

After you complete input setup, visit Input Diagnosis for testing and validation of the new input. Use this functionality to help troubleshoot any connection issues.