NetFlow Input

NetFlow, a network protocol developed by Cisco, provides IP traffic data that allows for monitoring and analysis. With Graylog, you can collect IP flow data to include source, destination, service data, and other associated data points. Support for NetFlow export is device-dependent.

Configure NetFlow Input in Graylog

After launching your new input, configure the following fields based on your preferences: 

• Global

  • Select this check box to enable the input on all Graylog nodes, or keep it unchecked to enable the input on a specific node.

• Node

  • Select the Graylog node this input will be associated with.

• Title

  • Assign a title for the input. Example : “NetFlow Input for XYZ Source”.

• Bind Address

  • Enter an IP address that this input will listen on. The source system/data will send logs to this IP/Input.

• Port:

  • Enter a port to use in conjunction with the IP. The default port of 2055 is the standard for most devices. However, if you need multiple inputs, you need to refer to vendor documentation on other port options (9555, 9995, 9025, and 9026 are common options).

• Receive Buffer Size (optional)

  • This setting determines the size of the buffer that stores incoming data before it is processed. A larger buffer can accommodate more data, reducing the chance of data loss during high traffic periods. Depending on the amount of traffic being ingested by the input, this value should be large enough to ensure proper flow of data but small enough to prevent the system from spending resources trying to process the buffered data. The optimal size depends on your network traffic volume. Graylog's default setting is somewhat conservative at 256 KB for testing and small deployments, so if you are dealing with high volumes of NetFlow data, increasing this value is advised. A practical recommendation is to start with a buffer size of at least 1 MB (1024 KB) and adjust based on observed performance.

• No. of worker threads

  • This setting controls how many concurrent threads are used to process incoming data. Increasing the number of threads can enhance data processing speed, resulting in improved throughput. The ideal number of threads to configure depends on the available CPU cores on your Graylog server. A common starting point is to align the number of worker threads with the number of CPU cores. However, it is crucial to strike a balance with other server demands.

• Override source (optional)

  • By default, messages parse the source field as the provided hostname in the log message. However, if you want to override this setting for devices that output non-standard or unconfigurable hostnames, you can set an alternate source name here.

• Encoding (optional)

  • All messages need to support the encoding configured for the input. For example, UTF-8 encoded messages should not be sent to an input configured to support UTF-16.

• NetFlow 9 field definitions (optional)

  • NetFlow v9 field definitions specify how each data type is interpreted. It is crucial to define fields accurately to ensure that the collected NetFlow data is correctly parsed and understood. You should customize field definitions to match the specific types of data your network devices export.

    Below is a sample .yml file structure for defining NetFlow v9 fields. This example includes commonly exported fields by a Juniper Networks EX series switch. Please note that the actual fields and their IDs may vary depending on the switch configuration and NetFlow version. Adjust these definitions based on the specific NetFlow data exported by your Juniper Networks EX series switch.

netflow_definitions:
  # Basic flow fields
  - id: 1
    name: IN_BYTES
    type: UNSIGNED64
    description: Incoming counter with length N x 8 bits for the number of bytes associated with an IP Flow.
  - id: 2
    name: IN_PKTS
    type: UNSIGNED64
    description: Incoming counter with length N x 8 bits for the number of packets associated with an IP Flow.
  - id: 10
    name: INPUT_SNMP
    type: UNSIGNED32
    description: Input interface index. Use this value to query the SNMP IF-MIB.
  - id: 14
    name: OUT_BYTES
    type: UNSIGNED64
    description: Outgoing counter with length N x 8 bits for the number of bytes associated with an IP Flow.
  - id: 15
    name: OUT_PKTS
    type: UNSIGNED64
    description: Outgoing counter with length N x 8 bits for the number of packets associated with an IP Flow.
  - id: 16
    name: OUTPUT_SNMP
    type: UNSIGNED32
    description: Output interface index. Use this value to query the SNMP IF-MIB.

Device Sampling Rate

The following table includes recommended sampling rates for your devices based on average traffic volume.