CSV File Data Adapter

The CSV File data adapter can read key/value pairs from a CSV file.

Prerequisites

Before proceeding, ensure that the following prerequisites are met:

The file uses utf-8 encoding.
The file is readable by every Graylog server node.
The first line in the CSV file needs to be a list of field or column names.

Warning: The CSV File adapter reads the entire contents of the file into HEAP memory. Ensure that you size the HEAP accordingly.

Configure the Data Adapter

You can create a data adapter during the lookup table creation workflow, or they can be created separately on the Data Adapters tab. The following configuration options are available for this data adapter:

Title	A short and unique title for this data adapter.
Description	Data adapter description.
Name	The name used to refer to this data adapter. This should be something unique and recognizable within your Graylog environment.
Custom Error TTL	Time-to-live for custom error messages in seconds.
File path	The path to the CSV file (specify the full path to the CSV file you want to ingest). Ensure Graylog has read permissions for this file. Note that this file must exist locally on all Graylog nodes.
Check interval	The interval to check if the CSV file needs a reload (in seconds). Hint: The CSV File adapter refreshes its contents within each check interval if the file was changed. If the cache was purged but the check interval has not elapsed, lookups might return expired values.
Separator	Define the character used to separate fields in your CSV file (e.g.`,` for comma-separated files).
Quote character	The character to use for quoted elements (e.g. `"`).
Key column	The column name that should be used for the key lookup.
Value column	The column name that should be used as the value for a key.
Allow case-insensitive lookups	Enable this checkbox if the key lookup should be case-insensitive.
CIDR lookup	Enable this checkbox if the keys in the lookup table are in CIDR notation and lookups are done with IP addresses. See CIDR Lookup below for more information about this option.

CIDR Lookup in CSV File Data Adapters

A Classless Inter-Domain Routing (CIDR) address is an IP address ending in a slash. The number following the slash represents the number of addresses in the range.

The CIDR lookup option is a checkbox at the bottom of the data adapters configuration page. If you do not select this option, the data adapter performs exact key matching and looks for an identical pattern. If you select the CIDR lookup option, lookups compare the key (which must be an IP address) to the CIDR address keys of the adapter. The CIDR addresses are searched to find a matching IP address.

An example list of key value pairs:

Copy

key: 192.168.100.0/24, value: "Finance Department subnet"
key: 192.168.101.0/24, value: "IT Department subnet"
key: 192.168.102.0/24, value: "HR Department subnet"

In this case, a lookup on the IP address 192.168.101.117 would return “IT Department subnet.”

Example CSV Pipeline Rule

This pipeline rule uses a CSV lookup table to translate numeric destination port values into readable descriptions (like “HTTPS” or “DNS”) and adds that description as a new field in each message.