Data Adapters

Data adapters enable the ingestion and processing of data from a wide range of sources. By ingesting inputs such as Syslog, GELF, JSON, or CSV, these adapters convert raw data into formats that can be analyzed and visualized within Graylog.

This article will explain what data adapters are, how they function within Graylog’s lookup system, outline available adapter types across Graylog editions, and guide you through creating a new data adapter in the Graylog interface.

Prerequisites

Before proceeding, ensure that the following prerequisites are met:

• You must have administrative privileges and be able to assign all roles to set up a data adapter. As the Graylog administrator, you create a custom role that includes the necessary permissions. You then assign the newly created role to the appropriate users. See Permission Management.

What Are Data Adapters? 

In addition to handling data ingestion, data adapters play a critical role in value lookups within Graylog’s pipeline rules. They serve as the mechanism that retrieves specific values during data processing. As a result, when configuring a lookup table, setting up data adapters is a fundamental step in ensuring seamless integration and functionality.

The image above illustrates a data processing flow involving a log message, processing pipeline, and lookup table. The log message enters the processing pipeline, where a pipeline rule is applied. This rule requires a lookup operation, which is handled by the lookup table. The lookup table consists of a cache and a data adapter, which retrieve the necessary data from the data source. The result of the lookup is then fed back into the processing pipeline.

Data Adapters in Graylog Open

The following data adapters are available in Graylog Open:

• Alien Vault OTX API

• CSV

• DNS Lookup

• DSV File from HTTP

• Geo IP - MaxMind Databases

• GreyNoise Quick IP Lookup

• HTTP JSONPath

• Spamhaus (E)DROP

• Tor Exit Node

• Whois for IPs

Data Adapters in Graylog Enterprise and Security

The following data adapters are available in licensed products, including Graylog Operations or Graylog Security:

• Active Directory User Lookup

• GreyNoise Full IP Lookup

• MongoDB

• ThreatFOX IOC Lookup

• URLhaus Malware Distribution URL Lookup

Create Data Adapter

In the Graylog interface, to set up a new data adapter:

1. Navigate to System > Lookup Tables > Data Adapters.

2. Click on the Create data adapter button located at the top right corner of the screen.

3. Select the required Data Adapter Type from the drop-down menu.
   
   

4. Input the required configuration parameters as required by the selected data adapter and select Create adapter to complete the process. For more information, review the individual knowledge article specific to your adapter type.

Further Reading

Explore the following additional resources and recommended readings to expand your knowledge on related topics:

• Lookup Tables

• Pipelines

• Pipeline Rule Logic