The Graylog Stack

The Graylog stack comprises three key components: Graylog, Data Node, and MongoDB.

In this article, we’ll describe these components of the Graylog Stack, including important optional elements. We'll explore how these components work together and highlight the interdependency of this stack in a Graylog deployment.

Warning: It is possible to deploy Graylog with self-managed OpenSearch; however, Data Node is the recommended search backend for use with Graylog.

Highlights

The following highlights provide a summary of the key takeaways from this article:

The Graylog Stack consists of Graylog for processing and visualization, Data Node for storage and indexing, and MongoDB for metadata and configuration.
The Data Node enhances scalability and security by simplifying cluster expansion and managing certificate-based communication.
Optional components, such as the Graylog Forwarder and Sidecar, improve log ingestion from remote sources and enable centralized management of log collectors.
Careful planning of both mandatory and optional components is essential for optimizing performance, security, and manageability in a Graylog deployment.

Graylog

Graylog serves as the processing and visualization layer of the Graylog Stack, centralizing log collection from various sources, transforming and normalizing data, and storing it in a searchable format within the Data Node. It provides an intuitive web interface for log analysis, offering powerful search and query capabilities, customizable dashboards, and real-time monitoring.

Beyond visualization, Graylog includes an alerting system that integrates with notification channels like Slack, PagerDuty, and email, ensuring timely responses to critical events. Its data processing pipeline enables advanced log enrichment, filtering, and routing, enhancing log organization and analysis. These features make Graylog a core component for efficient log management, security monitoring, and operational intelligence.

Data Node

The Graylog Data Node is an integral component that functions as the search backend.

Data Node also enhances the protection of the data layer by managing cluster membership and handling certificate issuance and renewal, either automatically or manually. This ensures secure communication between nodes and strengthens the overall security posture of the system.

Additionally, the Data Node simplifies cluster management by making it easier to scale deployments. Adding new nodes to the cluster becomes a streamlined process, and the Graylog web interface provides a centralized platform to monitor and manage the Data Node. Users can track node status, certificate expiration, and performance metrics efficiently through this interface.

Hint: Graylog provides a migration wizard to help transition your existing search backend to Data Node management.

MongoDB

MongoDB serves as a metadata store for Graylog, managing configurations and other operational data. Unlike the Graylog Data Node, which stores and indexes the actual log data, MongoDB is responsible for managing and storing the metadata and configuration settings that make the Graylog system operational. This data includes information such as user accounts, roles, permissions, alert configurations, dashboard layouts, and pipeline rules.

Optional Components of the Graylog Stack

When planning your Graylog deployment, the Graylog Forwarder and Graylog Sidecar are optional yet highly beneficial components that can enhance data collection and management.

Graylog Forwarder

A Forwarder is a lightweight application designed to ship log data directly to Graylog from remote locations or devices. It simplifies log ingestion in distributed setups, ensuring reliable and encrypted data transfer while reducing the need for complex configurations. The Graylog Forwarder is particularly useful for organizations with geographically dispersed systems or compliance requirements for secure log transmission.

Graylog Sidecar

The Graylog Sidecar acts as a lightweight control tool for managing log collectors, such as Filebeat, Winlogbeat, or NXLog, across your Windows, Linux, and Mac hosts. It bridges the gap between log sources and the Graylog server by providing centralized management of collector configurations. This setup ensures consistency across environments and simplifies the process of deploying and maintaining collectors on multiple endpoints.

Both the Forwarder and Sidecar are optional, but you should consider them based on the complexity and scale of your deployment. Use the Forwarder if you need to securely consolidate logs from remote systems or segregated networks. Deploy the Sidecar for central management of log collectors across your Windows, Linux, and Mac hosts.

Including these components in your deployment plan can streamline log collection, enhance security, and reduce the operational overhead associated with managing distributed environments.