Outputs

An output is an array of configuration settings that define where log messages should be sent after they have been processed by Graylog. When log data first comes into Graylog, it is written to the journal. Then, Graylog is able to retrieve these logs and decode them into a message object. These message objects are routed to the appropriate stream that allows them to be run through processing pipelines, which can send logs to configured outputs.

All streams send their messages to the OpenSearch output by default, but additional outputs can be configured on a per-stream basis. Importantly, in order to send logs out of Graylog, the output must be attached to a stream. You may also send logs internally, from one Graylog cluster to another.

In this article, we discuss the different types of outputs and walk you through how to create a new output, assign an output to a stream, and remove an existing output.

Hint: Outputs can help to queue messages safely if the remote side becomes disconnected or unavailable; however, using a journal output is preferred due to the improved reliability.

Output Types

There are three types of Graylog outputs:

1. The Cluster-to-Cluster Forwarder outputs, which forward messages from a Graylog source cluster to the Graylog destination cluster. (An Enterprise license is required.)

2. Enterprise Output Framework outputs, which write messages to an on-disk journal and then send the data to the external receiver.

3. Legacy (non-framework) outputs, which forward logs between Graylog clusters.

Please see the chart below for a detailed description of individual outputs available and their supported configuration parameters: 

Type	Description	Uses Output Framework	Uses Journal	TLS Available	mTLS Available
Cluster-to-Cluster Forwarder	Forwards logs from one Graylog cluster to another. It requires a corresponding cluster-to-cluster input running in the destination cluster.	no	yes	yes	no
Google Cloud BigQuery	Google Cloud BigQuery allows you to send data to your Google Cloud BigQuery table. Each message in the stream is inserted as a new row in the configured BigQuery table.	yes	yes	no	no
TCP Raw/Plaintext	Sends data as UTF-8 encoded text to an arbitrary TCP endpoint (server and port). No additional formatting or encapsulation is added.	yes	yes	yes	yes
UDP Raw/Plaintext	Sends data via UDP. No additional formatting or encapsulation is added.	yes	yes
TCP Syslog	Sends data as UTF-8 encoded text to an arbitrary TCP syslog receiver. The formatted payload is sent as the MSG portion of a standard syslog message per section 6.4 of the Syslog specification.	yes	yes	yes	yes
GELF	Sends GELF compliant JSON.	no	no	yes	no
STDOUT	Displays formatted messages on the system console. Primarily included as a debugging tool for pipeline changes.	no	no	no	no

The Output Framework and Data Journal

The following section exclusively pertains to a Graylog Enterprise feature. To learn more about obtaining an Enterprise license, please contact the Graylog Sales team.

The Enterprise Output Framework enables data forwarding from Graylog clusters to external systems through a variety of network transport methods and payload formats. Messages sent via this framework are stored in a data journal until they are ready to be sent. The Output Framework can use a separate partition for journals to ensure journal growth does not impact overall system performance.

Additionally, Enterprise Framework outputs can be set up using the full_message or pipeline_output fields. These selections are offered in the drop-down menu under Outbound Payload Format and are explained under Format Outbound Payloads. See Enterprise Output Framework for more details.

Hint: Outputs supported by the Enterprise Output Framework operate independently, which means you can configure each output separately according to your preferences.

Set Up a New Output

Outputs can be created on the System > Outputs page. Any pre-exisiting outputs will be listed on this page. Here you can launch or terminate as many outputs as you want and then assign them to streams. The selected output type determines how an output will be sent, including payload formatting and which network protocol will be used. Depending on the output type, outgoing messages may be enriched or modified. Their payloads may be formatted using JSON or user-defined pipelines. When you assign an output to a stream, you are able to forward all messages in that stream in real-time. Further information about available output plugins can be found in the Graylog Marketplace.

1. Select an output type from the drop-down menu on the System >Outputs page.

2. Click Launch new output.

3. In the Create New Output wizard, enter a descriptive title for your output and review the default values recommended by Graylog for the remaining configuration settings.

   Warning: For these configuration settings, which depend upon your output type, please note that the default values entered are generally recommended by Graylog and are applied based on your protocol. The default values do not need to be modified in most cases; however, you may need to use some discretion in modifying these settings based on your system specifications and how much throughput you plan to send. (For example, if you select to use TCP as a protocol, then the settings such as TCP Reconnect Delay and TCP Keep Alive will be pre-selected. Or if you choose to set up a GELF output for, syslog options will not be selected because the format will be GELF.)

4. When you have finished entering your configuration settings, select Create output.

Additional configurations may be available for Enterprise Output Framework output types. Please see the Enterprise Output Framework documentation for more details.

Assign the Output to a Stream

An output can only send messages when it is assigned to a stream. All messages that a stream receives are also forwarded to its assigned output.

Warning: Messages will not be sent via an output that has not been assigned to a stream.

1. You will find a list of current streams by selecting the Streams tab.

2. Find the stream to which you intend to assign an output and click on the More Actions button.

3. Select Manage Outputs from the menu.

4. Here you can choose an existing output or create a new one.

Remove an Existing Output

You can see all defined outputs in detail on the main Outputs page, found under the System menu. In order to remove an output from a stream:

1. Find the stream on the Streams page.

2. Click on Manage outputs under More Actions.

3. Click on Select Existing Output and choose the output you want to remove.

4. You will then be presented with two options:

   • Delete from stream: This option removes the output from the selected stream, but it will still be visible on the Outputs page.

   • Delete globally: Selecting this option will globally remove the output from all streams (including this one) and terminate it.