Set Up an Input

Before Graylog can begin collecting and processing log data, you need to create an input—a defined entry point for incoming messages. Inputs specify how Graylog will communicate with your data sources, allowing it to receive logs from a variety of servers, applications, or network devices.

This article guides you through the process of creating an input, configuring its parameters, and ensuring it is ready to receive data effectively.

Prerequisites

Before proceeding, ensure that the following prerequisites are met:

  • Ensure you have a log source capable of sending data in a format compatible with the selected Graylog input type. Refer to the documentation for your specific input type for detailed guidance on how to set up and configure it properly.

Launch a New Input

Once you’ve identified a log source from which you want to collect data, the next step is to configure an input. To initiate the input setup:

  1. Navigate to System > Inputs.​

  2. Select and launch input:

    • From the Inputs drop-down menu, choose the desired input type (e.g. Syslog UDP, GELF TCP).​

    • Click the Launch new input button.

    • Enter the required information to configure the input.​ (Refer to the input documentation for specific configuration settings related to your input type.) 

    • Click Launch input.​

  3. From the Inputs page, locate the new input and click the Setup Input button.

Set Up the Input

After initially creating the input, note that your input will not be in a running state, meaning that logs are not yet flowing from your log source into Graylog. You should immediately proceed to input setup to complete the process of input configuration and data routing. To do this, select Setup Input located next to the input you have just created. You are then presented with a setup wizard that streamlines the input setup process. The wizard consists of three parts, which are as follows

  1. Make your Illuminate pack selections.

    1. If applicable Illuminate content exists and is selected, then ingested log data is routed by the Illuminate processors to the associated Illuminate stream and index set.

    2. If Illuminate content does not exist or is not available, you are prompted to configure your data routing preferences.

  2. Determine your Data Routing process flow.

  1. You have the options to either route your data to a new stream or to an existing stream. As part of the routing configuration, you are also able to create pipeline and index sets.

  2. If you route your data to an existing stream, note that configurations attached to that stream also apply.

  1. Review the input diagnosis page.

  1. The setup process ends with the Input Diagnosis page, which provides relevant information about the launched input. If the input is launched successfully, the diagnosis displays confirmation messages, health check metrics, and connection status, helping you verify that the input is running as expected. If any issues are detected, the page highlights errors and suggests possible solutions for troubleshooting.

Select Illuminate Packs

Graylog Illuminate is available for use with Graylog Enterprise and Graylog Security. Contact sales to learn more about obtaining Graylog Illuminate.

Input setup mode begins by presenting a list of applicable Illuminate Processing packs associated with a selected input. Illuminate Processing packs contain parsing rules that translate incoming log data into the Graylog Information Model (GIM) Schema, providing normalization and enrichment of log data. For example, if you select the CrowdStrike input, the associated CrowdStrike Illuminate packs are listed for selection. However, if you select a GELF HTTP input, no Processing or Spotlight packs are listed because Illuminate has no associated packs specific to the generic GELF HTTP input.

After you select the necessary Illuminate Processing packs, Graylog displays a list of available Content and Spotlight packs associated with the input. Content and Spotlight packs install relevant dashboards, Sigma rules, and events that read logs parsed by Illuminate to the GIM schema.

Hint: Some Illuminate processing packs are only available with a Graylog Enterprise license.

Below are the two options available with the Illuminate process flow in the input setup wizard:

Select Illuminate Packs

  • Select the Illuminate Processing packs relevant to your input to enhance data parsing and enrichment. If a Content or Spotlight pack is already installed, it appears on the list as a non-selectable option.

  • Click the Next button to launch the input.

When Illuminate Processing packs are selected, the Content and Spotlight packs are imported and activated. With this selection, no stream or pipeline rules are added because messages from the input are routed by Illuminate processors to the corresponding Illuminate stream. Hence, when Illuminate Processing packs are selected, there is no routing process flow. The input is configured based on the associated Illuminate Processing packs.

Skip Illuminate Packs

  • If no Illuminate packs are applicable to the selected input or you choose not to use Illuminate processing, click Skip Illuminate to proceed to set up routing without enabling any Illuminate packs.​

Configure Data Routing

Next, if no Illuminate Processing packs are selected, you must configure how incoming data should be routed. You have two routing options: route to a new stream or route to an existing stream.

Route to a New Stream

  • Choose this option to create a dedicated stream for the input.

    Hint: We strongly recommend creating a new stream for each new input for efficient log data categorization.

  • Provide the following information:​

    • Title: Name of the new stream.

    • Description: Description of the messages routed to this stream.

    • Index Set: Select an existing index set or create a new one.

    • Optionally, you can also create a new pipeline associated with this stream for further message processing.​

If you choose to route to a new stream, messages are routed from the default stream to your new stream via a system-generated pipeline rule in the All Messages default pipeline.

Route to an Existing Stream

  • Select this option to route messages to an existing stream.​

  • Choose the desired stream from the list of existing streams.​

  • Click Next to apply the configurations and start the input based on the selected settings.

Warning: If you choose to route to an existing stream, a new default, immutable pipeline is attached to the All Messages stream, called the All Messages Routing pipeline. You can not detach, delete, or rename the pipeline, and you can not add or remove rules from it.

If the input already has associated stream rules or routing rules, a warning message is displayed.

Input Diagnosis

The final step in the setup process is the diagnosis of the launched input, which is presented on the Input Diagnosis page. This page provides insights into the status and health of the newly launched input. This step ensures that the input is functioning as expected and helps quickly identify and resolve any issues.

The Input Diagnosis page provides information on the address of the input, running nodes, health check metrics, connection status, error reporting, message traffic, and suggested fixes. Completing the diagnosis step is essential to validate that your input is correctly configured and ready to receive and process log data. If errors are reported, they should be addressed before considering the setup process complete.

After you complete the input setup, you can access the Input Diagnosis page for your specific inputs as follows:

  1. Navigate to System > Inputs.

  2. Locate the input, then click the More actions drop-down menu.

  3. Select Input Diagnosis.

Further Reading

Explore the following additional resources and recommended readings to expand your knowledge on related topics: