The following article exclusively pertains to a Graylog Enterprise feature or functionality. To learn more about obtaining an Enterprise license, please contact the Graylog Sales team.

Graylog allows you to store indexing and processing failure notifications in a dedicated Elasticsearch failure index.

Failure messages are logged and aggregated on a dashboard and used to set up alert notifications, ensuring that you can analyze the messages and understand why the error occurred.

Configuration

The configuration feature is disabled by default. To enable it, navigate to System > Configurations, and scroll down to the Failure Processing section. Here, you can individually activate each failure processing feature.

  • Log Indexing Failures
    • Stores indexer failure notifications in Elasticsearch and logs them in a dedicated Graylog stream.
  • Log Processing Failures
    • Processes failure notifications to be stored in Elasticsearch and logs them in a dedicated Graylog stream.
  • Include Failed Messages
    • Displays a full log message in the failure notification for investigation. Enable either log indexing failures or log processing failures to activate this selection.
  • Continue Processing on Error
    • Stores the original message alongside a new field (gl2_processing_error) with specific error details. Meanwhile, a failure message with the error details is stored in the dedicated Graylog stream. Enable log processing failures to activate this selection.

Once enabled, the widget in System Overview will display a failed-message counter.

Common Indexer Failure Reasons

The most common indexer failure is classified as a “MapperParsingException.”

For additional information on this type of failure, review Common Indexer Failure Reasons.

Common Processing Failure Reasons

A processing failure, which can occur within the Graylog processing stack, may have multiple causes. The following is a list of the most common reasons:

  • “RuleStatementEvaluationError”
    • Occurs when there is an error in the statement between the “then” and “end” values of the pipeline rule.
  • “RuleConditionEvaluationError”
    • Occurs when there is an error in the statement between the “when” and “then” values of the pipeline rule.
  • “ExtractorException”
    • Occurs when an extractor or converter incorrectly reads or extrapolates a message.
  • “MessageFilterException”
    • Occurs when there is a backend system failure involving the Graylog application; further troubleshooting with Graylog support may be required.
  • “InvalidTimestampException”
    • Occurs when there is a failure during an attempt to set or extract a value in the timestamp field. For example, a pipeline rule failed while attempting to extract a timestamp from a string and attempted to assign this null timestamp to a message.
  • “UNKNOWN”
    • The reason for this error is unknown and will require further investigation into the log data.