The Graylog Extended Log Format (GELF) is a log format that avoids the shortcomings of classic plain Syslog and is perfect for logging from your application layer. It comes with optional compression, chunking, and, most importantly, a clearly defined structure. The Input of GELF messages can be UDP, TCP, or HTTP. Additionally, a queue is possible.

Some applications like Docker can send GELF messages native. Also, fluentd speaks GELF.

There are dozens of GELF libraries for many frameworks and programming languages to get you started. Read more about GELF in the specification.

GELF via HTTP

You can send in all GELF types via HTTP, including uncompressed GELF that is just a plain JSON string.

After launching a GELF HTTP input you can use the following endpoints to send messages:

Copy 
http://graylog.example.org:[port]/gelf (POST)

Try sending an example message using curl:

Copy 
curl -XPOST http://graylog.example.org:12202/gelf -p0 -d '{"short_message":"Hello there", "host":"example.org", "facility":"test", "_foo":"bar"}'

Both keep-alive and compression are supported via the common HTTP headers. The server will return a  202 Accepted  when the message was accepted for processing.

Enable Bulk Receiving Option for HTTP GELF Input

Graylog offers users a new configuration option to Enable Bulk Receiving via HTTP GELF input, which supports bulk receiving of messages separated by new lines.

The input will automatically separate multiple GELF messages, which are new lines delimited (\n or \r\n) when this option is enabled.

Example curl request:

Warning: Individual GELF messages must be formatted as a valid JSON (containing no line breaks within). Attempts to post formatted JSON to this input will result in an error.

Copy 
curl -XPOST -v http://127.0.0.1:12202/gelf -p0 \
-d $'{"short_message":"Bulk message 1", "host":"example.org", "facility":"test", "_foo":"bar"}\r\n\
{"short_message":"Bulk message 2", "host":"example.org", "facility":"test", "_foo":"bar"}\r\n\
{"short_message":"Bulk message 3", "host":"example.org", "facility":"test", "_foo":"bar"}\r\n\
{"short_message":"Bulk message 4", "host":"example.org", "facility":"test", "_foo":"bar"}\r\n\
{"short_message":"Bulk message 5", "host":"example.org", "facility":"test", "_foo":"bar"}'

Hint: Existing support for Transfer-Encoding: chunked is present in the HTTP GELF input, and that support now extends to the new Bulk Receiving feature (when the new Enable Bulk Receiving config option is turned on.

GELF TCP

After launching your new input, configure the following fields based on your preferences: 

  • Global

    • Select this check box to enable the input on all Graylog nodes, or keep it unchecked to enable the input on a specific node.

  • Title  

    • Assign a unique title to the input. Example: “GELF TCP Input for XYZ Source”.

  • Bind address

    • Enter an IP address on which this input listens. The source system/data sends logs to this IP/input.

  • Port

    • Enter a port to use in conjunction with the IP address.

  • Timezone

    • Select the timestamp configured on the system that is sending GELF messages.

  • Receive Buffer Size (optional)

    • Depending on the amount of traffic being ingested by the input, this value should be large enough to ensure proper flow of data but small enough to prevent the system from spending resources trying to process the buffered data.

  • No. of worker threads

    • This setting controls how many concurrent threads are used to process incoming data. Increasing the number of threads can enhance data processing speed, resulting in improved throughput. The ideal number of threads to configure depends on the available CPU cores on your Graylog server. A common starting point is to align the number of worker threads with the number of CPU cores. However, it is crucial to strike a balance with other server demands.

Hint: The TLS-related settings that follow ensure that only valid sources can send messages to the input securely.

  • TLS cert file (optional)

    • The certificate file that is stored on a Graylog system. The value of this field is a path (/path/to/file) that Graylog should have access to.

  • TLS private key file (optional)

    • The certificate private key file that is stored on a Graylog system. The value of this field is a path (/path/to/file) that Graylog should have access to.

  • Enable TLS

    • Select if this input should use TLS.

  • TLS key password (optional)

    • The private key password.

  • TLS client authentication (optional)

    • If you want to require the source of the messages sending logs to this input to authenticate themselves, set to optional or required.

  • TLS Client Auth Trusted Certs (optional)

    • The path where the client (source) certificates are located on a Graylog system. The value of this field is a path (/path/to/file) which Graylog should have access to.

  • TCP keepalive

    • Enable this option if you want the input to support TCP keep-alive packets to prevent idle connections.

  • Null frame delimeter

    • This option is typically left unchecked. New line is the delimiter for each message.

  • Maximum message size

    • The maximum message size of the message. A default value should suffice but can be modified depending on message length. Each input type usually has specifications that note the maximum length of a message.

  • Override Source

    • By default, messages parse the source field as the provided hostname in the log message. However, if you want to override this setting for devices that output non-standard or unconfigurable hostnames, you can set an alternate source name here.

  • Encoding

    • All messages would need to support the encoding configured for the input. UTF-8 encoded messages shouldn’t be sent to an input configured to support UTF-16.

  • Decompressed size limit

    • The maximum size of the message after being decompressed.

GELF UDP

After launching your new input, configure the following fields based on your preferences: 

  • Global

    • Select this check box to enable the input on all Graylog nodes, or keep it unchecked to enable the input on a specific node.

  • Title  

    • Assign a unique title to the input. Example: “GELF UDP Input for XYZ Source”

  • Bind address

    • Enter an IP address that this input will listen on. The source system/data sends logs to this IP/input.

  • Port

    • Enter a port to use in conjunction with the IP address.

  • Timezone

    • Select the timestamp configured on the system that is sending GELF messages.

  • Receive Buffer Size (optional)

    • Depending on the amount of traffic being ingested by the input, this value should be large enough to ensure proper flow of data but small enough to prevent the system from spending resources trying to process the buffered data.

  • No. of worker threads

    • This setting controls how many concurrent threads are used to process incoming data. Increasing the number of threads can enhance data processing speed, resulting in improved throughput. The ideal number of threads to configure depends on the available CPU cores on your Graylog server. A common starting point is to align the number of worker threads with the number of CPU cores. However, it is crucial to strike a balance with other server demands.

  • Override Source

    • By default, messages parse the source field as the provided hostname in the log message. However, if you want to override this setting for devices that output non-standard or unconfigurable hostnames, you can set an alternate source name here.

  • Encoding

    • All messages would need to support the encoding configured for the input. UTF-8 encoded messages shouldn’t be sent to an input configured to support UTF-16.

  • Decompressed size limit

    • The maximum size of the message after being decompressed.