Raw HTTP Input

The Raw HTTP input allows the ingestion of plain-text HTTP requests. This input can be used to receive arbitrary log format messages in Graylog over HTTP protocol.

Hint: This input listens for HTTP posts on the /raw path.

Graylog Configuration

When launching a new Raw HTTP input from the Graylog Inputs tab, the following configuration parameters need to be completed:

  • Global

    • Select this check box to enable this input on all Graylog nodes, or keep it unchecked to enable the input on a specific node.

  • Node

    • Select the node on which to start this input. If the Global check box is selected, this option is not available.

  • Title

    • Provide a unique name for your input.

  • Bind Address

    • Enter an IP address for this input to listen on. The source system/data sends logs to this input via this IP address.

  • Port

    • Enter a port to use in conjunction with the IP address.

  • Receive Buffer Size (optional)

    • This setting determines the size of the buffer that stores incoming data before it is processed. A larger buffer can accommodate more data, reducing the chance of data loss during high traffic periods. Depending on the amount of traffic being ingested by the input, this value should be large enough to ensure proper flow of data but small enough to prevent the system from spending resources trying to process the buffered data. The optimal size depends on your network traffic volume. Graylog's default setting is somewhat conservative at 256 KB for testing and small deployments, so if you are dealing with high volumes of NetFlow data, increasing this value is advised. A practical recommendation is to start with a buffer size of at least 1 MB (1024 KB) and adjust based on observed performance.

  • No. of Worker Threads (optional)

    • This setting controls how many concurrent threads are used to process incoming data. Increasing the number of threads can enhance data processing speed, resulting in improved throughput. The ideal number of threads to configure depends on the available CPU cores on your Graylog server. A common starting point is to align the number of worker threads with the number of CPU cores. However, it is crucial to strike a balance with other server demands.

    • Hint: The TLS-related settings that follow ensure that valid sources can send messages to the input securely.

  • TLS Cert File (optional)

    • The certificate file that is stored on a Graylog system. The value of this field is a path (/path/to/file) that Graylog must have access to.

  • TLS Private Key File (optional)

    • The certificate private key file that is stored on a Graylog system. The value of this field is a path (/path/to/file) that Graylog must have access to.

  • Enable TLS

    • Select if this input should use TLS.

  • TLS Key Password (optional)

    • The private key password.

  • TLS Client Authentication (optional)

    • If you want to require the source of the messages sending logs to this input to authenticate themselves, set to optional or required.

  • TLS Client Auth Trusted Certs (optional)

    • The path where the client (source) certificates are located on a Graylog system. The value of this field is a path (/path/to/file) which Graylog must have access to.

  • TCP Keepalive

    • Enable this option if you want the input to support TCP keep-alive packets to prevent idle connections.

  • Enable Bulk Receiving

    • Enable this option to receive bulk messages separated by newlines (\n or \r\n).

  • Enable CORS

    • Enable Cross-Origin Resource Sharing (CORS) to configure your server to send specific headers in the HTTP response that instruct the browser to allow cross-origin requests.

  • Max. HTTP chunk size (optional)

    • For large data, it is common practice to chunk smaller blocks (e.g. 8 KB or 64 KB chunks) to prevent overwhelming buffers. The maximum HTTP chunk size is 65536 bytes.

  • Idle writer timeout (optional)

    • The maximum amount of time the server waits for a client to send data when writing to an output stream before closing the connection due to inactivity.

  • Authorization Header Name (optional)

    • Specify a custom authorization header name to optionally enforce authentication for all received messages. This setting is a way to add password-like security for this input.

  • Authorization Header Value (optional)

    • Specify the authorization header value to optionally enforce authentication for all received messages.

  • Override source (optional)

    • By default, messages parse the source field as the provided hostname in the log message. However, if you want to override this setting for devices that output non-standard or unconfigurable hostnames, you can set an alternate source name here.

  • Encoding (optional)

    • All messages need to support the encoding configured for the input. Default encoding is UTF-8. For example, UTF-8 encoded messages should not be sent to an input configured to support UTF-16.

After launching a Raw HTTP input you can use the following endpoints to send messages:

Copy 
http://graylog.example.org:[port]/raw (POST)

Try sending an example message using curl:

Copy 
curl -XPOST http://graylog.example.org:12202/raw -d 'Sample message'