The following content pack is available for use with a Graylog Illuminate license and Graylog Enterprise or Graylog Security. Contact sales to learn more about obtaining Illuminate.

HAProxy is an open-source software solution that provides a high-performance and highly available TCP and HTTP load balancer and proxy server.

About HAProxy Log Collection

  • HAProxy HTTP server on a Linux system

  • This pack will parse out and configure HAProxy TCP, HTTP, HTTPS, and TCP logs

Requirements

  • The content pack supports logging via syslog.

  • The official setup guide for HAProxy and syslog can be found here.

  • Syslog needs to be configured with the format rfc5424.

Hint: Graylog needs a syslog input with the matching IP, port, and protocol to receive the logs.

Supported Version(s)

  • Tested with HAProxy version 2.9

Stream Configuration

This technology pack includes one stream:

  • "Illuminate:HAProxy LoadBalancer Messages"

Hint: If this stream name is already defined, then nothing will be changed. If this stream name does not exist, then it will be created.

Index Set Configuration

This technology pack includes one index set definition:

  • "HAProxy LoadBalancer Messages"

Hint: If this index set is already defined, then nothing will be changed. If this index set does not exist, it will be created with retention settings of a daily rotation and 90 days of retention. These settings can be adjusted as required after installation.

Supported Log Formats

  • Default/Connection Logs

  • Error Logs

  • TCP Logs (option tcplog)

  • HTTP Logs (option httplog)

  • HTTPS Logs (option httpslog)

Input via Filebeat Together with Graylog Sidecar

Please follow the official documentation. You will need to configure your Graylog Server and your client(s).

  1. Create an input and an API key. Then set up Graylog sidecar.

  2. Add your clients, e.g. web server.

Hint:  It is possible to run HAProxy and Graylog on the same machine, but this is NOT recommended.

Graylog Server Settings

  1. Create a global beats input in Graylog.

  2. Create a Graylog REST API Access Token and save it.

  3. Create a (Linux) filebeat configuration under Sidecar > Configuration with a filebeat on Linux collector.

  4. After configuring the file:

    1. Add the Graylog server IP under hosts.

    2. Configure the log source to the desired value and configure the field event_source_product with the value haproxy. The below template is an example. Please make sure to replace the IP, port and path with the correct values.

Copy 
fields_under_root: true
fields.collector_node_id: ${sidecar.nodeName}
fields.gl2_source_collector: ${sidecar.nodeId}
filebeat.inputs:
    input_type: log
    paths:
        /var/log/haproxy.log
        type: filestream
        fields_under_root: true
        fields:
        event_source_product: haproxy
        output.logstash:
        hosts: ["192.168.122.212:5044"]
        path:
        data: ${sidecar.spoolDir!"/var/lib/graylog-sidecar/collectors/filebeat"}/data
        logs: ${sidecar.spoolDir!"/var/lib/graylog-sidecar/collectors/filebeat"}/log

There must be two spaces in front of event_source_product and - /var....

Ensure that you save the configuration upon completion.

Configure a Client with Filebeat and Graylog Sidecar

  1. Install Sidecar on the remote machine. See the official documentation for instructions.

  2. Edit the /etc/graylog/sidecar/sidecar.yml file and configure:

    1. server_url as: GraylogServerIP

    2. server_api_token as: your API token

  3. Install the Sidecar service.

  4. Enable and start the Sidecar service.

  5. Check the Sidecar status. The following are sample commands for Ubuntu.

    Copy 
    wget https://packages.graylog2.org/repo/packages/graylog-sidecar-repository_1-5_all.deb
    sudo dpkg -i graylog-sidecar-repository_1-5_all.deb
    sudo apt-get update && sudo apt-get install graylog-sidecar

    sudo gedit /etc/graylog/sidecar/sidecar.yml
    server_url: "http://192.168.122.52:9000/api/"
    server_api_token: "65ol7edseo24mub8o7pu86h2rsr8j9fjjpimtrm9nrpbjso7cnv"

    sudo graylog-sidecar -service install
    sudo systemctl enable graylog-sidecar
    sudo systemctl start graylog-sidecar
    sudo systemctl status graylog-sidecar

  6. Install Filebeat.

  7. Download the link for the OSS version.

    1. If you choose to install it manually, install it under /etc/filebeat.

    2. If you choose to install it via apt-get, no further action is required.

      Copy 
      wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add -
      sudo apt-get install apt-transport-https
      echo "deb https://artifacts.elastic.co/packages/oss-8.x/apt stable main" | sudo tee -a /etc/apt/sources.list.d/elastic-8.x.list
      sudo apt-get update && sudo apt-get install filebeat
      sudo systemctl enable filebeat
      sudo systemctl start filebeat
      sudo systemctl status filebeat

    3. You may edit thefilebeat.yml file via the Sidecar configuration in Graylog.

  8. Start the daemon. Any permission issues may be solved with sudo chown root filebeat.yml.

  9. Configure Filebeat to start automatically after reboot (recommended).

  10. Assign a configuration to your machine in Graylog.

Log Format Examples

Connection Log

haproxy[1234]: Connect from 10.0.1.2:33312 to 10.0.3.31:8012 (www/HTTP)

Error Log

haproxy[6103]: 127.0.0.1:56059 [03/Dec/2023:17:35:10.380] frt/f1: Connection error during SSL handshake

TCP Log

haproxy[14387]: 10.0.1.2:33313 [06/Feb/2023:12:12:51.443] graylog-in static/srv1 8/9/5007 212 PH -1/2/3/4/5 6/7

HTTP/HTTPS Log

haproxy[1234]: 10.0.1.2:33317 [06/Feb/2023:12:14:14.655] https-in static/srv1 11/0/31/68/108 200 2353 - - ---- 1/2/3/4/5 6/7 {graylog.de} {} "GET /index.html HTTP/1.1" 0/0/0/0/0 graylog.de/TLSv1.3/TLS_AES_256_GCM_SHA384

Warning: Custom log formats are not supported.

What is Provided

Parsing rules to extract HAProxy logs into Graylog schema-compatible fields.

  • HAProxy default logs get the GIM code "129999" (network.default)

  • Logs with the http_request get the GIM code "180000" (http.message)

Mappings

Default Log Structure

The default format is deprecated, but the content pack will support it.

Feb 16 12:12:29 localhost haproxy[14115]: Connect from 10.0.12.20:59312 to 10.0.23.131:2022 (www/HTTP)

Field Name Example Value Description
process_name haproxy Process name
process_id 14115 HAProxy's Process ID
source_ip 10.0.12.20 IP address of the client which initiated the connection
source_port 59312 TCP port of the client which initiated the connection
destination_ip 10.0.23.131 IP address the client connected to
destination_port 2022 TCP port the client connected to
vendor_frontend_name www The name of the frontend or listener
vendor_frontend_mode HTTP Mode the frontend is operating (TCP or HTTP)

Default Fields for TCP/HTTP and HTTPs Logs

Field Name Example Value Description
process_name haproxy Process name
process_id 14115 Process ID
source_ip 10.0.1.22 TCP port of the client that initiated the connection
source_port 33213 TCP port of the client that initiated the connection

event_received_time

02/Feb/2023:12:12:51.222

Exact date when the connection was received by HAProxy

vendor_frontend_name

graylog

Name of the frontend (or listener) that received and processed the connection

vendor_backend_name

backend1

Name of the backend (or listener)

vendor_server_name

srv1

Name of the last server to which the connection was sent

vendor_tw

1

Total time in milliseconds spent waiting

vendor_tc

2

Total time in milliseconds spent waiting for the connection to establish to the final server

vendor_tt

5007

Total time in milliseconds elapsed between the accept and the last close

destination_bytes_sent

212

Total number of bytes transmitted from the server to the client when the log is emitted

captured_request_cookie

optional field

Optional name=value entry indicating that the client had this cookie in the request

captured_response_cookie

optional field

Optional name=value entry indicating that the server has returned a cookie with its response

vendor_termination_state

-- or ----

The condition the session was in when the session ended

vendor_actconn

3

Total number of concurrent connections on the process when the session was logged

vendor_feconn

4

Total number of concurrent connections on the frontend when the session was logged

vendor_beconn

5

Total number of concurrent connections handled by the backend when the session was logged

vendor_srv_conn

6

Total number of concurrent connections still active on the server when the session was logged

vendor_retries

7

Number of connection retries experienced by this session when trying to connect to the server

vendor_srv_queue

8

Total number of requests which were processed before this one in the server queue

vendor_backend_queue

9

Total number of requests which were processed before this one in the backend's global queue

TCP log format

haproxy[14387]: 10.0.1.22:33213 [02/Feb/2023:12:12:51.222] graylog backend1/srv1 1/2/5007 212 -- 3/4/5/6/+7 8/9

HTTP/HTTPS format

haproxy[14389]: 10.0.1.2:33317 [06/Feb/2023:12:14:14.655] https-in graylog/srv1 10/0/30/69/109 200 27250 cookie - C--- 10/1/1/1/0 20/20 {1graylog.de} {fun.de} "GET /index.html HTTP/1.1" 20/21/22/23/24 grt.de/TLSv1.3/TLS_AES_256_GCM_SHA384

Additional fields for HTTP/HTTPS logs:

Field Name Example Value Description
vendor_trr 69 Total time in milliseconds spent waiting for the server to send a full HTTP response

vendor_ta

109

Total time the request remained active in HAProxy

http_response_code

200

HTTP status code returned to the client

http_request_cookie

cookie

An optional name=value entry indicating that the client had this cookie in the request

http_response_cookie

-

An optional name=value entry indicating that the server has returned a cookie with its response

vendor_captured_request_headers

1graylog.de

List of headers captured in the request due to the presence of the "capture request header" statement

vendor_captured_response_headers

fun.de

A list of headers captured in the response due to the presence of the "capture response header" statement

http_request_method

GET

HTTP request method

http_request_path

/index.html

HTTP request path

http_version

1.1

HTTP version

vendor_fc_err

20

Status of the connection on the frontend's side

vendor_ssl_fc_err

21

Last error of the first SSL error stack that was raised on the connection from the frontend's perspective

vendor_ssl_c_err

22

Status of the client's certificate verification process

vendor_ssl_c_ca_err

23

Status of the client's certificate chain verification process

vendor_ssl_fc_is_resumed

24

True if the incoming TLS session was resumed with the stateful cache or a stateless ticket

vendor_ssl_fc_sni

grt.de

SNI (Server Name Indication) presented by the client to select the certificate to be used

vendor_ssl-version

TLSv1.3

SSL version of the frontend

vendor_ssl_ciphers

TLS_AES_256_GCM_SHA38

SSL cipher used for the connection

HAProxy Spotlight Content Pack

HAProxy offers five dashboards: a general overview dashboard, a dashboard for default logs, one for TCP logs, one for HTTP and HTTPS logs, and an error dashboard.

Overview Dashboard

Default Logs Dashboard

TCP Logs Dashboard

HTTP and HTTPS Logs Dashboard

Error Dashboard