Graylog Illuminate is available for use with Graylog Enterprise and Graylog Security Graylog Security. Contact sales to learn more about obtaining Illuminate.

HAProxy is an open-source software solution that provides a high-performance and highly available TCP and HTTP load balancer and proxy server.

About HAProxy Log Collection

  • HAProxy HTTP server on a Linux system

  • This pack will parse out and configure HAProxy TCP, HTTP, HTTPS, and TCP logs

Requirements

  • The content pack supports logging via syslog.

  • The official setup guide for HAProxy and syslog can be found here.

Hint: Graylog needs a syslog input with the matching IP, port, and protocol to receive the logs.

Supported Version(s)

  • Tested with HAProxy version 2.9

Stream Configuration

This technology pack includes one stream:

  • "Illuminate:HAProxy LoadBalancer Messages"

Hint: If this stream name is already defined, then nothing will be changed. If this stream name does not exist, then it will be created.

Index Set Configuration

This technology pack includes one index set definition:

  • "HAProxy LoadBalancer Messages"

Hint: If this index set is already defined, then nothing will be changed. If this index set does not exist, it will be created with retention settings of a daily rotation and 90 days of retention. These settings can be adjusted as required after installation.

Supported Log Formats

  • Default/Connection Logs

  • Error Logs

  • TCP Logs (option tcplog)

  • HTTP Logs (option httplog)

  • HTTPS Logs (option httpslog)

Log Format Examples

Connection Log

haproxy[1234]: Connect from 10.0.1.2:33312 to 10.0.3.31:8012 (www/HTTP)

Error Log

haproxy[6103]: 127.0.0.1:56059 [03/Dec/2023:17:35:10.380] frt/f1: Connection error during SSL handshake

TCP Log

haproxy[14387]: 10.0.1.2:33313 [06/Feb/2023:12:12:51.443] graylog-in static/srv1 8/9/5007 212 PH -1/2/3/4/5 6/7

HTTP/HTTPS Log

haproxy[1234]: 10.0.1.2:33317 [06/Feb/2023:12:14:14.655] https-in static/srv1 11/0/31/68/108 200 2353 - - ---- 1/2/3/4/5 6/7 {graylog.de} {} "GET /index.html HTTP/1.1" 0/0/0/0/0 graylog.de/TLSv1.3/TLS_AES_256_GCM_SHA384

Warning: Custom log formats are not supported.

What is Provided

Parsing rules to extract HAProxy logs into Graylog schema-compatible fields.

  • HAProxy default logs get the GIM code "129999" (network.default)

  • Logs with the http_request get the GIM code "180000" (http.message)

Mappings

Default Log Structure

The default format is deprecated, but the content pack will support it.

Feb 16 12:12:29 localhost haproxy[14115]: Connect from 10.0.12.20:59312 to 10.0.23.131:2022 (www/HTTP)

Field Name Example Value Description
process_name haproxy Process name
process_id 14115 HAProxy's Process ID
source_ip 10.0.12.20 IP address of the client which initiated the connection
source_port 59312 TCP port of the client which initiated the connection
destination_ip 10.0.23.131 IP address the client connected to
destination_port 2022 TCP port the client connected to
vendor_frontend_name www The name of the frontend or listener
vendor_frontend_mode HTTP Mode the frontend is operating (TCP or HTTP)

Default Fields for TCP/HTTP and HTTPs Logs

Field Name Example Value Description
process_name haproxy Process name
process_id 14115 Process ID
source_ip 10.0.1.22 TCP port of the client that initiated the connection
source_port 33213 TCP port of the client that initiated the connection

event_received_time

02/Feb/2023:12:12:51.222

Exact date when the connection was received by HAProxy

vendor_frontend_name

graylog

Name of the frontend (or listener) that received and processed the connection

vendor_backend_name

backend1

Name of the backend (or listener)

vendor_server_name

srv1

Name of the last server to which the connection was sent

vendor_tw

1

Total time in milliseconds spent waiting

vendor_tc

2

Total time in milliseconds spent waiting for the connection to establish to the final server

vendor_tt

5007

Total time in milliseconds elapsed between the accept and the last close

destination_bytes_sent

212

Total number of bytes transmitted from the server to the client when the log is emitted

captured_request_cookie

optional field

Optional name=value entry indicating that the client had this cookie in the request

captured_response_cookie

optional field

Optional name=value entry indicating that the server has returned a cookie with its response

vendor_termination_state

-- or ----

The condition the session was in when the session ended

vendor_actconn

3

Total number of concurrent connections on the process when the session was logged

vendor_feconn

4

Total number of concurrent connections on the frontend when the session was logged

vendor_beconn

5

Total number of concurrent connections handled by the backend when the session was logged

vendor_srv_conn

6

Total number of concurrent connections still active on the server when the session was logged

vendor_retries

7

Number of connection retries experienced by this session when trying to connect to the server

vendor_srv_queue

8

Total number of requests which were processed before this one in the server queue

vendor_backend_queue

9

Total number of requests which were processed before this one in the backend's global queue

TCP log format

haproxy[14387]: 10.0.1.22:33213 [02/Feb/2023:12:12:51.222] graylog backend1/srv1 1/2/5007 212 -- 3/4/5/6/+7 8/9

HTTP/HTTPS format

haproxy[14389]: 10.0.1.2:33317 [06/Feb/2023:12:14:14.655] https-in graylog/srv1 10/0/30/69/109 200 27250 cookie - C--- 10/1/1/1/0 20/20 {1graylog.de} {fun.de} "GET /index.html HTTP/1.1" 20/21/22/23/24 grt.de/TLSv1.3/TLS_AES_256_GCM_SHA384

Additional fields for HTTP/HTTPS logs:

Field Name Example Value Description
vendor_trr 69 Total time in milliseconds spent waiting for the server to send a full HTTP response

vendor_ta

109

Total time the request remained active in HAProxy

http_response_code

200

HTTP status code returned to the client

http_request_cookie

cookie

An optional name=value entry indicating that the client had this cookie in the request

http_response_cookie

-

An optional name=value entry indicating that the server has returned a cookie with its response

vendor_captured_request_headers

1graylog.de

List of headers captured in the request due to the presence of the "capture request header" statement

vendor_captured_response_headers

fun.de

A list of headers captured in the response due to the presence of the "capture response header" statement

http_request_method

GET

HTTP request method

http_request_path

/index.html

HTTP request path

http_version

1.1

HTTP version

vendor_fc_err

20

Status of the connection on the frontend's side

vendor_ssl_fc_err

21

Last error of the first SSL error stack that was raised on the connection from the frontend's perspective

vendor_ssl_c_err

22

Status of the client's certificate verification process

vendor_ssl_c_ca_err

23

Status of the client's certificate chain verification process

vendor_ssl_fc_is_resumed

24

True if the incoming TLS session was resumed with the stateful cache or a stateless ticket

vendor_ssl_fc_sni

grt.de

SNI (Server Name Indication) presented by the client to select the certificate to be used

vendor_ssl-version

TLSv1.3

SSL version of the frontend

vendor_ssl_ciphers

TLS_AES_256_GCM_SHA38

SSL cipher used for the connection

HAProxy Spotlight Content Pack

HAProxy offers five dashboards: a general overview dashboard, a dashboard for default logs, one for TCP logs, one for HTTP and HTTPS logs, and an error dashboard.

Overview Dashboard

Default Logs Dashboard

TCP Logs Dashboard

HTTP and HTTPS Logs Dashboard

Error Dashboard