HAProxy is an open-source software solution that provides a high-performance and highly available TCP and HTTP load balancer and proxy server.
About HAProxy Log Collection
-
HAProxy HTTP server on a Linux system
-
This pack will parse out and configure HAProxy TCP, HTTP, HTTPS, and TCP logs
Requirements
-
The content pack supports logging via syslog.
-
The official setup guide for HAProxy and syslog can be found here.
Hint: Graylog needs a syslog input with the matching IP, port, and protocol to receive the logs.
Supported Version(s)
-
Tested with HAProxy version 2.9
Stream Configuration
This technology pack includes one stream:
-
"Illuminate:HAProxy LoadBalancer Messages"
Hint: If this stream name is already defined, then nothing will be changed. If this stream name does not exist, then it will be created.
Index Set Configuration
This technology pack includes one index set definition:
-
"HAProxy LoadBalancer Messages"
Hint: If this index set is already defined, then nothing will be changed. If this index set does not exist, it will be created with retention settings of a daily rotation and 90 days of retention. These settings can be adjusted as required after installation.
Supported Log Formats
-
Default/Connection Logs
-
Error Logs
-
TCP Logs (option tcplog)
-
HTTP Logs (option httplog)
-
HTTPS Logs (option httpslog)
Log Format Examples
Connection Log
haproxy[1234]: Connect from 10.0.1.2:33312 to 10.0.3.31:8012 (www/HTTP)
Error Log
haproxy[6103]: 127.0.0.1:56059 [03/Dec/2023:17:35:10.380] frt/f1: Connection error during SSL handshake
TCP Log
haproxy[14387]: 10.0.1.2:33313 [06/Feb/2023:12:12:51.443] graylog-in static/srv1 8/9/5007 212 PH -1/2/3/4/5 6/7
HTTP/HTTPS Log
haproxy[1234]: 10.0.1.2:33317 [06/Feb/2023:12:14:14.655] https-in static/srv1 11/0/31/68/108 200 2353 - - ---- 1/2/3/4/5 6/7 {graylog.de} {} "GET /index.html HTTP/1.1" 0/0/0/0/0 graylog.de/TLSv1.3/TLS_AES_256_GCM_SHA384
Warning: Custom log formats are not supported.
What is Provided
Parsing rules to extract HAProxy logs into Graylog schema-compatible fields.
-
HAProxy default logs get the GIM code "129999" (
network.default
) -
Logs with the
http_request
get the GIM code "180000" (http.message
)
Mappings
Default Log Structure
The default format is deprecated, but the content pack will support it.
Feb 16 12:12:29 localhost haproxy[14115]: Connect from 10.0.12.20:59312 to 10.0.23.131:2022 (www/HTTP)
Field Name | Example Value | Description |
---|---|---|
process_name | haproxy | Process name |
process_id | 14115 | HAProxy's Process ID |
source_ip | 10.0.12.20 | IP address of the client which initiated the connection |
source_port | 59312 | TCP port of the client which initiated the connection |
destination_ip | 10.0.23.131 | IP address the client connected to |
destination_port | 2022 | TCP port the client connected to |
vendor_frontend_name | www | The name of the frontend or listener |
vendor_frontend_mode | HTTP | Mode the frontend is operating (TCP or HTTP) |
Default Fields for TCP/HTTP and HTTPs Logs
Field Name | Example Value | Description |
---|---|---|
process_name | haproxy | Process name |
process_id | 14115 | Process ID |
source_ip | 10.0.1.22 | TCP port of the client that initiated the connection |
source_port | 33213 | TCP port of the client that initiated the connection |
event_received_time |
02/Feb/2023:12:12:51.222 |
Exact date when the connection was received by HAProxy |
vendor_frontend_name |
graylog |
Name of the frontend (or listener) that received and processed the connection |
vendor_backend_name |
backend1 |
Name of the backend (or listener) |
vendor_server_name |
srv1 |
Name of the last server to which the connection was sent |
vendor_tw |
1 |
Total time in milliseconds spent waiting |
vendor_tc |
2 |
Total time in milliseconds spent waiting for the connection to establish to the final server |
vendor_tt |
5007 |
Total time in milliseconds elapsed between the accept and the last close |
destination_bytes_sent |
212 |
Total number of bytes transmitted from the server to the client when the log is emitted |
captured_request_cookie |
optional field |
Optional |
captured_response_cookie |
optional field |
Optional |
vendor_termination_state |
-- or ---- |
The condition the session was in when the session ended |
vendor_actconn |
3 |
Total number of concurrent connections on the process when the session was logged |
vendor_feconn |
4 |
Total number of concurrent connections on the frontend when the session was logged |
vendor_beconn |
5 |
Total number of concurrent connections handled by the backend when the session was logged |
vendor_srv_conn |
6 |
Total number of concurrent connections still active on the server when the session was logged |
vendor_retries |
7 |
Number of connection retries experienced by this session when trying to connect to the server |
vendor_srv_queue |
8 |
Total number of requests which were processed before this one in the server queue |
vendor_backend_queue |
9 |
Total number of requests which were processed before this one in the backend's global queue |
TCP log format
haproxy[14387]: 10.0.1.22:33213 [02/Feb/2023:12:12:51.222] graylog backend1/srv1 1/2/5007 212 -- 3/4/5/6/+7 8/9
HTTP/HTTPS format
haproxy[14389]: 10.0.1.2:33317 [06/Feb/2023:12:14:14.655] https-in graylog/srv1 10/0/30/69/109 200 27250 cookie - C--- 10/1/1/1/0 20/20 {1graylog.de} {fun.de} "GET /index.html HTTP/1.1" 20/21/22/23/24 grt.de/TLSv1.3/TLS_AES_256_GCM_SHA384
Additional fields for HTTP/HTTPS logs:
Field Name | Example Value | Description |
---|---|---|
vendor_trr | 69 | Total time in milliseconds spent waiting for the server to send a full HTTP response |
vendor_ta |
109 |
Total time the request remained active in HAProxy |
http_response_code |
200 |
HTTP status code returned to the client |
http_request_cookie |
cookie |
An optional |
http_response_cookie |
- |
An optional |
vendor_captured_request_headers |
1graylog.de |
List of headers captured in the request due to the presence of the "capture request header" statement |
vendor_captured_response_headers |
fun.de |
A list of headers captured in the response due to the presence of the "capture response header" statement |
http_request_method |
GET |
HTTP request method |
http_request_path |
/index.html |
HTTP request path |
http_version |
1.1 |
HTTP version |
vendor_fc_err |
20 |
Status of the connection on the frontend's side |
vendor_ssl_fc_err |
21 |
Last error of the first SSL error stack that was raised on the connection from the frontend's perspective |
vendor_ssl_c_err |
22 |
Status of the client's certificate verification process |
vendor_ssl_c_ca_err |
23 |
Status of the client's certificate chain verification process |
vendor_ssl_fc_is_resumed |
24 |
True if the incoming TLS session was resumed with the stateful cache or a stateless ticket |
vendor_ssl_fc_sni |
grt.de |
SNI (Server Name Indication) presented by the client to select the certificate to be used |
vendor_ssl-version |
TLSv1.3 |
SSL version of the frontend |
vendor_ssl_ciphers |
TLS_AES_256_GCM_SHA38 |
SSL cipher used for the connection |
HAProxy Spotlight Content Pack
HAProxy offers five dashboards: a general overview dashboard, a dashboard for default logs, one for TCP logs, one for HTTP and HTTPS logs, and an error dashboard.
Overview Dashboard
Default Logs Dashboard
TCP Logs Dashboard
HTTP and HTTPS Logs Dashboard
Error Dashboard