Packetbeat Content Pack

The following content pack is available for use with a Graylog Illuminate license and Graylog Enterprise or Graylog Security. Contact sales to learn more about obtaining Illuminate.

Packetbeat is an open-source network packet analyzer that captures and analyzes network traffic in real-time. It provides insights into application performance, network latency, and security threats by monitoring communication between servers and applications. Packetbeat is designed to be lightweight and easy to deploy, making it a valuable tool for network monitoring and troubleshooting.

Supported Version(s)

Packetbeat 8.x (tested with 8.12.2)

Requirements

Packetbeat 8.x
Graylog 7.0.3+ with a valid Graylog Enterprise or Graylog Security license

Stream Configuration

This technology pack includes 1 stream:

"Illuminate:Packetbeat Messages"

Hint: If this stream does not exist prior to the activation of this pack then it will be created and configured to route messages to this stream and the associated index set. There should not be any stream rules configured for this stream.

Index Set Configuration

This technology pack includes 1 index set definition:

"Packetbeat Event Log Messages"

Hint: If this index set is already defined, then nothing will be changed. If this index set does not exist, then it will be created with retention settings of a daily rotation and 90 days of retention. These settings can be adjusted as required after installation.

Log Collection

This pack parses logs from the following sources:

Sending logs via Beats Input

Packetbeat Configuration Example

To successfully ship logs to Graylog, ensure you add the IP address of your Graylog instance within your packetbeat.yml file. For example:

Copy

# ------------------------------ Logstash Output
                    -------------------------------
                    output.logstash:
                    # The Logstash hosts
                    hosts: ["192.168.x.x:5044"]

Log Format Example

{"timestamp":1707945509.782,"version":"1.1","host":"ip-172-31-71-138.ec2.internal","short_message":"-","_packetbeat_host_os_version":"2","_packetbeat_method":"QUERY","_packetbeat_related_ip":"[172.31.71.138, 172.31.0.2, 3.218.180.17]","_packetbeat_type":"dns","_packetbeat_cloud_instance_id":"i-01cd7cd147686ddca","_packetbeat_event_type":"[connection, protocol]","_packetbeat_query":"class IN, type A, dynamodb.us-east-1.amazonaws.com","_packetbeat_dns_question_etld_plus_one":"dynamodb.us-east-1.amazonaws.com","_packetbeat_dns_answers_count":1,"_packetbeat_host_containerized":"false","_packetbeat_dns_flags_authentic_data":"false","_packetbeat_destination_ip":"172.31.0.2","_packetbeat_dns_flags_truncated_response":"false","_packetbeat_host_os_kernel":"5.10.209-198.812.amzn2.x86_64","_packetbeat_event_end":"2024-02-14T21:18:29.784Z","_packetbeat_@metadata_beat":"packetbeat","_packetbeat_agent_name":"ip-172-31-71-138.ec2.internal","_packetbeat_dns_flags_authoritative":"false","_packetbeat_status":"OK","_packetbeat_agent_version":"8.12.1","_packetbeat_dns_answers_0_name":"dynamodb.us-east-1.amazonaws.com","_packetbeat_dns_flags_recursion_desired":"true","_packetbeat_network_type":"ipv4","_packetbeat_dns_authorities_count":0,"_packetbeat_dn_header_flags":"[RD, RA]","_packetbeat_resource":"dynamodb.us-east-1.amazonaws.com","_packetbeat_server_ip":"172.31.0.2","_packetbeat_@timestamp":"2024-02-14T21:18:29.782Z","_packetbeat_agent_ephemeral_id":"1a6a4861-3672-4794-ad48-fa7f6ce9e17c","_packetbeat_cloud_provider":"aws","_packetbeat_event_start":"2024-02-14T21:18:29.782Z","_packetbeat_host_os_codename":"Karoo","_packetbeat_dns_answers_0_ttl":"4","_packetbeat_host_mac":"[06-FB-50-7E-81-B1]","_packetbeat_dns_question_type":"A","_packetbeat_server_port":53,"_packetbeat_destination_port":53,"_packetbeat_client_bytes":50,"_packetbeat_event_category":"[network]","_packetbeat_dns_id":2180,"_packetbeat_dns_question_registered_domain":"dynamodb.us-east-1.amazonaws.com","_packetbeat_dns_resolved_ip":"[3.218.180.17]","_packetbeat_destination_bytes":66,"_packetbeat_host_os_platform":"amzn","_packetbeat_network_community_id":"1:n2oEH592UySL71Gm21GwZS2Jiwo=","_packetbeat_network_bytes":116,"_packetbeat_dns_answers_0_data":"3.218.180.17","_packetbeat_source_ip":"172.31.71.138","_packetbeat_cloud_availability_zone":"us-east-1e","_packetbeat_dns_flags_recursion_available":"true","_packetbeat_host_os_type":"linux","_packetbeat_dns_question_name":"dynamodb.us-east-1.amazonaws.com","_packetbeatdns_question_top_level_domain":"us-east-1.amazonaws.com","_packetbeat_ecs_version":"8.0.0","_packetbeat_host_name":"ip-172-31-71-138.ec2.internal","_packetbeat_dns_additionals_count":0,"_packetbeat_dns_answers_0_type":"A","_packetbeat_network_direction":"egress","_packetbeat_cloud_machine_type":"t2.micro","_packetbeat_host_ip":"[172.31.71.138, fe80::4fb:50ff:fe7e:81b1]","_packetbeat_cloud_region":"us-east-1","_packetbeat_network_transport":"udp","_packetbeat_dns_response_code":"NOERROR","_packetbeat_client_ip":"172.31.71.138","_packetbeat_cloud_account_id":"335344888946","_packetbeat_agent_id":"0d245eab-3205-4f95-8b23-2878569f4dbd","_packetbeat_event_kind":"event","_packetbeat_dns_question_class":"IN","_packetbeat_dns_answers_0_class":"IN","_packetbeat_host_id":"0e7b0d07010a489ba9e1a74d38cc23de","_packetbeat_source_port":46704,"_packetbeat_client_port":46704,"_packetbeat_dns_type":"answer","_packetbeat_source_bytes":50,"_packetbeat_dns_op_code":"QUERY","_packetbeat_server_bytes":66,"_packetbeat_host_architecture":"x86_64","_packetbeat_network_protocol":"dns","_packetbeat_dns_flags_checking_disabled":"false","_packetbeat_host_os_name":"Amazon Linux","_packetbeat_@metadata_type":"_doc","_packetbeat_agent_type":"packetbeat","_packetbeat_cloud_service_name":"EC2","_packetbeat_event_dataset":"dns","_packetbeat_event_duration":1717000,"_packetbeat_host_os_family":"redhat","_packetbeat_@metadata_version":"8.12.1","_packetbeat_host_hostname":"ip-172-31-71-138.ec2.internal","_packetbeat_cloud_image_id":"ami-0aa7d40eeae50c9a9"}

What is Provided

Parsing rules to normalize and enrich Packetbeat log messages
Graylog Information Model message categorization for DNS, HTTP, and Flow events
Security Core support for name resolution, network, and HTTP event categories
DNS responses are categorized, but may generate GIM errors if they do not include IPs in their responses

Events Processed by This Technology Pack

The Packetbeat content pack supports parsing for all fields, and GIM categorization for DNS, Flow, and HTTP events.

GIM Categorization

GIM categorization is provided for the following messages:

vendor_subtype	gim_event_type_code	gim_event_category	gim_event_subcategory	gim_event_type
dns (query)	140000	name resolution	name resolution.dns request	dns query
dns (answer)	140200	name resolution	name resolution.dns answer	dns response
flow	120500	network	network.flow	flow record
http	180200	http	http.communication	http communication

Events Renamed by This Technology Pack

Field Mappings

Field Mappings ▼

vendor_field	graylog_schema	Field Description
packetbeat_event_dataset	vendor_subtype	Identifies the vendor event type or category.
packetbeat_destination_bytes	destination_bytes_sent	Total byte count sent to the destination.
packetbeat_destination_ip	destination_ip	IP address of the event destination.
packetbeat_destination_packets	destination_packets_sent	Count of packets sent to the destination.
packetbeat_destination_port	destination_port	Network port used at the destination.
packetbeat_host_hostname	host_hostname	The network name of the host machine.
packetbeat_host_id	host_id	A unique identifier for the host machine.
packetbeat_host_mac	host_mac_list	List of MAC addresses associated with the host.
-	host_mac	First MAC address from host_mac_list.
packetbeat_host_ip	host_ip_list	List of IP addresses assigned to the host.
-	host_ip	First IP address from host_ip_list.
packetbeat_network_bytes	network_bytes	Cumulative bytes transferred in a network session.
packetbeat_network_direction	network_direction	Directionality of the network traffic (inbound/outbound).
packetbeat_network_transport	network_transport	Protocol used for data transport (TCP/UDP).
packetbeat_network_type	network_protocol	Protocol governing the network communication.
packetbeat_source_ip	source_ip	Source IP address from where the event originated.
packetbeat_source_port	source_port	Port number at the source.
packetbeat_source_bytes	source_bytes_sent	Byte count sent from the source.
packetbeat_source_packets	source_packets_sent	Number of packets originated from the source.
packetbeat_network_packets	network_packets	Total packets exchanged in the network session.
packetbeat_dns_question_type	query_record_type	Type of DNS query record requested.
packetbeat_dns_question_class	query_class	Class of DNS query (e.g., IN for Internet).
packetbeat_dns_question_name	query_request	The domain name requested in the DNS query.
packetbeat_dns_response_code	query_result	Response code returned by the DNS query.
packetbeat_dns_resolved_ip	query_response	IP address response from a DNS query.
packetbeat_http_request_bytes	http_request_bytes	Size of the HTTP request in bytes.
packetbeat_http_request_method	http_request_method	Method used in the HTTP request (GET, POST, etc.).
packetbeat_http_response_bytes	http_response_bytes	Byte size of the HTTP response.
packetbeat_http_response_headers_content-type	http_content_type	MIME type of the content in the HTTP response.
packetbeat_http_response_status_code	http_response_code	Status code indicating the result of the HTTP request.
packetbeat_http_version	http_version	The HTTP protocol version used in the communication.
packetbeat_user_agent_original	http_user_agent	The raw user agent string identifying client software.
packetbeat_url_path	http_request_path	Request URI path.
packetbeat_url_domain	http_host	Request host domain.
packetbeat_url_scheme	http_uri_scheme	URI scheme (http or https).

Packetbeat Spotlight Content Pack

Packetbeat offers a dashboard with 3 tabs - An overview tab, a network flow tab, and a tab for an overview of http: