Zeek is an open-source network security monitoring tool capable of passively analyzing network traffic and generating a number of logs describing the activity it sees. Zeek can be used to detect attacks and probes with a variety of different analyzers and scripts. This technology pack will process Zeek log messages by providing normalization and enrichment of common events of interest.

Requirement(s)

  • Zeek 5.0.9+
  • Filebeat 7.13+ (ndjson parser)
  • Graylog Server with a valid enterprise license, running Graylog version 5.0.3 or later
  • Configured Beats input

Graylog Server Requirement

This spotlight requires a more recent version of Graylog than the other spotlights due to a change in how some of the extraction functions work. Fields will be improperly processed if using a version of Graylog Server earlier than version 5.0.

Not Supported

  • N/A

Stream Configuration

This technology pack includes one stream:

  • “Illuminate:Zeek Messages”

About Illuminate Streams

If this stream does not exist prior to the activation of this pack then it will be created and configured to route messages to this stream and the associated index set. There should not be any stream rules configured for this stream.

Index Set Configuration

This technology pack includes one index set definition:

  • “Zeek Logs”

About Illuminate Index Set Definitions

If this index set is already defined, then nothing will be changed. If this index set does not exist, then it will be created with retention settings of a daily rotation and 90 days of retention. These settings can be adjusted as required after installation.

Log Format Example

{"filebeat_id_orig_p":58405,"filebeat_log_offset":908201,"filebeat_agent_name":"snort1","filebeat_history":"ShADadFf","filebeat_resp_pkts":7,"filebeat_id_orig_h":"74.96.110.140","filebeat_event_source_product":"zeek","beats_type":"filebeat","filebeat_@metadata_beat":"filebeat","filebeat_ts":"2023-06-05T14:15:05.682220Z","filebeat_@timestamp":"2023-06-05T14:15:23.109Z","filebeat_agent_type":"filebeat","filebeat_@metadata_version":"8.6.2","filebeat_host_name":"snort1","filebeat_proto":"tcp","filebeat_agent_version":"8.6.2","filebeat_orig_ip_bytes":1234,"filebeat_agent_ephemeral_id":"6a114e83-ca97-4925-b4c6-8971c97d191b","filebeat_local_orig":true,"filebeat_orig_bytes":942,"filebeat_input_type":"filestream","filebeat_id_resp_h":"172.65.251.78","filebeat_orig_pkts":7,"filebeat_resp_bytes":1960,"filebeat_duration":10.197248935699463,"message":"-","filebeat_ecs_version":"8.0.0","filebeat_id_resp_p":443,"filebeat_uid":"CyRQBS1i6zJKpZXgHl","filebeat_service":"ssl","filebeat_missed_bytes":0,"filebeat_@metadata_type":"_doc","filebeat_conn_state":"SF","filebeat_resp_ip_bytes":2252,"filebeat_local_resp":false,"filebeat_agent_id":"c2c08007-50ba-4d8e-948b-ca484ef0ece2","filebeat_log_file_path":"/opt/zeek5/logs/current/conn.log","host":"bg-devel","level":6,"replayed_log":"true"}

Requirements

Zeek

Zeek must be configured to log in JSON format in order to use this technology pack. Any existing logging configuration can still be used separately but this technology pack requires the JSON configuration below to capture detailed event information. The essential settings and file locations are below.

The ISO8601 setting is crucial for proper timestamp processing.

Additional information regarding Zeek installation and usage can be found on the Zeek documentation page which is extremely detailed.

Zeek Configuration

json-logs.zeek - (/zeek_install_path/share/zeek/policy/tuning/)

Copy 
##! Loading this script will cause all logs to be written
##! out as JSON by default.

redef LogAscii::use_json=T;
redef LogAscii::json_timestamps = JSON::TS_ISO8601;
redef LogAscii::json_include_unset_fields=T;

local.zeek - (/zeek_install_path/share/zeek/site/)

Copy 
@load packages
@load policy/tuning/json-logs.zeek

Filebeat

Because Zeek is configured to log in JSON format, a delivery system that can parse each field and deliver them to Graylog is required. This technology pack supports log delivery via Filebeat with a specific configuration of inputs and outputs, identifying logs for parsing and delivery to Graylog for Illuminate processing.

The paths, fields, and parsers sections are required for this configuration. If the Zeek logs are being stored in a location other than below, these paths must be updated. The event_source_product: zeek and target: "zeek" fields are critical for proper identification and parsing of logs.

Filebeat Configuration

Additional information regarding the installation and configuration of Filebeat can be found below:

filebeat.yml

Copy 
- type: filestream

  id: zeek-filestream

  enabled: true

  paths:
    - /zeek_install_path/logs/current/*.log
  
  parsers:
    - ndjson:
        target: "zeek"
        add_error_key: true
        overwrite_keys: true

  fields:
    event_source_product: zeek
    fields_under_root: true

What is Provided

  • Rules to normalize and enrich Zeek log messages

Zeek Log Message Processing

The Illuminate processing of Zeek log messages provides the following:

  • Field extraction, normalization, and message enrichment for Zeek log messages (IDS and AppID)
  • GIM Categorization of the following messages:
Zeek Log Type GIM Category GIM Subcategory
http http http.communication
conn network network.flow
conn network network.default
dns name resolution name resolution.dns request
dns name resolution name resolution.dns request, name resolution.dns answer

Zeek Spotlight Content Pack

The Zeek Spotlight content pack contains:

  • Dashboard: Illuminate:Zeek Overview