Ingest from Files

Log files come in many different formats, which can make them difficult to handle.

That is why Graylog does not collect files directly. Instead, it uses a wide range of collectors and agents designed just for this purpose. Collectors can be configured and controlled by the already available configuration management software in the environment. They can be configured by the Graylog Sidecar (after it is installed and configured). Or you could just do it manually.

Of course, you could use any program supporting the GELF or Syslog protocols (among others) to send your logs to Graylog.

A highly recommended way to pick a log file from Windows or Linux systems is filebeat. This collector is built to collect log files and ship them to a central location. The output module in filebeat is called logstash. It is needed to send messages to a Graylog beats input.

A basic filebeat configuration for filebeat on Linux may look like the following:

fields_under_root: true
fields.collector_node_id: ${sidecar.nodeName}
fields.gl2_source_collector: ${sidecar.nodeId}

filebeat.inputs:
- input_type: log
  paths:
    - /var/log/*.log
  type: log
output.logstash:
   hosts: ["graylog:5044"]
path:
  data: /var/lib/graylog-sidecar/collectors/filebeat/data
  logs: /var/lib/graylog-sidecar/collectors/filebeat/log

For Windows, the basic filebeat configuration may look like this:

fields_under_root: true
fields.collector_node_id: ${sidecar.nodeName}
fields.gl2_source_collector: ${sidecar.nodeId}

output.logstash:
   hosts: ["graylog:5044"]
path:
  data: C:\Program Files\Graylog\sidecar\cache\filebeat\data
  logs: C:\Program Files\Graylog\sidecar\logs
tags:
 - windows
filebeat.inputs:
  type: log
  enabled: true
  paths:
    - C:\logs\log.log