Manage Collector Fleets

In Graylog 7.2, Graylog Collector has been released as a preview feature for evaluation and feedback purposes. This feature's design and behavior could change significantly in future releases. To share feedback on your experience with Graylog Collector, email feedback@graylog.com.

A fleet is a group of collectors that share one configuration, and that configuration is a set of collector sources defining what to collect. Adding or changing a source reaches every collector in the fleet within seconds. This article explains how to create fleets, add sources to them, and change what your collectors collect.

Prerequisites

Before proceeding, ensure that the following prerequisites are met:

• You must be a Graylog administrator or have the Collectors Manager role to manage fleets. A user with the Collectors Reader role can view fleets and their sources but cannot add or change them.

Create a Fleet

Plan fleets around what hosts should collect rather than where they are, so that hosts needing the same sources belong together. For information about how fleets and source interact, see Graylog Collector.

To create a fleet, go to System > Collectors, then select the Fleets tab:

1. Select Create fleet.

2. In the New Fleet dialog, enter a unique Name and an optional Description.

3. Select Create fleet.

A new fleet has no sources and collects nothing until you add them.

Add a Source to a Fleet

A source defines what a collector reads. That is, a source identifies the log data you want to collect from a host. A fleet can hold any number of sources.

To add a source, go to System > Collectors, then select the Fleets tab:

1. Select the fleet.

2. On the Sources tab, select Add Source.

3. Enter a Name and an optional Description.

4. Leave Enabled selected, or clear it to define the source without collecting from it.

5. Select a Source Type and complete its options.

   For File ▼

   Collects from log files on disk, on any platform.

   Configuration Option	Description
   File Path(s)	Sets the paths to read. Wildcard patterns are supported, for example /var/log/*.log or /var/log/**/*.log. The collector watches matched files for new lines and picks up files that appear later.
   Read Mode	Sets where collection starts. From end collects only lines written after the collector starts. From beginning collects the existing contents of each file first, then continues reading. Choose From beginning deliberately. On a host with large existing log files, the collector sends their full contents to Graylog on first start.

   Warning: On Linux, the collector cannot read files under /home, /root, or /run/user. File sources pointing at those directories collect nothing and report no error, so place collected logs elsewhere.

   For Journald ▼

   Collects from the systemd journal. Linux only.

   Configuration Option	Description
   Read Mode	Sets where collection starts. From end collects only entries written after the collector starts. From beginning collects existing journal entries first. Choose From beginning deliberately. On a host with a long journal history, the collector sends the whole journal to Graylog on first start.
   Priority	Sets the minimum severity to collect, from Emergency through Debug. Selecting a level also collects everything more severe, so selecting Error also collects Critical, Alert, and Emergency.
   Match Pattern (optional)	Restricts collection to a unit or process, using a journald match expression. For example, _SYSTEMD_UNIT=nginx.service.

   For Windows Event Log ▼

   Collects from Windows Event Log channels. Windows only.

   Configuration Option	Description
   Channels	Sets the channels to read, as a comma-separated list. For example, Application, Security, System.
   Include default channels	Collects a standard set in addition to any named above, if enabled. The set is Application, System, Security, Setup, Microsoft-Windows-Windows Defender/Operational, Microsoft-Windows-PowerShell/Operational, and Windows PowerShell.
   Read Mode	Sets where collection starts. From end collects only events recorded after the collector starts. From beginning collects existing events in the channel first. Choose From beginning deliberately. On a host with a large event log, the collector sends the entire channel to Graylog on first start.

   For macOS Unified Logging ▼

   Collects from the macOS unified logging system. macOS only.

   Configuration Option	Description
   Predicate	Filters what to collect, using the same syntax as the macOS log command. Arrives pre-filled with a predicate limited to security-relevant subsystems at error level or above, covering authentication, login, privacy and security policy, firewall and network extensions, disk and storage management, and process launching.
   Max poll interval	Sets how often the collector checks for new entries, with a minimum of one second. Defaults to 30 seconds.
   Max log age	Sets how far back to collect existing entries on first start. Defaults to 0, which collects only entries written after the collector starts.

   Warning: Clearing Predicate collects everything the unified logging system emits, which is a very large volume on a typical macOS host. Narrow the supplied predicate or replace it with one of your own, but do not leave the field empty.

   The unified logging system is the only source type that arrives filtered, because it is far more verbose than the others. Info and debug entries are never collected regardless of the predicate.

6. Select Create source.

The new source reaches every collector in the fleet within seconds. No restart is required.

Some source types apply to one platform only. If a fleet contains hosts running different operating systems, Graylog sends each source only to the hosts that support it. For example, a fleet holding a journald source and a Windows Event Log source delivers the journald source to its Linux hosts and the event log source to its Windows hosts.

Edit a Source

To edit a source, go to System > Collectors, then select the Fleets tab:

1. Select the fleet.

2. On the Sources tab, select Edit on the source's row.

3. Change the options you need.

4. Select Update source.

You cannot change a source's type after creating it, because Source Type is read-only when editing. To switch types, delete the source and add a new one.

To pause collection without losing the definition, clear Enabled and update the source. Use this option to quiet a noisy source temporarily. Edit the source and select Enabled again to resume collection.

Delete a Source

To delete a source, go to System > Collectors, then select the Fleets tab:

1. Select the fleet.

2. On the Sources tab, select More on the source's row.

3. Select Delete.

Deleting a source removes it from the fleet permanently and stops its collection on every collector. To stop collection temporarily, disable the source instead. A source belongs to one fleet and cannot be moved. To collect the same data in another fleet, create a matching source there.

Delete a Fleet

You cannot delete a fleet while collectors are assigned to it. Reassign those collectors to another fleet or delete them first. Use delete if all collectors have been uninstalled from their hosts. Otherwise, a collector that is still running reappears. See Manage Collectors.

To delete a fleet, go to System > Collectors, then select the Fleets tab:

1. Select More on the fleet's row.

2. Select Delete.

Use Case

A team runs a group of Ubuntu web servers and wants both authentication activity and web traffic from all of them.

They create a fleet named linux-web, then add a Journald source with Priority set to Info and Match Pattern set to _SYSTEMD_UNIT=ssh.service, and a File source with File Path(s) set to /var/log/nginx/*.log and Read Mode set to From end. They generate an enrollment token scoped to linux-web and install collectors on each host.

Every host enrolled with that token collects both sources. Adding a third source later applies to all of them without anyone touching a host again.

Troubleshooting and Common Issues

The following section outlines troubleshooting steps for common issues to assist you in resolving potential challenges you may encounter.

Source Collects Nothing

Confirm the source is enabled and that the fleet is the one the collectors actually joined, then check the cause for its type.

Source type	Common cause
File	The path matches no files. Check the pattern on the host itself, because the collector matches it directly against the file system. On Linux, paths under /home, /root, or /run/user are unreadable by the service and fail silently.
Journald	Priority is set more severe than the entries being written, or Match Pattern matches no unit.
Windows Event Log	A channel name is misspelled, or the channel is empty.
macOS Unified Logging	The default predicate is deliberately narrow, so a quiet host legitimately produces nothing. Widen it to confirm the source works, then narrow it again. A predicate matching only info or debug entries also yields nothing, because those levels are never collected.

Read Mode set to From end means nothing is collected until new entries are written. On a quiet host, that could be indistinguishable from a broken source.

Further Reading

Explore the following additional resources and recommended readings to expand your knowledge on related topics:

• Graylog Collector

• Install Graylog Collector

• Monitor Collectors

• Manage Collectors

• Configure Collector Settings