Plugins

Graylog offers various extension points to customize and extend its functionality through writing Java code.

The first step for writing a plugin is creating a skeleton that is the same for each type of plugin. The following explains how to do this and reviews plugin types in detail.

Plugin Types

Graylog comes with a stable plugin API for the following plugin types:

• Inputs: Accept/write any messages into Graylog
• Outputs: Forward ingested messages to other systems as they are processed
• Services: Run at startup and able to implement any functionality
• Event Notifications: Called when an event alert has been triggered
• Processors: Transform/drop incoming messages (can create multiple new messages)
• Filters: (Deprecated) Transform/drop incoming messages during processing
• REST API Resources: An resource exposed as part of the Graylog REST API
• Periodical: Called at periodical intervals during server runtime
• Decorators: Used during search time to modify the presentation of messages
• Authentication Realms : Allowing to implement different authentication mechanisms (like single sign-on or 2FA)

Write Your Own Plugins

What you need in your development environment before starting is:

• git
• maven
• rpm

If you plan to write a web plugin, you’ll also need:

• node
• yarn

There are lots of different ways to get those on your local machine, unfortunately we cannot list all of them, so please refer to your operating system-specific documentation,

Graylog uses a couple of conventions and techniques in its code, so be sure to read about the API concepts for an overview.

Sample Plugin

In the following sections we will create a plugin skeleton based on a maven archetype. The skeleton is similar to the sample plugin on Github.This documentation will link to specific parts for your reference. It is fully functional, even though it does not implement any useful functionality. Its purpose is to provide a reference for helping to implement your own plugins.

Create a Plugin Skeleton

The easiest way to get started is to use our Graylog meta project, which will create a complete plugin project infrastructure with all required classes, build definitions, and configurations. Using the meta project allows you to have the Graylog server project and your own plugins (or 3rd party plugins) in the same project, which means that you can run and debug everything in your favorite IDE or navigate seamlessly in the code base.

Maven is a widely used build tool for Java, that comes pre-installed on many operating systems or can be installed using most package managers. Make sure that you have at least version 3 before you go on.

First you should install the latest version of the graylog-project-cli on your workstation. Use this to bootstrap the meta project in your working directory with the following command:

graylog-project bootstrap github://Graylog2/graylog-project.git

This will create a checkout of the meta project in your current directory. You’ll see both graylog-project and graylog-project-repos. Those two directories contain the meta data and all repositories that are required. The directory graylog-project-repos will also be the home of your new plugin.

Now you can bootstrap the plugin you want to write, by running the following command, inside the graylog-project-repos directory:

mvn archetype:generate -DarchetypeGroupId=org.graylog -DarchetypeArtifactId=graylog-plugin-archetype

It will ask you a few questions about the plugin you are planning to build. Let’s say you work for a company called ACMECorp and want to build an alarm callback plugin that creates a JIRA ticket for each alarm that is triggered:

groupId: com.acmecorp
artifactId: graylog-plugin-jira-alarmcallback
version: 1.0.0
package: com.acmecorp
githubRepo: exampleGithubRepo
pluginClassName: JiraAlarmCallback

Note that you do not have to tell the archetype wizard what kind of plugin you want to build because it is creating the generic plugin skeleton for you, and nothing that is related to the actual implementation. More on this in the example plugin chapters later. It is important that your artifactId has the prefix graylog-plugin-.The githubRepo must be the desired plugins repo name, not the full GitHub URL. The repository is not required for the development, but a common part of the plugins meta information.

You now have a new folder called graylog-plugin-jira-alarmcallback, which includes a complete plugin skeleton including Maven build files. To be able to make a complete build of the project, you need to add the newly created plugin to the graylog-project POM as a module. Open pom.xml(residing in your graylog-project directory) and find a couple of <module> statements in the file. Add the following line(after adapting it to your naming):

<module>../graylog-project-repos/graylog-plugin-jira-alarmcallback</module>

Make sure to update the graylog-plugin-web-parent version inside the plugins pom.xml. You can find the current version inside the related relativePath property. The last necessary step, to get started with the development, is to execute mvn compile inside the graylog-project dir.

You should be finished now, and every Java IDE out there can now import the project automatically without any required further configuration.

In IntelliJ IDEA for example you can just use the File > Open dialog to open the graylog-project directory as a fully configured Java project, which should include the Graylog server and your plugin as submodules.

Please pay close attention to the README file of the Graylog meta project and follow any further instructions listed there to set up your IDE properly.

If you want to continue working on the command line, you can do the following to compile the server and your plugin:

mvn package

The Anatomy of a Plugin

Each plugin contains information to describe itself and register the extensions it contains.

Required Classes

At the very minimum you need to implement two interfaces:

• org.graylog2.plugin.Plugin This is the entry to your plugin code.

• org.graylog2.plugin.PluginMetaData This describes your plugin.

The bootstrap-plugin script generates these implementations for you, and you simply need to fill out the details.

Graylog uses Java’s ServiceLoader mechanism to find your plugin’s main class, so if you rename your Plugin implementation, you need to also adjust the service file. Please also see Google Guava’s AutoService which Graylog uses in conjunction with the plain ServiceLoader.

In addition to the service, Graylog needs an additional resource file called graylog-plugin.properties in a special location. This file contains information about the plugin, specifically which class loader the plugin needs to be in, so it needs to be read before the plugin is actually loaded. Typically you can simply take the default that has been generated for you .

Register Your Extension

So far the plugin itself does not do anything, because it neither implements any of the available extensions, nor could Graylog know which ones are available from your code.

Graylog uses dependency injection to wire up its internal components as well as the plugins. Thus the extensions a plugin provides need to be exposed as a PluginModule which provides you with a lot of helper methods to register the various available extensions to cut down the boiler plate code you have to write.

An empty module is created for you.

Please refer to the available Plugin Types for detailed information what you can implement. The Sample Plugin contains stub implementations for each of the supported extensions.

Create Web Plugins

Sometimes your plugin is not only supposed to work under the hoods inside a Graylog server as an input, output, alarm callback, etc. but you also want to contribute previously non existing functionality to Graylog’s web interface. Since version 2.0 this is now possible. When using the most recent Graylog meta project to bootstrap the plugin skeleton, you are already good to go for this. Otherwise please see our chapter about creating a plugin skeleton.

The Graylog web interface is written in JavaScript, based on React. It is built using webpack, which is bundling all JavaScript code (and other files you use, like stylesheets, fonts, images, even audio or video files if you need them) into chunks digestible by your browser and npm, which is managing our external (and own) dependencies. During the build process all of this will be bundled and included in the jar file of your plugin.

This might be overwhelming at first if you are not accustomed to JS-development, but fortunately we have set up a lot to make writing plugins easier for you!

If you use our proposed way for creating a plugin skeleton, and followed the part about Writing Plugins, you are already good to go for building a plugin with a web part. All you need is a running Graylog server on your machine. Everything else is fetched at build time!

Getting up and running with a web development environment is as easy as this (assuming you have node & yarn installed):

cd graylog2-server/graylog2-web-interface
yarn install
[...]
yarn start
[...]
open http://localhost:8080

This starts the development web server. It even tries to open a browser window going to it (probably working on Mac OS X only).

If your Graylog server is not running on https://localhost:9000/api/, then you need to edit graylog2-server/graylog2-web-interface/config.js (in your graylog-project directory) and adapt the gl2ServerUrl parameter.

Web Plugin Structure

These are the relevant files and directories in your plugin directory for the web part of it:

webpack.config.js
This is the configuration file for the webpack module bundler. Most of it is already preconfigured by our PluginWebpackConfig class, so the file is very small. You can override/extend every configuration option by passing a webpack snippet though.

build.config.js.sample
In this file you can customize some of the parameters of the build. There is one mandatory parameter named web_src_path which defines the absolute or relative location to a checkout of the Graylog source repository.

package.json
This is a standard npm JSON file describing the web part of your plugin, especially its dependencies. You can read more about its format.

src/web
This is where the actual code for the web part of your plugin goes to. For the start there is a simple index.jsx file, which shows you how to register your plugin and the parts it provides with the Graylog web interface. We will get to this in detail later.

Required Conventions for Web Plugins

Plugin Entrypoint

There is a single file which is the entry point of your plugin, which means that the execution of your plugin starts there. By convention this is src/web/index.jsx. You can rename/move this file, you just have to adapt your webpack configuration to reflect this change, but it is not recommended.

In any case, this file needs to contain the following code at the very top:

// eslint-disable-next-line no-unused-vars
import webpackEntry from 'webpack-entry';

This part is responsible to include and execute the webpack-entry file, which is responsible to set up webpack to use the correct URL format when loading assets for this plugin. If you leave this out, erratic behavior will be the result.

Link to other pages from your plugin

If you want to generate links from the web frontend to other pages of your plugin or the main web interface, you need to use the Routes.pluginRoute()helper method to generate the URLs properly.

See this file for more information.

Best practices for Web Plugin Development

Use ESLint

ESLint is a good tool for linting JavaScript code. It makes sure that any written code is in line with general best practices and the project-specific coding style/guideline. We at Graylog are striving to make the best use of this tool to help our developers and you to generate top quality code with few bugs. Therefore we highly recommend you enable it for any Graylog plugin you are writing.

Code Splitting

Both the web interface and plugins for it depend on a number of libraries like React, RefluxJS, and others. To prevent those getting bundled into both the web interface and plugin assets, therefore wasting space or causing problems (especially React does not like to be present more than once), we extract those into a commons chunk which is reused by the web interface and plugins.

This has no consequences for you as a plugin author, because the configuration to make use of this is already generated for you when using the meta project or the maven archetype. But here are some details about it:

Common libraries are built into a separate vendor bundle using an own configuration file named webpack.vendor.js. Using the DLLPlugin a manifest is extracted which allow us to reuse the generated bundle. This is then imported in our main web interface webpack configuration file and the corresponding generated webpack config file for plugins.

Build Plugins

Building the plugin is easy because the meta project has created all necessary files and settings for you. Just run mvn package either from the meta project’s directory graylog-project (to build the server and the plugin) or from the plugin directory (to build the plugin only):

mvn package

This will generate a .jar file in target/.
That is the complete plugin file:

ls target/jira-alarmcallback-1.0.0-SNAPSHOT.jar
target/jira-alarmcallback-1.0.0-SNAPSHOT.jar

Install and Load Plugins

The only thing you need to do to run the plugin in Graylog is to copy the .jar file to your plugins folder that is configured in your graylog.conf. The default is just plugin/relative from your graylog-server directory.

This is a list of default plugin locations for the different installation methods.

Plugin Installation Locations

Restart graylog-server and the plugin should be available to use from the web interface immediately.