Widgets

Graylog widgets offer a powerful way to visualize data and create meaningful dashboards. Widgets provide new perspectives on data because they primarily represent aggregated data, like a widget that displays firewall log data providing you valuable information about failed attempts to infiltrate your system within the last 24 hours. Or, for example, you may configure a widget to display the amount of incoming messages per input. This information, displayed in a graph or bar chart, can give you a better understanding of your system at a glance.

There are two types of widgets you can employ to visualize your data:

Read on to learn how widgets can help refine your search results for more targeted insights.

Aggregation Widgets

An aggregation is a collection of data that has been modified to provide an answer to a specific question. Performing aggregations on your data ensures that only desired fields, functions, or metrics are included in the results. This way, you get a clearer, more accurate portrayal of your search result. For example, as part of an investigation, you may only want to see messages that came from a specific IP within the last two days. In this case, you can aggregate your data to only show you these specific messages. This makes it easier to focus on the relevant data and can help expedite the investigation.

An aggregation widget is a tool that helps you get a clear picture of your search results via aggregation methods and visualization types. Aggregations can be designed to display data from a different point of view. Sometimes focusing on individual messages will not give you the answer you are looking for. In these cases, taking a broader look at your data may help you find the answers you are looking for. Aggregation widgets give you the option to modify your perspective and see your data from different angles.

You can make use of features like Show Top Values to perform data discovery, or apply a series of visualization options and make results more meaningful for human observers. Aggregation widgets are especially useful in dashboards. They can be used to compare multiple queries on the same page.

When you aggregate data, you gain insight. Aggregation widgets give you the option to group data or set metrics against a field. Additionally, you may also sort a field in ascending or descending order to reveal a clearer result. For example, if you want to see which page on your website takes the longest amount of time to render, you can aggregate on the took_ms field and sort the results in descending order to determine the answer.

Another example could be that you want to know which input receives the most log messages. You already have run a search that shows you all input messages. You can find the answer you are looking for by aggregating the results by log count.

Aggregation widgets include visualization options that make search results easier to read, and aggregations perform differently based on the visualization type selected.

Message Table Widgets

A message table widget displays messages and their fields. Below you see a message table widget that displays messages with their timestamp and source fields. These widgets allow for a more granular examination of individual messages. Message level searches are crucial for security investigations and any time you need detailed information. These widgets can be added to a dashboard for constant monitoring and reporting.

You may also see the original message by clicking on a message row. This opens the detailed view of a message with all its fields.

If you use a table widget, you can choose to aggregate all values in a column.

Create a New Widget

Both widget types are created via the sidebar menu.

To add a widget to your search or dashboard, click Create (+ ) in the sidebar. Make the following selections based on your desired widget type:

  • Empty Aggregation: You can configure empty aggregations via the Edit button.

  • Predefined Aggregation: Log View, Message Table, or Message Count.

  • Events Overview: An overview of all events and the event definitions related to the search.

  • Investigations Overview: Includes all available investigations in your search. See image below.

Configure a Widget

After you have created an empty widget, click the pen icon found in the top right corner of the widget, or click Edit to open the widget edit modal. Here you will be able to configure an aggregation based on:

  • Grouping (Group By)

  • Metrics

  • Sorting and Direction

  • Visualization

Group By

This option allows you to group your chart by rows and columns. The data points of a field will be aggregated to a selected row or column. For example, the avg() function can find the average of numeric data points of took_ms in the column.

When you create a new group using Group By, the values you select get rolled up into the result. This result can be presented in a variety of ways. You may present the data as a table, chart, or colored visualization.

For example, if the field timestamp is attributed to a row, it will divide the data points into intervals. Otherwise the aggregation will take up to 15 elements of the selected field by default and it will apply the selected metrics function to the data points.

If timestamp is aggregated with avg() on took_ms, the column action will give the average loading time for a page per action for every 5 minutes.

Metrics

Metrics are specifications or quantitative measurements that help you find answers. Metrics help you gain more detailed insight into your data by putting it in context. A metric may give you the average amount of time spent downloading a page or a comparison of sales made in several countries.

Metrics can be determined by selecting a function and a field to aggregate on from the drop down menus in the widget configuration modal. You can use metrics to obtain a numerical result. For example, sum counts bytes for all matched messages for the given time. If you have a bar chart with time, it will give you an idea how much data is pushed into storage over time. Such metrics can be used as a standard of measurement in decision making.

The Percentage Metric

The percentage metric helps to display results in the form of percentages instead of raw numbers. In some cases, percentages are easier to read and provide better insight into results. Results can be represented as percentages in bar charts, pie charts, and message tables.

To apply the percentage metric to your search results:

  1. Click the + icon in the left side bar.

  2. Select aggregation from among the options. An empty widget will appear.

  3. Click Edit and select percentage as the function.

Let's say an analyst wants to understand which controllers are receiving the most amount of calls and how much difference there is between each. They could quickly reach an opinion by viewing a comparison of the percentage of calls over controllers in a chart like the one below. Here the count metric displays the number of messages received from three different message controllers.

The Percentile Metric

The percentile metric helps to display the percentile or the relative standing of a certain value compared to the total. To apply the percentile metric to your search results:

  • Click the + icon in the left side bar.

  • Select Aggregation from among the options. An empty widget will appear.

  • Click Edit and select Percentile as the function.

You may then select the field you are looking for along with a percentile value from the drop down menu.

For example, let's say you have a web server application that reports its response times as GELF messages that are ingested by Graylog. You wish to understand your application’s 90th, 95th, and 99th percentile response times. These response times could be noted as the normal range.

Sorting and Direction

The order of result values can be configured here. Sorting defines which field the sorting should be done by and Direction configures whether it will be ascending or descending.

  • Interpolation: The area chart and line chart support different interpolation types. Interpolation is the action of deducing between two data points. You can select how you would like to interpolate by selecting one of the available interpolation types under Visualization: Linear, Step-after and Spline.

  • Event Annotations: All visualizations which can display a timeline (i.e. area chart, bar chart, line chart, scatter plot) support event annotations. Each event will be displayed as an entry on the time axis.

Visualization

A graph view makes it easier to compare large amounts of results. The visual display of search results can make it easier to understand and interpret data. Graphs can clearly display large spikes in web traffic. Such visual displays can attract attention and help analysts to notice and respond to these events rapidly.

Graylog offers a multitude of visualization types, such as: area charts, bar charts, heat maps, and world maps. Choose a visualization type based on the type of data you are working with. Note that a world map needs geographical points in the form of latitude and longitude. And if you choose to use a heat map for visualization, you need to provide the x and y values.

Value and Field Actions

Values and fields are visible in the sidebar and in Data Tables and Detail Message Rows. When you click a value or a field within a widget, you will see a drop-down menu. You can execute any displayed actions by clicking on them.

Field Actions

Various Field actions are displayed based on field type and location whenever a field name (not its value) is selected.

  • Chart: This will generate a new Widget containing a line chart where the field's average value is displayed over time. This chart can be taken as a starting point for a more defined aggregation. This is only possible in fields that are numerical.

  • Show Top Values: This action will generate a new widget containing a data table where the field values are listed in rows and the number of occurrences will be displayed next to it.

  • Statistics: Here field values are given to various statistics functions depending on field type. The result will be displayed in a Data Table Widget.

  • Add to Table: Add the field to the displayed fields of the message table where the Field Actions menu is shown.

  • Add to All Tables: Add the field to the displayed fields of all tables.

  • Remove from Table: Remove the field from the list displayed fields in this table.

  • Remove from All Tables: Remove the field from the list displayed fields in all tables.

Value Actions

Value actions produce different results depending on the type of value and where the menu is opened. The following actions can be executed.

  • Insert into View: Will open up a modal where a view can be selected. A selectable list of parameters will appear in the selected view. After choosing a parameter a new browser tab which contains the view with the value used in the parameter will appear. (This action is only available in Graylog Enterprise.)

  • Exclude from Results: Will add to the query to exclude all results where the field contains the value of the value action.

  • Add to Query: Will add NOT field:value to the query to filter the results additionally for where the field has the value of the value action.

  • Use in New Query: Will add field:value open a new view tab with a query string.

  • Show Documents for Value: This is available in Data Tables. It will display documents that were aggregated to display this value.

  • Create Extractor: This provides a short cut to create an extractor for values of type string in Message Tables.

  • Highlight this Value: This action will highlight this value for this field in all Message Tables and Data Tables.

Field Unit Values

Graylog converts field unit values to display the best outcome in a widget view. Search results can include messages with field unit values that are not easy to understand or display. Graylog formats large numbers or numbers with multiple digits to make them easier to understand. For example, if ingest_time is set to nanoseconds, Graylog converts this value to seconds to achieve a readable number. Graylog also modifies field unit values to display only one decimal place by default.

Hint: Units in a widget view can be different from the original unit. This is because Graylog converts units to the best format for a widget display.

Graylog converts field unit values to allow for an accurate comparison between fields. In some cases, a widget can include multiple fields with the same unit type (e.g. size) but different units (e.g. kilobytes and megabytes). Graylog formats these values to make sure that the comparison is made between fields with the same units.

Configure Field Unit Settings

You can configure field unit settings when creating or editing a widget. Any edits made are reflected across the widget only.

Unit type determines which units are offered. If you select Size as the field type, you are presented with sizing units such as byte, kilobyte and megabyte. The unit settings configuration option is available for most numeric fields and metric functions.

Warning: An icon representing the predefined unit is shown for internal fields such as gl2_ prefixed fields. Only modify a predefined unit if you are sure that it is incorrect or if you do not want to view unit values in the widget.

To select field unit settings:

  1. Click Edit found on the new widget display.

  2. Locate the field you want to configure.

  3. Click on the three dots icon found to the right of the field box.

  4. Select unit type and the unit in the modal that appears.

  5. Click Update preview if you wish to preview the widget.

  6. Click Update widget.

To modify field unit settings:

  1. Locate the desired widget.

  2. Click the Edit icon (pen) found in the top right corner.

  3. Locate the desired field in the widget configuration menu.

  4. Click the icon found to the right of the desired field.

  5. Select unit type and the unit in the modal that appears.

  6. Click Update preview if you wish to preview the changes you made.

  7. Click Update widget.

You can now view the converted values in the widget display.

Each selected unit type is represented with an axis. Multiple fields with the same unit type can be displayed on the same axis. If you select to view a field with unit type size and another field with unit type time, an axis is displayed for each unit type. There are four available axes: number, size, time, and percentage.

Graylog converts units for most visualization types. If you hover over a value in a widget display, you see the converted unit value. With Data Table, you can see the original field unit value when you click on the drop down arrow to the right of the converted value.

In the image below, field values are grouped by unit type. The Mode is Stack, so the two fields with the same unit type are stacked on top of each other. If you select Overlay, you see one field layered over the other.

 

Hint:The Heatmap visualization can only be used with fields that have the same unit type. There is a single scale used for this type of visualization, so different unit types cannot be distinguished.

Manage Widgets

Widgets can be duplicated or deleted by clicking the chevron in the top right corner.

You can also add widgets to a list of existing dashboards or place one in a new dashboard by selecting the Copy to Dashboard check box.

Widgets can be freely placed inside the search result grid. You can drag and drop them by clicking and holding the three lines to the left of the widget name. You can resize them by using the gray arrow in the bottom right corner. To expand a widget to full grid width, click the arrow in its top-right corner.

If you want to expand the view of aggregated data in your Log View widget, see Focus on the Widget.