Docker Installation

Docker is a set of platform-as-a-service products that use OS-level virtualization to deliver software in packages called containers. This guide walks you through setting up Graylog and Data Node with Docker Compose.

Prerequisites

You will need a recent version of Docker, at least version 20.10.10. In addition, use the following Docker image:

MongoDB: mongo
Graylog Data Node: graylog/graylog-datanode
Graylog Enterprise: graylog/graylog-enterprise OR Graylog Open: graylog/graylog

Warning: All configuration examples are created to run on a local computer. When using external servers, change GRAYLOG_HTTP_EXTERNAL_URI to your server IP and add GRAYLOG_HTTP_PUBLISH_URI.

Graylog Data Node Compose File

See the Docker documentation for best practices on using Docker Compose. The following example Docker Compose file may be used to set up Graylog and Data Node; however, please note that you may need to adjust configuration settings to best fit your environment.

Copy

version: '3'
services:
  # MongoDB: https://hub.docker.com/_/mongo/
  mongodb:
    image: "mongo:6.0.18"
    ports:
      - "27017:27017"   
    restart: "on-failure"
    networks:
      - graylog
    volumes:
      - "mongodb_data:/data/db"  
  #Graylog Data Node: https://hub.docker.com/r/graylog/graylog-datanode
  datanode:
    image: "graylog/graylog-datanode:6.1"
    hostname: "datanode"
    environment:
      GRAYLOG_DATANODE_NODE_ID_FILE: "/var/lib/graylog-datanode/node-id"
      GRAYLOG_DATANODE_PASSWORD_SECRET: "somepasswordpepper"
      GRAYLOG_DATANODE_ROOT_PASSWORD_SHA2: "8c6976e5b5410415bde908bd4dee15dfb167a9c873fc4bb8a81f6f2ab448a918"
      GRAYLOG_DATANODE_MONGODB_URI: "mongodb://mongodb:27017/graylog"
    ulimits:
      memlock:
        hard: -1
        soft: -1
      nofile:
        soft: 65536
        hard: 65536
    ports:
      - "8999:8999/tcp"   # DataNode API
      - "9200:9200/tcp"
      - "9300:9300/tcp"
    networks:
      - graylog  
    volumes:
      - "graylog-datanode:/var/lib/graylog-datanode"
    restart: "on-failure"  
  # Graylog: https://hub.docker.com/r/graylog/graylog/
  graylog:
    hostname: "server"
    image: "graylog/graylog-enterprise:6.1"
    # To install Graylog Open: "graylog/graylog:6.1"
    depends_on:
      mongodb:
        condition: "service_started"
      datanode:
        condition: "service_started"
    entrypoint: "/usr/bin/tini -- /docker-entrypoint.sh"
    environment:
      GRAYLOG_NODE_ID_FILE: "/usr/share/graylog/data/config/node-id"
      GRAYLOG_HTTP_BIND_ADDRESS: "0.0.0.0:9000"
      GRAYLOG_MONGODB_URI: "mongodb://mongodb:27017/graylog"
      # To make reporting (headless_shell) work inside a Docker container
      GRAYLOG_REPORT_DISABLE_SANDBOX: "true"
      # CHANGE ME (must be at least 16 characters)!
      GRAYLOG_PASSWORD_SECRET: "somepasswordpepper"
      # Password: "admin"
      GRAYLOG_ROOT_PASSWORD_SHA2: "8c6976e5b5410415bde908bd4dee15dfb167a9c873fc4bb8a81f6f2ab448a918"
      GRAYLOG_HTTP_EXTERNAL_URI: "http://127.0.0.1:9000/"
    ports:
      # Graylog web interface and REST API
      - "9000:9000/tcp"
      # Beats
      - "5044:5044/tcp"
      # Syslog TCP
      - "5140:5140/tcp"
      # Syslog UDP
      - "5140:5140/udp"
      # GELF TCP
      - "12201:12201/tcp"
      # GELF UDP
      - "12201:12201/udp"
      # Forwarder data
      - "13301:13301/tcp"
      # Forwarder config
      - "13302:13302/tcp"
    restart: "on-failure"
    networks:
      - graylog
    volumes:
      - "graylog_data:/usr/share/graylog/data/data"
      - "graylog_config:/usr/share/graylog/data/config"
      - "graylog_journal:/usr/share/graylog/data/journal"  
networks:
  graylog:
    driver: "bridge"
volumes:
  mongodb_data:
  graylog-datanode:
  graylog_data:
  graylog_config:
  graylog_journal:

After starting all three Docker containers by running docker-compose up, you can open the URL http://127.0.0.1:9000 in a web browser and log in with username admin and password, which you get in the console output during the start:

Copy

graylog-1   |                                                              ---
graylog-1   |                                                              ---
graylog-1   |                                                              ---
graylog-1   |     ########  ###   ######### ##########   ####         #### ---         .----               ----
graylog-1   |   ###############   ###################### #####       ####  ---      ------------       .----------- --
graylog-1   |  #####     ######   #####              #### ####      ####   ---     ---        ---     ---        -----
graylog-1   | ####         ####   ####       ############  ####     ####   ---    --           ---   ---           ---
graylog-1   | ###           ###   ####     ##############   ####   ####    ---   ---            --   --             --
graylog-1   | ####         ####   ####    ####       ####    #### ####     ---   ---            --   --            .--
graylog-1   | #####       #####   ####    ####       ####     #######      ---    ---          ---   ---           ---
graylog-1   |  ################   ####     ##############     ######-       --     ----      ----      ---       -----
graylog-1   |    ##############   ####      #############      #####        -----   -----------         ----------  --
graylog-1   |              ####                                ####                                                ---
graylog-1   | #####       ####                                ####                                     -          .--
graylog-1   |   #############                                ####                                     -----     ----
graylog-1   |      ######                                   ####                                          -------
graylog-1   | 
graylog-1   | ========================================================================================================
graylog-1   | 
graylog-1   | It seems you are starting Graylog for the first time. To set up a fresh install, a setup interface has
graylog-1   | been started. You must log in to it to perform the initial configuration and continue.
graylog-1   | 
graylog-1   | Initial configuration is accessible at 0.0.0.0:9000, with username 'admin' and password 'ghWgeIAkKl'.
graylog-1   | Try clicking on http://admin:ghWgeIAkKl@0.0.0.0:9000
graylog-1   | 
graylog-1   | ========================================================================================================
graylog-1   |

Configurations

Every configuration option can be set via environment variables. Simply prefix the parameter name with GRAYLOG_ for the Graylog server and GRAYLOG_DATANODE_ for the Data Node, put it all in upper case.

Graylog comes with a default configuration that works out of the box, but you have to set a password for the admin user and the web interface needs to know how to connect from your browser to the Graylog REST API.

Both settings can be configured via environment variables.

Copy

GRAYLOG_ROOT_PASSWORD_SHA2: "8c6976e5b5410415bde908bd4dee15dfb167a9c873fc4bb8a81f6f2ab448a918"
GRAYLOG_HTTP_EXTERNAL_URI: "http://127.0.0.1:9000/"

In this case you can log into Graylog with the username and password admin.

Generate your own admin password with the following command and put the SHA-256 hash into the GRAYLOG_ROOT_PASSWORD_SHA2 environment variable:

Copy

echo -n "Enter Password: " && head -1 < /dev/stdin | tr -d '\n' | sha256sum | cut -d " " -f1

All these settings and command line parameters can be put in a docker-compose.yml file, so that they don’t have to be executed one after the other.

Warning: The following example does not persist any data and configurations. You should read the section Persisting Data to add persistence to your docker-compose file.

Configurations via environment properties

For example, setting up the SMTP configuration for sending Graylog alert notifications via email, the docker-compose.yml would look like this:

Copy

version: '3'
services:
  mongo:
    image: "mongo:6.0.18"
    # Other settings [...]
  datanode:
    image: "graylog/graylog-datanode:6.1"
    # Other settings [...]
  graylog:
    image: "graylog/graylog:6.1"
    # Other settings [...]
    environment:
      GRAYLOG_TRANSPORT_EMAIL_ENABLED: "true"
      GRAYLOG_TRANSPORT_EMAIL_HOSTNAME: smtp
      GRAYLOG_TRANSPORT_EMAIL_PORT: 25
      GRAYLOG_TRANSPORT_EMAIL_USE_AUTH: "false"
      GRAYLOG_TRANSPORT_EMAIL_USE_TLS: "false"
      GRAYLOG_TRANSPORT_EMAIL_USE_SSL: "false"

Another option would be to store the configuration file outside of the container and edit it directly.

Custom Configuration Files

Instead of using a long list of environment variables to configure Graylog, you can also overwrite the bundled Graylog configuration files.

The bundled configuration files are stored in /usr/share/graylog/data/config/ inside the Docker container.

Create the new configuration directory next to the docker-compose.yml file and copy the default files from GitHub.

Copy

mkdir -p ./graylog/config
cd ./graylog/config
wget https://raw.githubusercontent.com/Graylog2/graylog-docker/6.1/config/graylog.conf
wget https://raw.githubusercontent.com/Graylog2/graylog-docker/6.1/config/log4j2.xml

The newly created directory ./graylog/config/ with the custom configuration files now has to be mounted into the Graylog Docker container.

This can be done by adding an entry to the volumes section of the docker-compose.yml file.

Example:

Copy

services:
  mongodb:
    image: "mongo:6.0.18"
# Other settings [...]
  datanode:
    image: "graylog/graylog-datanode:6.1"
# Other settings [...]
  graylog:
    image: "graylog/graylog-enterprise:6.1"
# Other settings [...]
volumes:
# Mount local configuration directory into Docker container
- ./graylog/config:/usr/share/graylog/data/config

Warning: Graylog is running as USER graylog with the ID 1100 in Docker. That ID needs to be able to read the configuration files you place into the container.

Reading Individual Configuration Settings from Files

The Graylog Docker image supports reading individual configuration settings from a file. This can be used to secure configuration settings with Docker secrets or similar mechanisms.

This offers the benefit of not needing to include sensitive configuration settings in a custom configuration file or store them as plaintext in an environment variable.

The Graylog Docker image checks for the existence of environment variables with the naming scheme GRAYLOG_<CONFIG_NAME>__FILE on startup and expects the environment variable to contain the absolute path to a readable file.

For example, if the environment variable GRAYLOG_ROOT_PASSWORD_SHA2__FILE contains the value /run/secrets/root_password_hash, the Graylog Docker image would use the contents of /run/secrets/root_password_hash as value for the root_password_sha2 configuration setting.

Docker Secrets

Hint: Docker secrets are only available in Docker Swarm services starting with Docker 1.13.

Example for using Docker secrets in a Docker Swarm service:

Copy

# Create SHA-256 hash of our password
echo -n 'password' | sha256sum | awk '{ print $1 }' 5e884898da28047151d0e56f8dc6292773603d0d6aabbdd62a11ef721d1542d8
# Create a Docker secret named "root_password_hash"
printf '5e884898da28047151d0e56f8dc6292773603d0d6aabbdd62a11ef721d1542d8' | \
docker secret create root_password_hash -
nlujwooo5uu6z0m91bmve79uo
docker secret ls
ID                          NAME                 DRIVER              CREATED             UPDATED
nlujwooo5uu6z0m91bmve79uo   root_password_hash                       34 seconds ago      34 seconds ago
# Create Docker Swarm service named "graylog" with access
# to the secret named "root_password_hash"
docker service create --name graylog \
--secret root_password_hash \
-e GRAYLOG_ROOT_PASSWORD_SHA2__FILE=/run/secrets/root_password_hash \
-p 9000:9000 graylog/graylog-enterprise:6.0
mclk5gm39ingk51s869dc0htz
overall progress: 1 out of 1 tasks
1/1: running   [==================================================>]
verify: Service converged
docker service ls
ID                  NAME                MODE                REPLICAS            IMAGE                                  PORTS
mclk5gm39ing        graylog             replicated          1/1                 graylog/graylog-enterprise:6.0      *:9000->9000/tcp

Persisting Data

In order to make the recorded data persistent, you can use external volumes to store all data. In the case of a container restart, this will simply re-use the existing data from the former instances.

Using Docker volumes for the data of MongoDB, DataNode, and Graylog, the docker-compose.yml file looks as follows:

MongoDB

Copy

volumes:
  - "/your/local/path:/data/db"

DataNode

Copy

volumes:
  - "/your/local/path:/var/lib/graylog-datanode/opensearch/data"

Graylog

Copy

volumes:
  - "/your/local/path:/usr/share/graylog/data/data"
  - "/your/local/path:/usr/share/graylog/data/config"
  - "/your/local/path:/usr/share/graylog/data/journal"

Kubernetes Automatic Leader Selection

Running Graylog in Kubernetes opens the challenge to set the is_master=true setting only for one node in the cluster. The problem can be solved by calculating the name of the pod if Graylog is running in a stateful set with the following environment variable.

Copy

env:
  - name: POD_NAME
    valueFrom:
      fieldRef:
        fieldPath: metadata.name

For a stateful set, the name of the first pod in a cluster always ends with -0. The leader selection mechanism in docker-entrypoint.sh file does the following:

Examines if Graylog is running inside Kubernetes

Verifies that the pod name ends in -0
Sets is_master=true for this container

Nomad Automatic Leader Selection

When running Graylog in a Nomad cluster, you must ensure that only one node in the cluster has the setting is_master=true in the server.conf file.

Whether the container is running on Nomad may be identified with an environmental check on NOMAD_ALLOC_INDEX. Should that variable be set to 0, the container will set Graylog to is_master=true. If the variable is set to anything other than 0, it will set Graylog to is_master=false.

Troubleshooting

If you encounter warnings about the open file limit, try setting the ulimit outside of the container.

Copy

ulimits:
    nofile:
      soft: "65536"
      hard: "65536"

The devicemapper storage driver can produce problems with Graylog's disk journal on some systems. In this case, choose another driver like aufs or overlay.

Testing a Beta Version

Warning: We recommend using pre-release versions only if you are an experienced Graylog user and confident in managing the process.

You can also run a pre-release (alpha, beta, or release candidate) version of Graylog using Docker. The pre-releases are tagged in the graylog/graylog or graylog/graylog-datanode Docker image.

Select from the available tags for the Graylog DataNode image on Docker Hub .

Copy

datanode:
    image: "graylog/graylog-datanode:6.1.0-beta.1-1"
graylog:
    hostname: "server"
    image: "graylog/graylog-enterprise:6.1.0-beta.1-1"

How to Get in Log Data

You can create different kinds of inputs under System > Inputs. However, you can only use ports that have been properly mapped to your Docker container; otherwise, data will not show up in the Graylog UI.

Copy

graylog:
    image: "graylog/graylog-enterprise:6.1"
    # Other settings [...]
    GRAYLOG_HTTP_EXTERNAL_URI: "http://localhost:9000/"
    ports:
      # Graylog web interface and REST API
      - "9000:9000/tcp"
      # Beats
      - "5044:5044/tcp"
      # Syslog TCP
      - "5140:5140/tcp"
      # Syslog UDP
      - "5140:5140/udp"
      # GELF TCP
      - "12201:12201/tcp"
      # GELF UDP
      - "12201:12201/udp"
      # Forwarder data
      - "13301:13301/tcp"
      # Forwarder config
      - "13302:13302/tcp"
    # Other settings [...]

For example, to start a Graylog Docker container listening on port 5555, stop your container and recreate it, while appending -p 5555:5555 to your docker run command:

Similarly, the same can be done for UDP by appending -p 5555:5555/udp.

After ensuring that your Graylog Docker container is listening on :5555, create a raw/plaintext input by navigating to your Graylog port, e.g. https://localhost:9000/system/inputs.

On the Inputs page, search for Raw/Plaintext TCP and click Launch new input.

After launching the input, a configuration form will pop up with several options. You can leave most of these options as their defaults, but note that you’ll need to provide a name for the input, as well as select the node, or select the Global option for the input location.

After that you can send a plain text message to the Graylog raw/plaintext TCP input running on port 5555 using the following command:

Copy

echo 'First log message' | nc localhost 5555

You can then view the message in the Graylog UI.