The Graylog Data Node is a management component designed to configure and optimize OpenSearch for use with Graylog. This feature also enhances the security of the data layer in Graylog by implementing certificates, managing cluster membership, and facilitating the addition of new nodes. Graylog Data Node ensures that the correct version of OpenSearch and its necessary extensions are installed to enable proper functionality of Graylog.

Prerequisites

• This guide assumes that you have followed the official Graylog installation documentation and have opted to deploy Data Node rather than self-managed OpenSearch. Prior to installing Graylog and the Graylog Data Node (as detailed in the following section), ensure that you have installed and enabled MongoDB according to the instructions provided in the relevant guides:

  • Graylog Installation on Ubuntu
  • Graylog Installation on Debian
  • Graylog Installation on Red Hat
  • Graylog Installation on SUSE

Install Graylog with Data Node

1. Install the Data Node package:

wget https://packages.graylog2.org/repo/packages/graylog-6.0-repository_latest.deb
sudo dpkg -i graylog-6.0-repository_latest.deb
sudo apt-get update
sudo apt-get install graylog-datanode

2. As noted in the OpenSearch documentation, ensure that the Linux setting vm.max_map_count is set to at least 262144. To check the current value, run this command:

cat /proc/sys/vm/max_map_count

To increase the value, add the following line to /etc/sysctl.conf:

vm.max_map_count=262144

3. Run the following command to reload this configuration setting: 

sudo sysctl -p

4. Now, open the Data Node configuration file:

sudo nano /etc/graylog/datanode/datanode.conf

5. Add the password_secret value to the Data Node configuration file:

6. To create your password_secret, run the following command:

< /dev/urandom tr -dc A-Z-a-z-0-9 | head -c${1:-96};echo;

7. Enable the Data Node service and start:

sudo systemctl enable graylog-datanode.service
sudo systemctl start graylog-datanode

8. Install the Graylog repository configuration and Graylog itself with the following commands.

For Graylog Open: 

sudo apt-get install graylog-server

For Graylog Enterprise:

sudo apt-get install graylog-enterprise

9. Now, open the Graylog configuration file:

sudo nano /etc/graylog/server/server.conf

10. Retrieve the password secret from the Data Node configuration file as indicated in the previous steps and add it to the Graylog configuration file.

11. To be able to connect to Graylog, set the http_bind_address value in the Graylog configuration file to the public host name or a public IP address for the machine to which you can connect. More information about these settings can be found in Configuring the Web Interface. Alternatively, you can also set this configuration using this command:

sudo sed -i 's/#http_bind_address = 127.0.0.1.*/http_bind_address = 0.0.0.0:9000/g' /etc/graylog/server/server.conf

12. The last step is to enable Graylog during the operating system’s startup:

sudo systemctl daemon-reload
sudo systemctl enable graylog-server.service
sudo systemctl start graylog-server.service
sudo systemctl --type=service --state=active | grep graylog

Log into the Graylog Interface

Your initial login credentials for the Graylog web interface can be found in the log file upon starting Graylog for the first time. These credentials are for the initial configuration administrative user and will only need to be used upon first installing and setting up Graylog. This is to ensure that, if you have not yet set up HTTPS to connect to Graylog, your administrative password is not compromised.

To view your initial password and the instructions included in the log fie, enter the following:

tail /var/log/graylog-server/server.log

Upon subsequent logins you will be able to use the password_secret you selected upon installation.

Configure the Data Node in Preflight

Upon initial start up of your Graylog instance, you will be prompted by the Data Node preflight user interface to complete your initial configuration and secure your node(s).

Preview Graylog Data Nodes

In the preflight menu you will find a list of the available data nodes discovered by Graylog upon installation. Note that all data nodes to be used initially should be started before they are provisioned, but additional nodes can be added by starting them after the initial setup has completed and Graylog is running. They will be provisioned and join the cluster automatically.

Provision Certificates

To secure your data node(s), you can choose either to upload an existing certificate authority or to provision a certificate directly in the UI.

If you are uploading an existing certificate, .crt and .pem file extensions are supported. Additionally, you will be prompted to enter the password for your certificate file.

Provision a Certificate Authority with Graylog

Issuing a certificate authority may be completed directly in the preflight UI.

1. Begin by selectingCreate new CA.

2. Then, select the Create CA button.

3. You will need to establish your certificate renewal policy. If you select Automatic, then Graylog will reissue the CA based upon the certificate lifetime you select. If you select Manual, then you will be able to manually reissue the CA when the designated lifetime has completed. This policy can also be changed at a later point in time as desired in the Graylog Data Node interface.

4. Once you have made your selections, click Create policy to continue.

5. When the CAs have been assigned, you can select Provision certificate to issue the CA to your data nodes. This may take a few moments.

6. Select Resume startup to complete the process. Graylog will start up within a few moments, and you will be presented with the login screen to log in with the credentials you entered in the Graylog configuration file.

Create Client Certificate for Third-Party Tools

Fundamentally, Graylog Data Node secures OpenSearch. Therefore, in order to query it directly (like via HTTPS), you will need to generate a client certificate.

1. Navigate to System > Data Nodes.

2. Go to the Configuration tab.

3. Select Generate Client Certificate.

4. Complete the following fields as directed:

   Principal: fully qualified domain name (FQDN) of the data node server to which you wish to connect. For example, if the hostname of the client is server123, and its domain is network.local, so its FQDN would be server123.network.local.

   Role: defaults to all_access. See users and roles for a list of predefined roles.

   Password: password used to generate the private key that was used to generate the certificate.

5. Save the certificates into separate files with the following naming conventions:

   Private Key: client-cert.key

   Certificate: client-cert.crt

Use Third-Party Tools

Once your client certificate is issued, you may use these third-party tools such as curl or Grafana to query OpenSearch's API.

Query via Curl

If you choose to use a command-line tool like curl, save the private key and certificate text files and name them as above. You may run this command from your terminal:

curl "https://hostname.domain.tld:9200/_cluster/health?pretty" -k --cert client-cert.crt --key client-cert.key

Query via Grafana

If you choose to use Grafana, you can copy and paste the certificates into the relevant boxes.

We encourage you to read the Grafana documentation for full instructions on using Grafana with OpenSearch. Note the following significant steps in the configuration process as follows:

• Add OpenSearch as a new data source.

• Specify the URL using HTTPS, like https://hostname.domain.tld:9200.

• After pasting the certificate files into their relevant Grafana boxes, you may need to correct the certificate formats so that -END CERTIFICATE and -END RSA PRIVATE KEY are on new lines. This change must be applied to the CA, client certificate, and client key.