Google Inputs

The following article exclusively pertains to a Graylog Enterprise feature or functionality. To learn more about obtaining an Enterprise license, please contact the Graylog Sales team.

Similar to O365 and Okta, Graylog can gather logs from Google Services. The process to launch inputs for services is described below:

  • Google Cloud (GCP)
  • Google Workspace
  • Gmail

Depending on the integration, the steps differ by service. Each section is identified as:

  • [All], if the instructions apply to all services mentioned above.
  • [GCP], if the instructions only apply to the Google Cloud Platform.
  • [Gmail], if the steps only apply to Gmail.
  • [Workspace], if the steps apply to Google Workspace configuration.

Requirements

To successfully pull logs from these Google products into Graylog, you must have:

  • a running instance of Graylog.
  • Google Cloud account. See the Cloud subdomain.

[All] Caveats

Both the GPC and Gmail plugins create log sinks to fetch logs. Log data is then stored in Google BigQuery in your account. The Google inputs clean up the BigQuery tables periodically, but additional Google Cloud charges for BigQuery usage may apply.

Like Okta and O365, Google inputs poll for data. Therefore, run them on a single node. Avoid running Google inputs as global inputs.

[All] Service Account Creation

To collect logs for a project, enter the Google Cloud Console, and select the project. Note the Project ID, as it is required during Graylog input setup.

Set up a new service account.

  1. Select IAM & Admin > IAM from the cloud dashboard.

  2. Select Service Accounts from the menu on the left.

            gcp-service-account

  1. Select +CREATE SERVICE ACCOUNT at the top of the page.

            

  1. Note the Unique ID associated with the service account, as it is needed to set up inputs.

[All] Generate Service Account Key

Generate a key file for the service account that will be placed on the Graylog server to allow inputs to authenticate with Google’s APIs.

  1. Navigate to the Service Accounts page. Select the intended service account.
  2. Click the KEYS tab on the sub-menu.

            gcp-select-key

  1. Click on the ADD KEY button, and select Create a new key.

            gcp-create-key

  1. Select JSON as the key type. Click CREATE.

            gcp-json-key

  1. Save the key in a safe location for input setup.
  2. [Workspace] Create and download a P12 key for the Google Workspace input.

[All] Grant Permissions to the Service Account

The service account requires permissions to access log data and needs to store log data in Google BigQuery to allow Graylog to fetch the data.

  1. Click on the pencil icon to edit the Principal for the service account (found on the IAM page).

            gcp-edit-principal

  1. Grant the service account the BigQuery Data Editor role.
  2. Grant the service account the BigQuery Jobs User role.
  3. Grant the service account the Logs Configuration Writer role.

[GCP] Enable Logging

To collect VPC flow logs, enable log. For more information, see Using VPC Flow Logs.

To collect firewall logs, enable them in the firewall configuration.

[Workspace] Enable API Access

To enable access to Workspace endpoints:

  1. Log in as a user account in the Google Workspace with the Super Admin role.

            gcp-admin-roles-privileges

  1. Logged in as the super admin user, go to Google Cloud Platform to create a new project or select an existing project. The project will need a service account as described above.
  2. Navigate to APIs & Services > Library.

           gcp-api-library

  1. Search for Admin SDK API. Click Enable.

            gcp-enable-sdk

  1. Return to the Google Workspace console. Navigate to Security > API Controls to link the service account to the API.

           gcp-api-config

  1. Select Manage Domain Wide Delegation, and add a new API client.

            gcp-add-api-client

  1. Use the numeric Unique ID of the service account for the Client ID, and add the following to the OAuth Scopes:

[GCP] Input Setup

Key Value
Input name Add a unique name for the input
Project ID Alphanumeric project ID for the Google Cloud project
Application (client) ID Unique numeric ID of the service account
Service account key path Path to .json file for the service account

BigQuery Dataset Region

(Optional) The region where the Bigquery dataset will be created

[Workspace] Input Setup

Key Value
Input name A unique name for the input
Client ID Unique numeric ID of the service account
Service Account ID Email address of the service account
Account User Email Workspace email address of the user that owns the project
Service account key path Path to .p12 file for the service account

[Gmail] Input Setup

Key Value
Input name < Add a unique name for the input >
Project ID Alpha-numeric project ID for the Google Cloud project
Application (client) ID Unique numeric ID of the service account
Service account key path Path to .json file for the service account