Configuring the Grafana agent

Reviewed on 15 October 2024 • Published on 10 January 2023

This page explains how to configure the Grafana agent and the Zipkin collector to push your metrics, logs, and traces. You can use it to push your data from Scaleway resources or external resources.

Important

The Grafana agent has been deprecated by Grafana. Find out how to configure Grafana Alloy which is Grafana’s new telemetry collector.

Before you start

To complete the actions presented below, you must have:

Owner status or IAM permissions allowing you to perform actions in the intended Organization
A Scaleway account logged into the console
Installed Docker and Docker compose

Important

Having the default configuration on your agents might lead to more of your resources’ metrics being sent, a high consumption and a high bill at the end of the month.
Sending metrics and logs for Scaleway resources or personal data using an external path is a billable feature. In addition, any data that you push yourself is billed, even if you send data from Scaleway products. Refer to the product pricing for more information.

Configuring and running the Grafana agent

Important

It is not currently possible to collect logs if you are using OSX.

Create a token and select the push permission for metrics, traces, and logs.
Create a folder where you will keep your configuration files.
Create a configuration file inside your folder and name it config.yaml. This file will contain the Grafana agent configuration.

Copy the following YAML template.

Important

Make sure you replace <your metrics push url> with https://metrics.cockpit.fr-par.scw.cloud/api/v1/push, <your logs push url> with https://logs.cockpit.fr-par.scw.cloud/loki/api/v1/push and <your traces push url> with https://traces.cockpit.fr-par.scw.cloud/otlp
Make sure you replace <your metrics push token>, <your logs push token> and <your traces push token> with the token you created in step 1

metrics:
  wal_directory: /tmp/agent
  global:
    scrape_interval: 60s
    remote_write:
    - url: <your metrics push url>
      headers:
        "X-Token": <your metrics push token>
logs:
  positions_directory: /tmp/positions/
  configs:
  - name: scw
    clients:
    - url: <your logs push url>
      tenant_id: <your logs push token>
    scrape_configs:
    - job_name: varlogs
      static_configs:
      - targets: [localhost]
        labels:
          job: varlogs
          __path__: /host/root/var/log/*.log
traces:
  configs:
  - name: default
  remote_write:
    - endpoint: <your traces push url>
      headers:
        X-TOKEN: <your traces push token>
      protocol: http
      format: otlp
  receivers:
        otlp:
          protocols:
            grpc:
            http:
        zipkin:
          endpoint: 0.0.0.0:9411
integrations:
  node_exporter:
    enabled: true
    rootfs_path: /host/root
    sysfs_path: /host/sys
    procfs_path: /host/proc
    # Anything not provided in the list below will be disabled by default
    set_collectors:
    - uname
    - cpu
    - loadavg
    - meminfo
    - filesystem
    - netdev
    - boottime
    - time
    - stat

Your configuration file should look like the following:

metrics:
  wal_directory: /tmp/agent
  global:
    scrape_interval: 60s
    remote_write:
    - url: https://metrics.cockpit.fr-par.scw.cloud/api/v1/push
      headers:
        "X-Token": B6GI9xwdtWBxNcFlSxAWOM65R2aGpbTIbS4THIE4JmYZ6eQfj9upxPk5SfhdBm3
logs:
  positions_directory: /tmp/positions/
  configs:
  - name: scw
    clients:
    - url: https://logs.cockpit.fr-par.scw.cloud/loki/api/v1/push
      tenant_id: B6GI9xwdtWBxNcFlSxAWOM65R2aGpbTIbS4THIE4JmYZ6eQfj9upxPk5SfhdBm3
    scrape_configs:
    - job_name: varlogs
      static_configs:
      - targets: [localhost]
        labels:
          job: varlogs
          __path__: /host/root/var/log/*.log
traces:
  configs:
    - name: default
      remote_write:
        - endpoint: https://traces.cockpit.fr-par.scw.cloud/otlp
          headers:
            X-TOKEN: B6GI9xwdtWBxNcFlSxAWOM65R2aGpbTIbS4THIE4JmYZ6eQfj9upxPk5SfhdBm3
          protocol: http
          format: otlp
      receivers:
        otlp:
          protocols:
            grpc:
            http:
        zipkin:
          endpoint: 0.0.0.0:9411
integrations:
  node_exporter:
    enabled: true
    rootfs_path: /host/root
    sysfs_path: /host/sys
    procfs_path: /host/proc
    # Anything not provided in the list below will be disabled by default
    set_collectors:
      - uname
      - cpu
      - loadavg
      - meminfo
      - filesystem
      - netdev

You can also specify the data source containing your data types by:
- Adding the ID of the data source to the endpoint. You can retrieve the ID of the data source you want to configure using the ListDataSources API command, or
- Adding the X-Datasource-Id header in the headers section, under X-Token.
  Important
  If you do not specify a data source, your data will automatically be pushed to the default data source for the concerned data type.
Below is an example of what your Traces configuration should look like if you add the X-Datasource-Id header:
```
    traces:
      configs:
        - name: default
          remote_write:
            - endpoint: https://traces.cockpit.fr-par.scw.cloud/otlp
              headers:
                X-TOKEN: B6GI9xwdtWBxNcFlSxAWOM65R2aGpbTIbS4THIE4JmYZ6eQfj9upxPk5SfhdBm3
                X-Datasource-Id: <your traces data source id>
              protocol: http
              format: otlp
          receivers:
            otlp:
              protocols:
                grpc:
                http:
            zipkin:
              endpoint: 0.0.0.0:9411
```
In the same folder, create another file and name it docker-compose.yaml.

Copy the following template and paste it into your file:

version: "3"
services:
  agent:
    image: grafana/agent:latest
    restart: always
    ports:
      - "9411:9411"
    volumes:
      - "./config.yaml:/etc/agent-config/agent.yaml:ro"
      - "/:/host/root:ro"
      - "/sys:/host/sys:ro"
      - "/proc:/host/proc:ro"
      - "/tmp/agent:/etc/agent"
    entrypoint:
      - /bin/grafana-agent
      - -config.file=/etc/agent-config/agent.yaml
      - -metrics.wal-directory=/tmp/agent/wal
    network_mode: "host"
    pid: "host"

Run the following command to open your folder containing your configuration files:

cd <your folder>

Run the following command to bring the Grafana Agent container up:

docker compose up

Important

Run sudo docker compose up if you are using Linux and your user is not in the docker group.

Visualizing metrics in Grafana

Log into Grafana using your Grafana credentials.
Click the Toggle menu icon.
Click Dashboards.
Click New then click Import to import your Grafana dashboard.
Enter 1860, which is the ID of the Node Exporter Full dashboard, in the Import via grafana.com field and click Load.
Enter a name for your dashboard in the Name field.
Select the Metrics data source and click Import. Your dashboard displays with your metrics.

Visualizing logs in Grafana

Log into Grafana using your Grafana credentials.
Click Explore.
Select the Scaleway logs data source in the drop-down.
Configure a query using the instructions that display on your screen.
Click Run query. Your logs display.
Optionally, click the Live button on the top right corner of your screen to see your logs arriving in real time.

Visualizing traces in Grafana

Log into Grafana using your Grafana credentials.
Click the Toggle menu icon.
Click Explore next to the compass icon.
Select the Traces data source from the drop-down.
Configure the queries you wish to apply to your traces. Your traces display.

Tip
Refer to the Grafana documentation to learn more about how to visualize your traces.