HomeObservabilityCockpitAPI/CLI
Configuring the Grafana agent
Jump toUpdate content

Configuring the Grafana agent

Reviewed on 28 November 2023Published on 10 January 2023

This page explains how to configure the Grafana agent and the Zipkin collector to push your metrics, logs and traces and visualize them in Grafana.

Security & Identity (IAM):

You may need certain IAM permissions to carry out some actions described on this page. This means:

  • you are the Owner of the Scaleway Organization in which the actions will be carried out, or
  • you are an IAM user of the Organization, with a policy granting you the necessary permission sets
Requirements:
Important:
  • Having the default configuration on your agents might lead to more of your resources’ metrics being sent, a high consumption and a high bill at the end of the month.
  • Sending metrics and logs for Scaleway resources or personal data using an external path is a billable feature. In addition, any data that you push yourself is billed, even if you send data from Scaleway products. Refer to the product pricing for more information.

Configuring and running the Grafana agent

Important:

It is not currently possible to collect logs if you are using OSX.

  1. Create a token and select the push permission for metrics, traces and logs.

  2. Create a folder where you will keep your configuration files.

  3. Create a configuration file inside your folder and name it config.yaml. This file will contain the Grafana agent configuration.

  4. Copy the following yaml template.

    Important:
    • Make sure you replace <your metrics push url> with https://metrics.cockpit.fr-par.scw.cloud/api/v1/push, <your logs push url> with https://logs.cockpit.fr-par.scw.cloud/loki/api/v1/push and <your traces push url> with https://traces.cockpit.fr-par.scw.cloud/otlp
    • Make sure you replace <your metrics push token>, <your logs push token> and <your traces push token> with the token you created in step 1
    metrics:
    wal_directory: /tmp/agent
    global:
    scrape_interval: 60s
    remote_write:
    - url: <your metrics push url>
    headers:
    "X-Token": <your metrics push token>
    logs:
    positions_directory: /tmp/positions/
    configs:
    - name: scw
    clients:
    - url: <your logs push url>
    tenant_id: <your logs push token>
    scrape_configs:
    - job_name: varlogs
    static_configs:
    - targets: [localhost]
    labels:
    job: varlogs
    __path__: /host/root/var/log/*.log
    traces:
    configs:
    - name: default
    remote_write:
    - endpoint: <your traces push url>
    headers:
    X-TOKEN: <your traces push token>
    protocol: http
    format: otlp
    receivers:
    otlp:
    protocols:
    grpc:
    http:
    zipkin:
    endpoint: 0.0.0.0:9411
    integrations:
    node_exporter:
    enabled: true
    rootfs_path: /host/root
    sysfs_path: /host/sys
    procfs_path: /host/proc
    # Anything not provided in the list below will be disabled by default
    set_collectors:
    - uname
    - cpu
    - loadavg
    - meminfo
    - filesystem
    - netdev

    Your configuration file should look like the following:

    metrics:
    wal_directory: /tmp/agent
    global:
    scrape_interval: 60s
    remote_write:
    - url: https://metrics.cockpit.fr-par.scw.cloud/api/v1/push
    headers:
    "X-Token": B6GI9xwdtWBxNcFlSxAWOM65R2aGpbTIbS4THIE4JmYZ6eQfj9upxPk5SfhdBm3
    logs:
    positions_directory: /tmp/positions/
    configs:
    - name: scw
    clients:
    - url: https://logs.cockpit.fr-par.scw.cloud/loki/api/v1/push
    tenant_id: B6GI9xwdtWBxNcFlSxAWOM65R2aGpbTIbS4THIE4JmYZ6eQfj9upxPk5SfhdBm3
    scrape_configs:
    - job_name: varlogs
    static_configs:
    - targets: [localhost]
    labels:
    job: varlogs
    __path__: /host/root/var/log/*.log
    traces:
    configs:
    - name: default
    remote_write:
    - endpoint: https://traces.cockpit.fr-par.scw.cloud/otlp
    headers:
    X-TOKEN: B6GI9xwdtWBxNcFlSxAWOM65R2aGpbTIbS4THIE4JmYZ6eQfj9upxPk5SfhdBm3
    protocol: http
    format: otlp
    receivers:
    otlp:
    protocols:
    grpc:
    http:
    zipkin:
    endpoint: 0.0.0.0:9411
    integrations:
    node_exporter:
    enabled: true
    rootfs_path: /host/root
    sysfs_path: /host/sys
    procfs_path: /host/proc
    # Anything not provided in the list below will be disabled by default
    set_collectors:
    - uname
    - cpu
    - loadavg
    - meminfo
    - filesystem
    - netdev
  5. In the same folder, create another file and name it docker-compose.yaml.

  6. Copy the following template and paste it into your file:

    version: "3"
    services:
    agent:
    image: grafana/agent:latest
    restart: always
    ports:
    - "9411:9411"
    volumes:
    - "./config.yaml:/etc/agent-config/agent.yaml:ro"
    - "/:/host/root:ro"
    - "/sys:/host/sys:ro"
    - "/proc:/host/proc:ro"
    - "/tmp/agent:/etc/agent"
    entrypoint:
    - /bin/grafana-agent
    - -config.file=/etc/agent-config/agent.yaml
    - -metrics.wal-directory=/tmp/agent/wal
    network_mode: "host"
    pid: "host"
  7. Run the following command to open your folder containing your configuration files:

    cd <your folder>
  8. Run the following command to bring the Grafana Agent container up:

    docker compose up
    Important:

    Run sudo docker compose up if you are using Linux and your user is not in the docker group.

Visualizing metrics on Grafana

  1. Log into Grafana using your Grafana credentials.
  2. Click the Toggle menu icon.
  3. Click Dashboards.
  4. Click New then click Import to import your Grafana dashboard.
  5. Enter 1860, which is the ID of the Node Exporter Full dashboard, in the Import via grafana.com field and click Load.
  6. Enter a name for your dashboard in the Name field.
  7. Select the Scaleway Metrics data source and click Import. Your dashboard displays with your metrics.

Visualizing logs on Grafana

  1. Log into Grafana using your Grafana credentials.
  2. Click Explore.
  3. Select the Scaleway logs data source in the drop-down.
  4. Configure a query using the instructions that display on your screen.
  5. Click Run query. Your logs display.
  6. Optionally, click the Live button on the top right corner of your screen to see your logs arriving in real time.

Visualizing traces on Grafana

  1. Log into Grafana using your Grafana credentials.
  2. Click the Toggle menu icon.
  3. Click Explore next to the compass icon.
  4. Select the Traces data source from the drop-down.
  5. Configure the queries you wish to apply to your traces. You traces display.