Configuring the Grafana agent
This page explains how to configure the Grafana agent and the Zipkin collector to push your metrics, logs and traces and visualize them in Grafana.
You may need certain IAM permissions to carry out some actions described on this page. This means:
- you are the Owner of the Scaleway Organization in which the actions will be carried out, or
- you are an IAM user of the Organization, with a policy granting you the necessary permission sets
- You have an account and are logged into the Scaleway console
- You have activated your Cockpit
- You have installed docker and docker compose
- Having the default configuration on your agents might lead to more of your resources’ metrics being sent, a high consumption and a high bill at the end of the month.
- Sending metrics and logs for Scaleway resources or personal data using an external path is a billable feature. In addition, any data that you push yourself is billed, even if you send data from Scaleway products. Refer to the product pricing for more information.
Configuring and running the Grafana agent
It is not currently possible to collect logs if you are using OSX.
-
Create a token and select the push permission for metrics, traces and logs.
-
Create a folder where you will keep your configuration files.
-
Create a configuration file inside your folder and name it
config.yaml
. This file will contain the Grafana agent configuration. -
Copy the following yaml template.
Important:- Make sure you replace
<your metrics push url>
withhttps://metrics.cockpit.fr-par.scw.cloud/api/v1/push
,<your logs push url>
withhttps://logs.cockpit.fr-par.scw.cloud/loki/api/v1/push
and<your traces push url>
withhttps://traces.cockpit.fr-par.scw.cloud/otlp
- Make sure you replace
<your metrics push token>
,<your logs push token>
and<your traces push token>
with the token you created in step 1
metrics:wal_directory: /tmp/agentglobal:scrape_interval: 60sremote_write:- url: <your metrics push url>headers:"X-Token": <your metrics push token>logs:positions_directory: /tmp/positions/configs:- name: scwclients:- url: <your logs push url>tenant_id: <your logs push token>scrape_configs:- job_name: varlogsstatic_configs:- targets: [localhost]labels:job: varlogs__path__: /host/root/var/log/*.logtraces:configs:- name: defaultremote_write:- endpoint: <your traces push url>headers:X-TOKEN: <your traces push token>protocol: httpformat: otlpreceivers:otlp:protocols:grpc:http:zipkin:endpoint: 0.0.0.0:9411integrations:node_exporter:enabled: truerootfs_path: /host/rootsysfs_path: /host/sysprocfs_path: /host/proc# Anything not provided in the list below will be disabled by defaultset_collectors:- uname- cpu- loadavg- meminfo- filesystem- netdevYour configuration file should look like the following:
metrics:wal_directory: /tmp/agentglobal:scrape_interval: 60sremote_write:- url: https://metrics.cockpit.fr-par.scw.cloud/api/v1/pushheaders:"X-Token": B6GI9xwdtWBxNcFlSxAWOM65R2aGpbTIbS4THIE4JmYZ6eQfj9upxPk5SfhdBm3logs:positions_directory: /tmp/positions/configs:- name: scwclients:- url: https://logs.cockpit.fr-par.scw.cloud/loki/api/v1/pushtenant_id: B6GI9xwdtWBxNcFlSxAWOM65R2aGpbTIbS4THIE4JmYZ6eQfj9upxPk5SfhdBm3scrape_configs:- job_name: varlogsstatic_configs:- targets: [localhost]labels:job: varlogs__path__: /host/root/var/log/*.logtraces:configs:- name: defaultremote_write:- endpoint: https://traces.cockpit.fr-par.scw.cloud/otlpheaders:X-TOKEN: B6GI9xwdtWBxNcFlSxAWOM65R2aGpbTIbS4THIE4JmYZ6eQfj9upxPk5SfhdBm3protocol: httpformat: otlpreceivers:otlp:protocols:grpc:http:zipkin:endpoint: 0.0.0.0:9411integrations:node_exporter:enabled: truerootfs_path: /host/rootsysfs_path: /host/sysprocfs_path: /host/proc# Anything not provided in the list below will be disabled by defaultset_collectors:- uname- cpu- loadavg- meminfo- filesystem- netdev - Make sure you replace
-
In the same folder, create another file and name it
docker-compose.yaml
. -
Copy the following template and paste it into your file:
version: "3"services:agent:image: grafana/agent:latestrestart: alwaysports:- "9411:9411"volumes:- "./config.yaml:/etc/agent-config/agent.yaml:ro"- "/:/host/root:ro"- "/sys:/host/sys:ro"- "/proc:/host/proc:ro"- "/tmp/agent:/etc/agent"entrypoint:- /bin/grafana-agent- -config.file=/etc/agent-config/agent.yaml- -metrics.wal-directory=/tmp/agent/walnetwork_mode: "host"pid: "host" -
Run the following command to open your folder containing your configuration files:
cd <your folder> -
Run the following command to bring the Grafana Agent container up:
docker compose upImportant:Run
sudo docker compose up
if you are using Linux and your user is not in the docker group.
Visualizing metrics on Grafana
- Log into Grafana using your Grafana credentials.
- Click the Toggle menu icon.
- Click Dashboards.
- Click New then click Import to import your Grafana dashboard.
- Enter
1860
, which is the ID of the Node Exporter Full dashboard, in the Import via grafana.com field and click Load. - Enter a name for your dashboard in the Name field.
- Select the Scaleway Metrics data source and click Import. Your dashboard displays with your metrics.
Visualizing logs on Grafana
- Log into Grafana using your Grafana credentials.
- Click Explore.
- Select the Scaleway logs data source in the drop-down.
- Configure a query using the instructions that display on your screen.
- Click Run query. Your logs display.
- Optionally, click the Live button on the top right corner of your screen to see your logs arriving in real time.
Visualizing traces on Grafana
- Log into Grafana using your Grafana credentials.
- Click the Toggle menu icon.
- Click Explore next to the compass icon.
- Select the Traces data source from the drop-down.
- Configure the queries you wish to apply to your traces. You traces display.