How to federate Scaleway metrics with your own Prometheus

In this page, we will show you how to federate your Scaleway metrics using the /federate endpoint of your Scaleway data sources with a Prometheus configuration. Metrics federation cnsists of collecting metrics from multiple sources or systems into a central place so you can visualize everything at once.

Before you startLink to this anchor

To complete the actions presented below, you must have:

• A Scaleway account logged into the console
• Owner status or IAM permissions allowing you to perform actions in the intended Organization
• Created a Cockpit token with the query permission in the same region as the metrics data source
• A running Prometheus instance

Prometheus federation can be used to gather all your metrics in a unique Prometheus instance, or to downsample different metrics to reduce storage costs in the long run. Since you have full control over your Prometheus instance, you can also use federation to increase metrics retention.

Configure your PrometheusLink to this anchor

Create a prometheus.yml file to configure your Prometheus instance, using the example below. Make sure you replace $SCW_DATASOURCE_URL with the URL of your data source. The URL can be found under the API URL section of the Data sources tab in the Scaleway console. Replace $SCW_COCKPIT_TOKEN with your Cockpit token with the query permission.

global:
  scrape_interval: 3m
scrape_configs:
  - job_name: 'federate-scaleway-metrics'
    honor_labels: true
    metrics_path: '/federate'
    http_headers:
      X-Token:
        values:
          - '$SCW_COCKPIT_TOKEN'
    params:
      'match[]':
        - '{__name__="instance_.*"}'
        - '{region="fr-par"}'
    static_configs:
      - targets:
        - '$SCW_DATASOURCE_URL'

Modify the different match[] parameters according to the metrics you want to retrieve. In this example, you will federate:

• All metrics with a name that starts with instance_
• All metrics with the label region with the value fr-par

You can also set honor_labels to false if you do not want to override the different labels set by your Prometheus that conflict with Scaleway metrics label. For example, with honor_labels set to false, the Scaleway label job will be relabelled as exported_job to avoid collision.

Run your Prometheus instanceLink to this anchor

Once you have configured your prometheus.yml file, you can use docker-compose to run your Prometheus instance with your configuration and start federating Scaleway metrics.

Create a docker-compose.yml file and paste the following template into it:

services:
  prometheus:
    image: prom/prometheus:latest
    container_name: prometheus
    volumes:
      - ./prometheus.yml:/etc/prometheus/prometheus.yml:ro
    ports:
      - 9090:9090
    command:
      - '--config.file=/etc/prometheus/prometheus.yml'

Access your localhost to see your newly created target federating your Scaleway metrics.

Monitor your federation volumesLink to this anchor

You can monitor the amount of samples federated with the /federate endpoint by looking at the observability_cockpit_federate_exported_sample_total:increase5m in your Cockpit. This metric indicates the total volume of exported samples through the federation endpoint over a 5 minutes interval.

1. Click Cockpit in the Monitoring section of the console side menu. The Cockpit overview page displays.
2. Click Open dashboards to access your Cockpit Grafana.
3. Click the Toggle menu icon in the top left corner of your screen, then click Explore.
4. In the Metric drop-down, select the observability_cockpit_federate_exported_sample_total:increase5m metrics and if necessary, update the time range in the top right corner of your screen.

Your metrics should display in the graph.