Cockpit API

Introduction

Scaleway's Cockpit allows you to monitor your applications and their infrastructure by giving you insights and context into their behavior. Cockpit also enables you to visualize your metrics, logs, and traces through a Grafana dashboard. With Cockpit, you can also push data from your Scaleway resources, if they are integrated with Cockpit.

The Observability Cockpit provides you with two Prometheus Remote Write endpoints for pushing metrics and logs. You can push metrics with any Prometheus Remote Write compatible agent such as Prometheus, Grafana or OpenTelemetry Collector.

You can push logs with any Loki compatible agent such as Promtail, Fluentd, Fluent Bit, OpenTelemetry Collector, or Logstash.

Refer to the Cockpit Global API for information on how to perform Cockpit visualization actions such as retrieving your dashboard URLs, managing Grafana users, and synchronizing your data through Grafana dashboards.

Concepts

Refer to our dedicated concepts page to find definitions of the different terms referring to the Observability Cockpit.

Quickstart

1. Configure your environment variables.

   Note

   This is an optional step that seeks to simplify your usage of the API.

   export SCW_SECRET_KEY="<API secret key>"
   export SCW_PROJECT_ID="<Scaleway Project ID>"
   

2. Build push URLs.

   Retrieve your endpoints for metrics, logs, and traces from the Data sources tab of the Scaleway console. Make sure that you replace xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx and fr-par with the ID and the actual Availibility Zone and region of your data source.

   {
       [...],
       "endpoints": {
           "metrics_url": "https://xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx.metrics.cockpit.fr-par.scw.cloud",
           "logs_url": "https://xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx.logs.cockpit.fr-par.scw.cloud",
           "alertmanager_url": "https://alertmanager.cockpit.fr-par.scw.cloud",
           "grafana_url": "<project_id>.dashboard.obs.fr-par.scw.cloud",
           "traces_url": "https://xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx.traces.cockpit.fr-par.scw.cloud"
       },
       [...]
   }
   

   The metrics_url is a domain that exposes a Prometheus-like API to manage metrics, and the logs_url exposes a Loki API to manage logs.

   Important

   To be able to send metrics, logs, and traces, you need the exact URL which you can find in the Data sources tab of the Scaleway console under the API URL section. This URL contains the ID of your data source.

   • The Prometheus Remote Write endpoint to push your metrics is the following: https://xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx.metrics.cockpit.fr-par.scw.cloud/api/v1/push
   • The Remote Write endpoint to push your logs is the following: https://xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx.logs.cockpit.fr-par.scw.cloud/loki/api/v1/push
   • The Prometheus Alertmanager endpoint is the following: https://alertmanager.cockpit.fr-par.scw.cloud/alertmanager
   • Traces have different endpoints depending on the push path you use. The traces endpoint for the OpenTelemetry agent is: https://xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx.traces.cockpit.fr-par.scw.cloud/otlp.

3. Create your Cockpit token to push metrics, logs, and traces.

   Find out how to create your Cockpit token via the console or run the following command to create it via the API.

   curl -X POST \
   "https://api.scaleway.com/cockpit/v1beta1/tokens" \
   -H "Content-Type: application/json" \
   -H "X-Auth-Token: $SCW_SECRET_KEY" \
   -d '{
     "project_id": "<Scaleway Project ID>",
     "name": "token-name",
     "scopes": {
       "query_metrics": false,
       "write_metrics": true,
       "setup_metrics_rules": false,
       "query_logs": false,
       "write_logs": true,
       "setup_logs_rules": false,
       "setup_alerts": false,
       "query_traces": false,
       "write_traces": true
     }
   }'
   

   Important

   Your Cockpit token's secret_key only displays once. Make sure that you save it. We strongly recommend that you only give the minimal amount of permissions to your token.

4. Configure the Grafana alloy agent.

   Find out how to configure the Grafana alloy agent to push your data to Cockpit.

5. Configure rules and alerts.

   Upon creation, your Grafana instance is already configured by default with your logs and metrics data sources and with the alert manager data source. This means that you will be able to configure and manage advanced alerting rules, for metrics and logs, configure contact points and notification policies, from Grafana.

   To do so, access your Grafana dashboard and read our dedicated documentation to find out how to configure alert rules for Scaleway resources. Refer to the External links section of this page if you are not familiar with the alert manager or alert rules.

   Important

   • Make sure that you select the Scaleway Alerting alert manager, as Scaleway does not support the Grafana alert manager.
   • Make sure that you create rules on the data source containing the data you wish to be alerted on rather than on Grafana. (datasource-managed-alerts)

Technical information

Regional availability

Scaleway's infrastructure is spread across different regions and Availability Zones. Your Cockpit resources are available in the Paris, Amsterdam and Warsaw regions.

Current supported endpoints

Cockpit supports telemetry ingestion and querying using the following open-source technologies:

• Mimir for metrics
• Loki for logs
• Tempo for traces
• Prometheus for the alert manager

Cockpit exposes a subset of their HTTP APIs so users can push, query, and manage telemetry data from Cockpit.

Refer to the dedicated documentation to find out which API endpoints are supported by Cockpit for data source management.

Troubleshooting

Refer to our troubleshooting documentation if your pusher does not support HTTP headers or if you want to reset your Grafana password.

Technical limitations

• Metrics and logs data is retained 31 days. After this period, data older than 31 days is deleted.
• The number of active metrics time series is limited to 250,000 per Cockpit by default.
• The number of active log streams is limited to 5000 per Cockpit.
• It is not yet possible to downsample metrics.

Going further

For more information about Cockpit, you can check out the following pages:

• Cockpit Documentation
• Scaleway Slack Community join the #observability-beta and the #observability channels
• Contact our support team

External links

If you are interested in learning more, you can check out the following pages:

• Pushing logs using Fluentbit. Fluentbit is another commonly used agent that collects and forwards logs.
• Pushing metrics using Prometheus's agent mode. Prometheus can be used in an "agent-mode" that can scrape local targets and forward them using remote-write. This is useful for scenarios where targets are exposed in a private network (like a VPN or a single host) but need to be forwarded outside of it.
• How to create a Mimir or Loki managed alerting rule using Grafana.