Cockpit API

Introduction

Scaleway's Cockpit allows you to monitor your applications and their infrastructure by giving you insights and context into their behavior. Cockpit also enables you to visualize your metrics and logs through a Grafana dashboard. With Cockpit, you can also push your own data as Scaleway products' data is included by default.

The Observability Cockpit provides you with two Prometheus Remote Write endpoints for pushing metrics and logs. You can push metrics with any Prometheus Remote Write compatible agent such as Prometheus, Grafana or OpenTelemetry Collector.

You can push logs with any Loki compatible agent such as Promtail, Fluentd, Fluent Bit or Logstash.

Concepts

Refer to our dedicated concepts page to find definitions of the different terms referring to the Observability Cockpit.

Quickstart

1. Configure your environment variables.

   Note:

   This is an optional step that seeks to simplify your usage of the API.

   export SCW_SECRET_KEY="<API secret key>"
   export SCW_PROJECT_ID="<Scaleway Project ID>Copy

2. Activate your Cockpit.

   Run the following command to activate your Cockpit:

   curl -X POST \
     -H "X-Auth-Token: $SCW_SECRET_KEY" \
     https://api.scaleway.com/cockpit/v1beta1/activate \
     -H "Content-Type: application/json" \
     -d '{"project_id": "'"$SCW_PROJECT_ID"'"}'Copy

3. Build push URLs.

   A Cockpit has 4 endpoints which are given when creating or getting a Cockpit. The endpoints look like the following:

   {
       [...],
       "endpoints": {
           "metrics_url": "https://metrics.cockpit.fr-par.scw.cloud",
           "logs_url": "https://logs.cockpit.fr-par.scw.cloud",
           "alertmanager_url": "https://alertmanager.cockpit.fr-par.scw.cloud"
           "grafana_url": "<project_id>.dashboard.obs.fr-par.scw.cloud"
       },
       [...]
   }Copy

   The metrics_url is a domain that exposes a Prometheus-like API to manage metrics, and the logs_url exposes a Loki API to manage logs.

   Important:

   To be able to send metrics and logs, you need the exact URL to use with your pushers.

   • The Prometheus Remote Write endpoint to push your metrics is the following: https://metrics.cockpit.fr-par.scw.cloud/api/v1/push
   • The Remote Write endpoint to push your logs is the following: https://logs.cockpit.fr-par.scw.cloud/loki/api/v1/push
   • The Prometheus Alertmanager endpoint is the following: https://alertmanager.cockpit.fr-par.scw.cloud/alertmanager

4. Create your token to push metrics and logs.

   Find out how to create your token via the console or run the following command to create your token via API.

   curl -X POST \
   "https://api.scaleway.com/cockpit/v1beta1/tokens" \
   -H "Content-Type: application/json" \
   -H "X-Auth-Token: $SCW_SECRET_KEY" \
   -d '{
     "project_id": "00000000-0000-0000-0000-000000000000",
     "name": "token-name",
     "scopes": {
       "query_metrics": false,
       "write_metrics": true,
       "setup_metrics_rules": false,
       "query_logs": false,
       "write_logs": true,
       "setup_logs_rules": false,
       "setup_alerts": false
     }
   }'Copy

   Important:

   Your token's secret_key only displays once. Make sure you save it. We strongly recommend that you only give the minimal amount of permissions to your token. Metric pushers should only have the write_metrics scope, and log pushers the write_logs one.

5. Configure the Grafana agent.

   Find out how to configure, start the Grafana agent and see your metrics and logs in our documentation.

   Note:

   The promtail configuration for the logs does not support custom headers. The tenant_id field corresponds to the header X-Scope-OrgID which is one of the supported headers. For more details on the different supported headers, see our troubleshooting documentation for when your pusher does not support custom HTTP headers.

6. Configure rules and alerts.

   Upon its creation, a Grafana instance is already configured by default with logs and metrics data sources. The alert manager data source is also configured. This means that you will be able to configure your alerting rules, for metrics and logs, configure your contact points and notification policies and manage your alert silences from Grafana.

   To do so, on your Grafana instance, click the Bell icon on the left of your screen. The Alerting page displays with tabs. Refer to the External links section of this page if you are not familiar with the alert manager or alert rules.

   Important:

   For the alerts and rules to be created in your Cockpit, you must use the data sources provided by Scaleway.

   In the Alert rules tab, you must select Mimir or Loki alert when creating your alert rules in the Rule type field. Then, on the Select data source field that appears on the right, select the data source you wish to create the rule on.

   For all the other tabs, from Contact points to Alert groups, you will be configuring the alert manager. The Choose Alertmananger field displays at the top of the screen.

   Important:

   Make sure that you select Scaleway Alerting as your alert manager as the alerts are managed by the platform. If you use Grafana you will work with the local alert manager which is deactivated.

Technical information

Regional availability

Scaleway's Cockpit is available globally. Find out about our product availability in our dedicated documentation.

Current supported endpoints

• The current supported endpoints for metrics are the following. For more information about the endpoints, refer to the Mimir documentation

  # Remote write endpoints
  path /api/v1/push
  # Query endpoints
  path /prometheus/api/v1/query 
  path /prometheus/api/v1/query_range 
  path /prometheus/api/v1/query_exemplars 
  path /prometheus/api/v1/series 
  path /prometheus/api/v1/labels
  path /prometheus/api/v1/label/*
  path /prometheus/api/v1/metadata 
  path /prometheus/api/v1/read 
  path /prometheus/api/v1/status/buildinfo
  # Ruler endpoints
  path /prometheus/api/v1/rules 
  path /prometheus/api/v1/alerts 
  path /prometheus/config/v1/rules 
  path /prometheus/config/v1/rules/*Copy

• The current supported endpoints for logs are the following. For more information about the endpoints, refer to the Loki documentation

  # Remote write endpoints 
  path /loki/api/v1/push
  # Query endpoints 
  path /loki/api/v1/query 
  path /loki/api/v1/query_range 
  path /loki/api/v1/labels 
  path /loki/api/v1/label 
  path /loki/api/v1/label/* 
  path /loki/api/v1/tail 
  path /loki/api/v1/series
  # Ruler endpoints 
  path /loki/api/v1/rules 
  path /loki/api/v1/rules/* 
  path /api/prom/rules 
  path /api/prom/rules/* 
  path /prometheus/api/v1/rules 
  path /prometheus/api/v1/alertsCopy

• The current supported endpoints for the alert manager are the following. For more information on the endpoints, refer to the Prometheus documentation

  # Prometheus alertmanager
  path /alertmanager/* 
  # Alertmanager configuration (see Mimir doc)
  path /api/v1/alertsCopy

Troubleshooting

Refer to our troubleshooting documentation if your pusher does not support HTTP headers or if you want to reset your Grafana password.

Technical limitations

• Metrics and logs data is retained 31 days. After this period, data older than 31 days is deleted.
• The number of active metrics time series is limited to 250,000 per Cockpit by default.
• The number of active log streams is limited to 5000 per Cockpit.
• It is not yet possible to downsample metrics.

Going further

For more information about Cockpit, you can check out the following pages:

• Cockpit Documentation
• Scaleway Slack Community join the #observability-beta and the #observability channels
• Contact our support team

External links

If you are interested in learning more, you can check out the following pages:

• Introduction to Prometheus. This page gives a general introduction to what Prometheus is.
• Introduction to Promtail. Promtail is an agent that collects and forwards logs to a Grafana Loki instance.
• Introduction to Grafana Agent. Grafana Agent is an agent that embeds multiple other agents like Prometheus or Promtail.
• Pushing logs using Fluentbit. Fluentbit is another commonly used agent that collects and forwards logs.
• Pushing metrics using Prometheus's agent mode. Prometheus can be used in an "agent-mode" that can scrape local targets and forward them using remote-write. This is useful for scenarios where targets are exposed in a private network (like a VPN or a single host) but need to be forwarded outside of it.
• Prometheus remote write configuration. This page gives the exact configuration parameters that can be used to configure Prometheus's remote-write.
• Prometheus data model. This page explains the Prometheus data model to help understand the concepts of metrics, time series, labels and samples, and how they work together.
• Article on Prometheus label cardinality and performance. When working with metrics, the cardinality, which is the number time series for a given metric, is extremely important. This article explains why.
• Integrations supported by Grafana Agent. Grafana Agent has many other embedded agents that can be used to monitor various software, like PostgreSQL or MongoDB. All of them can be found on the left menu of this page.
• Introduction to alerting using Prometheus
• How to create a Mimir or Loki managed alerting rule using Grafana
• How to manage alerting rules using Grafana
• How to manage contact points using Grafana
• How to manage notification policies using Grafana
• How to manage silences using Grafana.

Cockpits

Datasources

Tokens

Alert Contact Points

Managed Alerts

Grafana Users

Pricing Plans

Product Dashboards