NavigationContentFooter

Cockpit Regional API

Introduction

Scaleway's Cockpit allows you to monitor your applications and their infrastructure by giving you insights and context into their behavior. Cockpit also enables you to visualize your metrics, logs, and traces through a Grafana dashboard. With Cockpit, you can also push data from your Scaleway resources, if they are integrated with Cockpit.

The Observability Cockpit provides you with two Prometheus Remote Write endpoints for pushing metrics and logs. You can push metrics with any Prometheus Remote Write compatible agent such as Prometheus, Grafana or OpenTelemetry Collector.

You can push logs with any Loki compatible agent such as Promtail, Fluentd, Fluent Bit or Logstash.

Note

This API concerns the Cockpit service.

Concepts

Refer to our dedicated concepts page to find definitions of the different terms referring to the Observability Cockpit.

Quickstart

  1. Configure your environment variables.

    Note

    This is an optional step that seeks to simplify your usage of the API.

    export SCW_SECRET_KEY="<API secret key>"
    export SCW_PROJECT_ID="<Scaleway Project ID>
  2. Build push URLs.

    Retrieve your endpoints for metrics, logs, and traces from the Data sources tab of the Scaleway console. Your endpoints should look like the following:

    {
    [...],
    "endpoints": {
    "metrics_url": "https://xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx.metrics.cockpit.fr-par.scw.cloud",
    "logs_url": "https://xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx.logs.cockpit.fr-par.scw.cloud",
    "alertmanager_url": "https://alertmanager.cockpit.fr-par.scw.cloud",
    "grafana_url": "<project_id>.dashboard.obs.fr-par.scw.cloud",
    "traces_url": "https://xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx.traces.cockpit.fr-par.scw.cloud"
    },
    [...]
    }

    The metrics_url is a domain that exposes a Prometheus-like API to manage metrics, and the logs_url exposes a Loki API to manage logs.

    Important

    To be able to send metrics, logs, and traces, you need the exact URL to use with your pushers. You can find the URL from the Data sources tab of the Scaleway console.

    • The Prometheus Remote Write endpoint to push your metrics is the following: https://xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx.metrics.cockpit.fr-par.scw.cloud/api/v1/push
    • The Remote Write endpoint to push your logs is the following: https://xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx.logs.cockpit.fr-par.scw.cloud/loki/api/v1/push
    • The Prometheus Alertmanager endpoint is the following: https://alertmanager.cockpit.fr-par.scw.cloud/alertmanager
    • Traces have different endpoints depending on the push path you use. The traces endpoint for the OpenTelemetry agent is: https://xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx..cockpit.fr-par.scw.cloud/otlp.
  3. Create your token to push metrics, logs, and traces.

    Find out how to create your token via the console or run the following command to create your token via the API.

    curl -X POST \
    "https://api.scaleway.com/cockpit/v1beta1/tokens" \
    -H "Content-Type: application/json" \
    -H "X-Auth-Token: $SCW_SECRET_KEY" \
    -d '{
    "project_id": "00000000-0000-0000-0000-000000000000",
    "name": "token-name",
    "scopes": {
    "query_metrics": false,
    "write_metrics": true,
    "setup_metrics_rules": false,
    "query_logs": false,
    "write_logs": true,
    "setup_logs_rules": false,
    "setup_alerts": false,
    "query_traces": false,
    "write_traces": true
    }
    }'
    Important

    Your token's secret_key only displays once. Make sure that you save it. We strongly recommend that you only give the minimal amount of permissions to your token.

  4. Configure the Grafana agent.

    Find out how to configure, start the Grafana agent and see your metrics and logs in our documentation.

    Note

    The promtail configuration for the logs does not support custom headers. The tenant_id field corresponds to the header X-Scope-OrgID which is one of the supported headers. For more details on the different supported headers, see our troubleshooting documentation for when your pusher does not support custom HTTP headers.

  5. Configure rules and alerts.

    Upon creation, your Grafana instance is already configured by default with your logs and metrics data sources and with the alert manager data source. This means that you will be able to configure and manage advanced alerting rules, for metrics and logs, configure contact points and notification policies, from Grafana.

    To do so, access your Grafana dashboard and read our dedicated documentation to find out how to configure alert rules for Scaleway resources. Refer to the External links section of this page if you are not familiar with the alert manager or alert rules.

    Important

    For the alerts and rules to be created in your Cockpit, you must use the data sources provided by Scaleway.

    Important

    Make sure that you select Scaleway Alerting as your alert manager when configuring as the alerts are managed by the platform. If you use Grafana you will work with the local alert manager which is deactivated.

Requirements
  • You have a Scaleway account
  • You have created an API key and that the API key has sufficient IAM permissions to perform the actions described on this page
  • You have installed curl
  • You have installed docker
  • You have installed docker compose
  • You have a Linux machine to run the container on. You can still run the container on macOS or Windows, but your logs will not be available in your Cockpit.

Technical information

Regional availability

Scaleway's infrastructure is spread across different regions and Availability Zones. Your Cockpit resources are available in the Paris, Amsterdam and Warsaw regions.

Current supported endpoints

  • The current supported endpoints for metrics are the following. For more information about the endpoints, refer to the Mimir documentation

    # Remote write endpoints
    path /api/v1/push
    # Query endpoints
    path /prometheus/api/v1/query
    path /prometheus/api/v1/query_range
    path /prometheus/api/v1/query_exemplars
    path /prometheus/api/v1/series
    path /prometheus/api/v1/labels
    path /prometheus/api/v1/label/*
    path /prometheus/api/v1/metadata
    path /prometheus/api/v1/read
    path /prometheus/api/v1/status/buildinfo
    # Ruler endpoints
    path /prometheus/api/v1/rules
    path /prometheus/api/v1/alerts
    path /prometheus/config/v1/rules
    path /prometheus/config/v1/rules/*
  • The current supported endpoints for logs are the following. For more information about the endpoints, refer to the Loki documentation

    # Remote write endpoints
    path /loki/api/v1/push
    # Query endpoints
    path /loki/api/v1/query
    path /loki/api/v1/query_range
    path /loki/api/v1/labels
    path /loki/api/v1/label
    path /loki/api/v1/label/*
    path /loki/api/v1/tail
    path /loki/api/v1/series
    # Ruler endpoints
    path /loki/api/v1/rules
    path /loki/api/v1/rules/*
    path /api/prom/rules
    path /api/prom/rules/*
    path /prometheus/api/v1/rules
    path /prometheus/api/v1/alerts
  • The current supported endpoints for the alert manager are the following. For more information on the endpoints, refer to the Prometheus documentation

    # Prometheus alertmanager
    path /alertmanager/*
    # Alertmanager configuration (see Mimir doc)
    path /api/v1/alerts

Troubleshooting

Refer to our troubleshooting documentation if your pusher does not support HTTP headers or if you want to reset your Grafana password.

Technical limitations

  • Metrics and logs data is retained 31 days. After this period, data older than 31 days is deleted.
  • The number of active metrics time series is limited to 250,000 per Cockpit by default.
  • The number of active log streams is limited to 5000 per Cockpit.
  • It is not yet possible to downsample metrics.

Going further

For more information about Cockpit, you can check out the following pages:

  • Cockpit Documentation
  • Scaleway Slack Community join the #observability-beta and the #observability channels
  • Contact our support team

If you are interested in learning more, you can check out the following pages:

  • Introduction to Prometheus. This page gives a general introduction to what Prometheus is.
  • Introduction to Promtail. Promtail is an agent that collects and forwards logs to a Grafana Loki instance.
  • Introduction to Grafana Agent. Grafana Agent is an agent that embeds multiple other agents like Prometheus or Promtail.
  • Pushing logs using Fluentbit. Fluentbit is another commonly used agent that collects and forwards logs.
  • Pushing metrics using Prometheus's agent mode. Prometheus can be used in an "agent-mode" that can scrape local targets and forward them using remote-write. This is useful for scenarios where targets are exposed in a private network (like a VPN or a single host) but need to be forwarded outside of it.
  • Prometheus remote write configuration. This page gives the exact configuration parameters that can be used to configure Prometheus's remote-write.
  • Prometheus data model. This page explains the Prometheus data model to help understand the concepts of metrics, time series, labels and samples, and how they work together.
  • Article on Prometheus label cardinality and performance. When working with metrics, the cardinality, which is the number time series for a given metric, is extremely important. This article explains why.
  • Integrations supported by Grafana Agent. Grafana Agent has many other embedded agents that can be used to monitor various software, like PostgreSQL or MongoDB. All of them can be found on the left menu of this page.
  • Introduction to alerting using Prometheus
  • How to create a Mimir or Loki managed alerting rule using Grafana
  • How to manage alerting rules using Grafana
  • How to manage contact points using Grafana
  • How to manage notification policies using Grafana
  • How to manage silences using Grafana.

Data sources

Data sources are regional backends that allow you to store and query metrics, logs and traces Data sources for Scaleway metrics and logs are enabled by default if you are using Scaleway resources which are integrated into Cockpit. You can create additional data sources to store and query metrics, logs and traces from external sources.

GET
/cockpit/v1/regions/{region}/data-sources
POST
/cockpit/v1/regions/{region}/data-sources
GET
/cockpit/v1/regions/{region}/data-sources/{data_source_id}
PATCH
/cockpit/v1/regions/{region}/data-sources/{data_source_id}
DELETE
/cockpit/v1/regions/{region}/data-sources/{data_source_id}
GET
/cockpit/v1/regions/{region}/usage-overview

Tokens

Tokens are secret keys that allow you to authenticate against your Cockpit’s regional data sources and Alert manager. Make sure that you create your tokens in the same regions as the data sources you want to use them for

GET
/cockpit/v1/regions/{region}/tokens
POST
/cockpit/v1/regions/{region}/tokens
GET
/cockpit/v1/regions/{region}/tokens/{token_id}
DELETE
/cockpit/v1/regions/{region}/tokens/{token_id}

Alert manager

The Alert manager endpoints allow you to manage Cockpit's alerting services by region Once you have enabled the Alert manager in your desired regions, you can create contact points and enable managed alerts. Your contact points will be notified when your Scaleway resources in the same regions as the Alert manager, may require your attention.

GET
/cockpit/v1/regions/{region}/alert-manager
GET
/cockpit/v1/regions/{region}/alert-manager/contact-points
POST
/cockpit/v1/regions/{region}/alert-manager/contact-points
POST
/cockpit/v1/regions/{region}/alert-manager/contact-points/delete
POST
/cockpit/v1/regions/{region}/alert-manager/disable
POST
/cockpit/v1/regions/{region}/alert-manager/enable
POST
/cockpit/v1/regions/{region}/alert-manager/managed-alerts/disable
POST
/cockpit/v1/regions/{region}/alert-manager/managed-alerts/enable
POST
/cockpit/v1/regions/{region}/alert-manager/trigger-test-alert
GET
/cockpit/v1/regions/{region}/managed-alerts
Docs APIScaleway consoleDedibox consoleScaleway LearningScaleway.comPricingBlogCarreer
© 2023-2024 – Scaleway