How to send metrics and logs to your Cockpit
This page shows you how to send metrics and logs to your Cockpit. The Observability Cockpit provides you with 2 Remote Write endpoints to be able to push metrics and logs. You can push metrics with any Prometheus Remote Write
compatible agent such as the Prometheus, Grafana or OpenTelemetry Collector agents.
You can push logs with any Loki compatible agent such as the Promtail, Fluentd, Fluent Bit or Logstash agents.
You may need certain IAM permissions to carry out some actions described on this page. This means:
- you are the Owner of the Scaleway Organization in which the actions will be carried out, or
- you are an IAM user of the Organization, with a policy granting you the necessary permission sets
- You have an account and are logged into the Scaleway console
- You have activated your Cockpit
- Having the default configuration on your agents might lead to more of your resources’ metrics being sent, a high consumption and a high bill at the end of the month.
- Sending metrics and logs for Scaleway resources or personal data using an external path is a billable feature. In addition, any data that you push yourself is billed, even if you send data from Scaleway products. Refer to the product pricing for more information.
-
Click Cockpit in the Observability section of the console side menu. The Cockpit overview page displays.
-
Generate a token from the APIs and Tokens tab.
-
Check Metrics and Logs for the Push permission when selecting your token permissions, so your token can write logs and metrics.
Important:Once you have created your token, your secret key displays only once. Make sure you copy and save it before you exit the page.
-
Configure your applications to send metrics and logs to your Cockpit endpoints, using your newly-created token for authentication.
Important:- The
Prometheus Remote Write
endpoint to push your metrics is the following:https://metrics.cockpit.fr-par.scw.cloud/api/v1/push
- The Remote Write endpoint to push your logs is the following:
https://logs.cockpit.fr-par.scw.cloud/loki/api/v1/push
Example: send Python logs to your Scaleway Cockpit on a new logs dashboardWe need to install logging_loki using the command
pip install python-logging-loki
.The following output displays:
snippet_python_to_scw_cockpit.pyimport loggingimport logging_lokihandler = logging_loki.LokiHandler(url="https://logs.cockpit.fr-par.scw.cloud/loki/api/v1/push",tags={"job": "logs_from_python"},auth=("api_key", "YOUR_COCKPIT_API_KEY_WITH_PUSH_LOGS_AUTHORIZATION"),version="1",)logger = logging.getLogger("my-first-python-logger")logger.addHandler(handler)logger.error("Logging a python error with Scaleway cockpit example",extra={"tags": {"service": "my-service"}},)Tip:If you are using the Scaleway Loki API, you can
POST
through theAuthorization: Bearer <token>
header, orPOST
directly using the following endpoint:https://api_key:YOUR_COCKPIT_API_KEY@logs.cockpit.fr-par.scw.cloud/loki/api/v1/push
Python logs are sent to your Cockpit from the Scaleway Loki API. Complete the following steps to retrieve them on the Scaleway Cockpit with Grafana:
- Choose a Grafana logs dashboard.
- Log into Grafana using your Grafana credentials.
- Click the Dashboards icon, then click Browse.
- Click New in the top right corner of your screen and Import to import your Grafana dashboard.
- Load the URL for the Grafana logs dashboard or enter
13639
in the ID field. - You can view your log line from the Python snippet.
- The
-
Create your Grafana dashboard and start monitoring your data.
To go further, find out how to configure the Grafana agent, push your metrics and logs and visualize them on your dashboard.