- Having the default configuration on your agents might lead to more of your resources’ metrics being sent, a high consumption, and a high bill at the end of the month.
- Sending metrics and logs for Scaleway resources or personal data using an external path is a billable feature. In addition, any data that you push yourself is billed, even if you send data from Scaleway products. Refer to the product pricing for more information.
How to send metrics and logs to your Cockpit
This page shows you how to send metrics and logs to your Cockpit. You can push metrics with any Prometheus Remote Write
compatible agent such as the Prometheus, Grafana or OpenTelemetry Collector agents.
You can push logs with any Loki compatible agent such as the Promtail, Fluentd, Fluent Bit or Logstash agents.
Before you start
To complete the actions presented below, you must have:
- A Scaleway account logged into the console
- Owner status or IAM permissions allowing you to perform actions in the intended Organization
- Configured an agent. Read our dedicated documentation to find out how to configure the Grafana agent.
Send metrics and logs to your Cockpit
-
Click Cockpit in the Observability section of the console side menu. The Cockpit overview page displays.
-
Create a token with the Push permission for metrics and logs.
Important- Make sure that you create your token in the same region as the data sources you want to use it for.
- Make sure that you copy your token’s secret key and save it before you exit the page.
-
Configure your applications to send metrics and logs to your Cockpit endpoints, using your newly created token for authentication.
Important- The
Prometheus Remote Write
endpoint to push your metrics is the following:https://metrics.cockpit.fr-par.scw.cloud/api/v1/push
- The Remote Write endpoint to push your logs is the following:
https://logs.cockpit.fr-par.scw.cloud/loki/api/v1/push
- The
-
Create your Grafana dashboard and start monitoring your data.
Sending Python logs to Cockpit
- Open a terminal and run
pip install python-logging-loki
orpip3 install python-logging-loki
if you are using Python 3, to install the logging_loki package to send your Python logs. - Create a token with the Push permission for logs.
- Copy and save the following code snippet into a text editor. Make sure that you replace
$SCW_SECRET_KEY
and$COCKPIT_TOKEN
with your values.
import loggingimport logging_lokiimport timehandler = logging_loki.LokiHandler(url="https://logs.cockpit.fr-par.scw.cloud/loki/api/v1/push",tags={"job": "logs_from_python"},auth=("$SCW_SECRET_KEY", "$COCKPIT_TOKEN"),version="1",)logger = logging.getLogger("my-first-python-logger")logger.addHandler(handler)def main():# Start generating log messagesfor i in range(10):logger.error(f"Error message {i}")time.sleep(1)if __name__ == "__main__":main())This script sets up a logger that sends error log messages to the Loki endpoint in Cockpit. It then generates 10 error log messages with a one-second delay between each message. These log messages are sent to Cockpit and can be viewed and analyzed via the Grafana dashboard we will create in the next part.
TipIf you are using the Scaleway Loki API, you can
POST
through theAuthorization: Bearer <token>
header, orPOST
using the following endpoint:https://api_key:YOUR_COCKPIT_TOKEN@logs.cockpit.fr-par.scw.cloud/loki/api/v1/push
. - Open a terminal and run
Creating a Grafana dashboard and visualizing your Python logs
-
Log into Grafana using your Grafana credentials.
-
Click the Home icon, then click Dashboards.
-
Click New in the top right corner of your screen and Import to import your Grafana dashboard.
-
Load the URL of the premade Grafana Log Viewer Dashboard for Loki or enter
13639
in the ID field.
Your log lines from the Python snippet should display in the dashboard.
To go further, find out how to configure the Grafana agent to push your metrics and logs, and visualize them on your dashboard.