- Having the default configuration on your agents might lead to more of your resources’ metrics being sent, a high consumption, and a high bill at the end of the month.
- Sending metrics and logs for Scaleway resources or personal data using an external path is a billable feature. In addition, any data that you push yourself is billed, even if you send data from Scaleway products. Refer to the product pricing for more information.
How to send metrics and logs to your Cockpit
This page shows you how to send data such as metrics and logs to your Cockpit. You can push metrics with any Prometheus Remote Write
compatible agent such as the Prometheus, Grafana or OpenTelemetry Collector agents.
You can push logs with any Loki compatible agent such as the Promtail, Fluentd, Fluent Bit or Logstash agents.
Before you startLink to this anchor
To complete the actions presented below, you must have:
- A Scaleway account logged into the console
- Owner status or IAM permissions allowing you to perform actions in the intended Organization
- Configured an agent. Read our dedicated documentation to find out how to configure the Grafana Alloy agent
- Created an API key and retrieved your API secret key
Send metrics and logs to your CockpitLink to this anchor
-
Click Cockpit in the Monitoring section of the console side menu. The Cockpit overview page displays.
-
Create a token with the Push permission for metrics and logs.
Important- Make sure that you create your token in the same region as the data sources you want to use it for.
- Make sure that you copy your token’s secret key and save it before you exit the page.
-
Configure your applications to send metrics and logs to your Cockpit endpoints, using your token’s secret key for authentication.
ImportantRetrieve your metrics’ and logs’ push endpoints in the Data sources tab of the Scaleway console.
-
Create your Grafana dashboard and start monitoring your data.
Sending Python logs to CockpitLink to this anchor
The section below provides you with a Python script that sends test log messages to Cockpit. It is a safe way to ensure that your custom log setup works before integrating real application logs.
- You need custom logs in the
fr-par
region to follow the procedure below. Find out how to create custom logs. - You must have created a Scaleway API key and retrieved your API secret key
- Open a terminal and run
pip install python-logging-loki
orpip3 install python-logging-loki
if you are using Python 3, to install the logging_loki package to send your Python logs. - Create a Cockpit token with the Push permission for logs.
- Create a
python_logs.py
Python file and paste the following code snippet into it. Make sure that you replace000avb0d-34ae-66hh-643b-f9e0n3k17773
with the ID of your logs data source,$SCW_API_SECRET_KEY
with your API secret key and$COCKPIT_TOKEN_SECRET_KEY
with your Cockpit token’s secret key.
import loggingimport logging_lokiimport timehandler = logging_loki.LokiHandler(url="https://000avb0d-34ae-66hh-643b-f9e0n3k17773.cockpit.fr-par.scw.cloud/loki/api/v1/push",tags={"job": "logs_from_python"},auth=("$SCW_API_SECRET_KEY", "$COCKPIT_TOKEN_SECRET_KEY"),version="1",)logger = logging.getLogger("my-first-python-logger")logger.addHandler(handler)def main():# Start generating log messagesfor i in range(10):logger.error(f"Error message {i}")time.sleep(1)if __name__ == "__main__":main()
This script sets up a logger that sends error log messages to the Loki endpoint in Cockpit. It then generates 10 error log messages with a one-second delay between each message. These log messages are sent to Cockpit and can be viewed and analyzed via the Grafana dashboard we will create in the next part.
If you are using the Scaleway Loki API, you can POST
through the Authorization: Bearer <token>
header, or POST
using the following endpoint: https://api_key:YOUR_COCKPIT_TOKEN@logs.cockpit.fr-par.scw.cloud/loki/api/v1/push
.
Creating a Grafana dashboard and visualizing your Python logsLink to this anchor
-
Log into Grafana using your Grafana credentials.
-
Click Toggle menu, then click Dashboards.
-
Click New in the top right corner of your screen, then Import.
-
Upload the JSON file of the Grafana Log Viewer dashboard for Loki or enter
13639
in the Import via grafana.com field. -
Click Load.
-
Enter a name for the dashboard or leave the default name.
-
Select a folder in which to store the dashboard.
-
Select your custom logs data source in the Loki drop-down. Make sure that the data source you select is in the
fr-par
region. -
Click Import.
-
Open a terminal and enter the following command.
python3 python_logs.py -
Click the Home icon > Dashboards. Your Logs / App dashboard should display.
-
Click the Logs / App dashboard. You should see 10 log lines labeled
Error message 0
throughError message 9
, one per second. This confirms that your setup is working.If your logs do not display, try changing the time range to match the time at which you ran your Python file.
-
Click the View in Explore button for more configuration options.
Read our more in depth-documentation on how to configure the Grafana alloy agent to push your Scaleway metrics, and visualize them in a Grafana dashboard.