HomeObservabilityCockpitHow to
Send metrics and logs to your Cockpit
Jump toUpdate content

How to send metrics and logs to your Cockpit

Reviewed on 16 August 2023Published on 31 October 2022

This page shows you how to send metrics and logs to your Cockpit. The Observability Cockpit provides you with 2 Remote Write endpoints to be able to push metrics and logs. You can push metrics with any Prometheus Remote Write compatible agent such as the Prometheus, Grafana or OpenTelemetry Collector agents. You can push logs with any Loki compatible agent such as the Promtail, Fluentd, Fluent Bit or Logstash agents.

Security & Identity (IAM):

You may need certain IAM permissions to carry out some actions described on this page. This means:

  • you are the Owner of the Scaleway Organization in which the actions will be carried out, or
  • you are an IAM user of the Organization, with a policy granting you the necessary permission sets
Requirements:
Important:
  • Having the default configuration on your agents might lead to more of your resources’ metrics being sent, a high consumption and a high bill at the end of the month.
  • Sending metrics and logs for Scaleway resources or personal data using an external path is a billable feature. In addition, any data that you push yourself is billed, even if you send data from Scaleway products. Refer to the product pricing for more information.
  1. Click Cockpit in the Observability section of the console side menu. The Cockpit overview page displays.

  2. Generate a token from the APIs and Tokens tab.

  3. Check Metrics and Logs for the Push permission when selecting your token permissions, so your token can write logs and metrics.

    Important:

    Once you have created your token, your secret key displays only once. Make sure you copy and save it before you exit the page.

  4. Configure your applications to send metrics and logs to your Cockpit endpoints, using your newly-created token for authentication.

    Important:
    • The Prometheus Remote Write endpoint to push your metrics is the following: https://metrics.cockpit.fr-par.scw.cloud/api/v1/push
    • The Remote Write endpoint to push your logs is the following: https://logs.cockpit.fr-par.scw.cloud/loki/api/v1/push

    Example: send Python logs to your Scaleway Cockpit on a new logs dashboard

    We need to install logging_loki using the command pip install python-logging-loki.

    The following output displays:

    snippet_python_to_scw_cockpit.py
    import logging
    import logging_loki
    handler = logging_loki.LokiHandler(
    url="https://logs.cockpit.fr-par.scw.cloud/loki/api/v1/push",
    tags={"job": "logs_from_python"},
    auth=("api_key", "YOUR_COCKPIT_API_KEY_WITH_PUSH_LOGS_AUTHORIZATION"),
    version="1",
    )
    logger = logging.getLogger("my-first-python-logger")
    logger.addHandler(handler)
    logger.error(
    "Logging a python error with Scaleway cockpit example",
    extra={"tags": {"service": "my-service"}},
    )
    Tip:

    If you are using the Scaleway Loki API, you can POST through the Authorization: Bearer <token> header, or POST directly using the following endpoint: https://api_key:YOUR_COCKPIT_API_KEY@logs.cockpit.fr-par.scw.cloud/loki/api/v1/push

    Python logs are sent to your Cockpit from the Scaleway Loki API. Complete the following steps to retrieve them on the Scaleway Cockpit with Grafana:

    • Choose a Grafana logs dashboard.
    • Log into Grafana using your Grafana credentials.
    • Click the Dashboards icon, then click Browse.
    • Click New in the top right corner of your screen and Import to import your Grafana dashboard.
    • Load the URL for the Grafana logs dashboard or enter 13639 in the ID field.
    • You can view your log line from the Python snippet.
  5. Log in to your Grafana account.

  6. Create your Grafana dashboard and start monitoring your data.

See Also