---
title: How to configure custom alerts in Grafana
description: Learn how to configure custom alerts for Scaleway resources in Grafana. Follow the steps to create alert rules, define conditions, and set up notifications for your monitored resources.
dates:
validation: 2026-01-26
posted: 2023-11-06
---
import Requirements from '@macros/iam/requirements.mdx'
import AdvancedOptionsGrafana from './assets/scaleway-advanced-options.webp'
import DataSourceManaged from './assets/scaleway-datasource-managed.webp'
import MetricsExplorer from './assets/scaleway-metrics-explorer.webp'
import FiringAlertTabs from './assets/scaleway-firing-alert-tabs.webp'
import AlertsFiringGrafana from './assets/scaleway-alert-firing.webp'
import AlertsViaGrafanaUI from './assets/scaleway-alerts-via-grafana-ui.webp'
Cockpit does not support the Grafana alert manager nor Grafana-managed alert rules. You should only use the Grafana interface to define alert rules, not to evaluate or receive alert notifications. Alerting rules are evaluated at the data source level and notifications are handled and sent out by the Scaleway alert manager only.
Once the conditions of your alert rule are met, the rule evaluation engine of your data source forwards the firing alert to the Scaleway alert manager, which then sends a notification to the contacts you have configured in the Scaleway console or in Grafana.
You can also create alerting rules on both Scaleway data sources or your custom data sources.
This page shows you how to create alert rules in Grafana for monitoring Scaleway resources integrated with Cockpit, such as Object Storage or Kubernetes. These alerts rely on Scaleway-provided metrics, which are available in the **Metrics browser** drop-down when using the **Scaleway Metrics data source** in the Grafana interface. This page explains how to use the `Scaleway Metrics` data source, interpret metrics, set alert conditions, and activate alerts.
- A Scaleway account logged into the [console](https://console.scaleway.com)
- [Owner](/iam/concepts/#owner) status or [IAM permissions](/iam/concepts/#permission) allowing you to perform actions in the intended Organization
- Scaleway resources you can monitor
- [Enabled](/cockpit/how-to/enable-alert-manager/) the Scaleway alert manager in the same region as the resources you want to be alerted for
- [Added](/cockpit/how-to/enable-alert-manager/#how-to-add-contacts) contacts in the Scaleway console or contact points in Grafana (with the `Scaleway Alerting` alert manager of the same region as your `Scaleway Metrics` data source), otherwise alerts will not be delivered
- Read the [dedicated documentation about limits](/cockpit/reference-content/cockpit-limitations/#limits-for-recording-rules-and-alerts) to ensure your alert condition contains valid parameters
## Interactive demo
Find out how to configure custom alerts in Grafana using our interactive demo.
## Switch to the data source-managed tab
Data source managed alert rules allow you to configure alerts managed by the data source of your choice, instead of using Grafana's managed alerting system **which is not supported by Cockpit**.
1. [Log in to Grafana](/cockpit/how-to/access-grafana-and-managed-dashboards/) using your Scaleway credentials.
2. Click the Grafana icon in the top left side of your screen to open the menu.
3. Click the arrow next to **Alerting** on the left-side menu, then click **Alert rules**.
4. Click **+ New alert rule**.
5. Enter a name for your alert.
6. In the **Define query and alert condition** section, toggle **Advanced options**.
7. Select the **Scaleway Metrics** data source in the drop-down.
8. In the **Rule type** subsection, click the **Data source-managed** tab.
Data source managed alert rules allow you to configure alerts managed by the data source of your choice, instead of using Grafana's managed alerting system **which is not supported by Cockpit**.
This step is **mandatory** because Cockpit does not support Grafana’s built-in alerting system, but only alerts configured and evaluated by the data source itself.
## Define your metric and alert conditions
Switch between the tabs below to create alerts for an Object Storage bucket, a Kubernetes cluster Pod, or Cockpit logs ingestion.
The steps below explain how to create the metric selection and configure an alert condition that triggers when **the object count in your bucket exceeds a specific threshold**.
1. In the query field next to the **Loading metrics... >** button, paste the following query. Make sure that the values for the labels you have selected (for example, `resource_id` and `region`) correspond to those of the target resource.
```bash
object_storage_bucket_objects_total{region="fr-par", resource_id="my-bucket"} > 2000
```
The `object_storage_bucket_objects_total` metric indicates the total number of objects stored in a given Object Storage bucket. It is useful to monitor and control object growth in your bucket and avoid hitting thresholds.
2. In the **Set alert evaluation behavior** section, specify how long the condition must be met before triggering the alert.
3. Enter a name in the **Namespace** and **Group** fields to categorize and manage your alert rules. Rules that share the same group will use the same configuration, including the evaluation interval which determines how often the rule is evaluated (by default: every 1 minute). You can modify this interval later in the group settings.
The evaluation interval is different from the pending period set in step 2. The evaluation interval controls how often the rule is checked, while the pending period defines how long the condition must be continuously met before the alert fires.
4. In the **Configure labels and notifications** section, click **+ Add labels**. A pop-up appears.
5. Enter a label and value name and click **Save**. You can skip this step if you want your alerts to be sent to the contacts you may already have created in the Scaleway console.
In Grafana, notifications are sent by matching alerts to notification policies based on labels. This step is about deciding how alerts will reach you or your team (Slack, email, etc.) based on labels you attach to them. Then, you can set up rules that define who receives notifications in the **Notification policies** page.
For example, if an alert has the label `team = object-storage-team`, you are telling Grafana to send a notification to the Object Storage team when your alert is firing. Find out how to [configure notification policies in Grafana](/tutorials/configure-slack-alerting/#configuring-a-notification-policy).
6. Click **Save rule and exit** in the top right corner of your screen to save and activate your alert.
7. Optionally, check that your configuration works by temporarily lowering the threshold. This will trigger the alert and notify your [contacts](/cockpit/concepts/#contact-points).
The steps below explain how to create the metric selection and configure an alert condition that triggers when **no new Pod activity occurs, which could mean your cluster is stuck or unresponsive.**
1. In the query field next to the **Loading metrics... >** button, paste the following query. Make sure that the values for the labels you have selected (for example, `resource_name`) correspond to those of the target resource.
```bash
rate(kubernetes_cluster_k8s_shoot_nodes_pods_usage_total{resource_name="k8s-par-quizzical-chatelet"}[15m]) == 0
```
The `kubernetes_cluster_k8s_shoot_nodes_pods_usage_total` metric represents the total number of Pods currently running across all nodes in your Kubernetes cluster. It is helpful to monitor current Pod consumption per node pool or cluster, and help track resource saturation or unexpected workload spikes.
2. In the **Set alert evaluation behavior** field, specify how long the condition must be true before triggering the alert.
3. Enter a name in the **Namespace** and **Group** fields to categorize and manage your alert rules. Rules that share the same group will use the same configuration, including the evaluation interval which determines how often the rule is evaluated (by default: every 1 minute). You can modify this interval later in the group settings.
The evaluation interval is different from the pending period set in step 2. The evaluation interval controls how often the rule is checked, while the pending period defines how long the condition must be continuously met before the alert fires.
4. In the **Configure labels and notifications** section, click **+ Add labels**. A pop-up appears.
5. Enter a label and value name and click **Save**. You can skip this step if you want your alerts to be sent to the contacts you may already have created in the Scaleway console.
In Grafana, notifications are sent by matching alerts to notification policies based on labels. This step is about deciding how alerts will reach you or your team (Slack, email, etc.) based on labels you attach to them. Then, you can set up rules that define who receives notifications in the **Notification policies** page.
For example, if an alert has the label `team = kubernetes-team`, you are telling Grafana to send a notification to the Kubernetes team when your alert is firing. Find out how to [configure notification policies in Grafana](/tutorials/configure-slack-alerting/#configuring-a-notification-policy).
6. Click **Save rule and exit** in the top right corner of your screen to save and activate your alert.
7. Optionally, check that your configuration works by temporarily lowering the threshold. This will trigger the alert and notify your [contacts](/cockpit/concepts/#contact-points).
The steps below explain how to create the metric selection and configure an alert condition that triggers when **no logs are stored for 5 minutes, which may indicate your app or system is broken**.
1. In the query field next to the **Loading metrics... >** button, paste the following query. Make sure that the values for the labels you have selected (for example, `resource_name`) correspond to those of the target resource.
```bash
observability_cockpit_loki_chunk_store_stored_chunks_total:increase5m{resource_id="xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"} == 0
```
The `observability_cockpit_loki_chunk_store_stored_chunks_total:increase5m` metric represents the number of chunks (log storage blocks) that have been written over the last 5 minutes for a specific resource. It is useful to monitor log ingestion activity and detect issues such as a crash of the logging agent, or your application not producing logs.
2. In the **Set alert evaluation behavior** field, specify how long the condition must be true before triggering the alert.
3. Enter a name in the **Namespace** and **Group** fields to categorize and manage your alert rules. Rules that share the same group will use the same configuration, including the evaluation interval which determines how often the rule is evaluated (by default: every 1 minute). You can modify this interval later in the group settings.
The evaluation interval is different from the pending period set in step 2. The evaluation interval controls how often the rule is checked, while the pending period defines how long the condition must be continuously met before the alert fires.
4. In the **Configure labels and notifications** section, click **+ Add labels**. A pop-up appears.
5. Enter a label and value name and click **Save**. You can skip this step if you want your alerts to be sent to the contacts you may already have created in the Scaleway console.
In Grafana, notifications are sent by matching alerts to notification policies based on labels. This step is about deciding how alerts will reach you or your team (Slack, email, etc.) based on labels you attach to them. Then, you can set up rules that define who receives notifications in the **Notification policies** page.
For example, if an alert has the label `team = cockpit-team`, you are telling Grafana to send a notification to the Cockpit team when your alert is firing. Find out how to [configure notification policies in Grafana](/tutorials/configure-slack-alerting/#configuring-a-notification-policy).
6. Click **Save rule and exit** in the top right corner of your screen to save and activate your alert.
7. Optionally, check that your configuration works by temporarily lowering the threshold. This will trigger the alert and notify your [contacts](/cockpit/concepts/#contact-points).
You can also build your PromQL expressions and find the right metrics from the Grafana-managed tab, with the metrics explorer (book icon) and **+ Operations** button. However, remember to click the **Data source-managed** tab once you are done building them to proceed with the final configuration steps.
## View firing alerts
1. [Log in to Grafana](/cockpit/how-to/access-grafana-and-managed-dashboards/) using your Scaleway credentials.
2. Click the Grafana icon in the top left side of your screen to open the menu.
3. Click the arrow next to **Alerting** on the left-side menu, then click **Alert rules**.
4. Click the **Firing** tab under the **State** section to filter for firing rules.
5. Click the **Alert** tab under the **Rule type** section to filter for alerting rules.
Your firing alerts should display.
Find out how to send Cockpit's alert notifications to Slack using a webhook URL in our [dedicated documentation](/tutorials/configure-slack-alerting/).