Configuring a SQL storage backend in Helm on Scaleway Kubernetes Kapsule

When deploying large Helm releases on Scaleway Kapsule Kubernetes, you might encounter errors like the following:

rpc error: code = ResourceExhausted desc = etcd quota exceeded: size 55000000/55000000, burst 424242/100000000

This happens because Helm stores release information in Kubernetes Secrets or ConfigMaps, which are limited to 1 MB due to etcd storage constraints.

To resolve this issue, you can configure Helm to use an SQL storage backend such as PostgreSQL, bypassing the etcd size limitations and enabling larger Helm releases.

Before you start

To complete the actions presented below, you must have:

• A Scaleway account logged into the console
• Owner status or IAM permissions allowing you to perform actions in the intended Organization
• Created a Kubernetes Kapsule cluster

Setting up a PostgreSQL database

You can set up a PostgreSQL database using one of the two options below.

Option A: Scaleway's managed PostgreSQL Service

1. Create a PostgreSQL instance using Scaleway's Managed Database service to create a PostgreSQL database.
2. Whitelist your cluster's IPs to allow your Kubernetes cluster to connect to the database.
3. Note down the connection details. Save the host, port, database name, username, and password for later use.

Option B: Deploy PostgreSQL in Kubernetes

1. Deploy PostgreSQL using Helm in your cluster:
   helm repo add bitnami https://charts.bitnami.com/bitnami
   helm install helm-postgres bitnami/postgresql \
       --set auth.username=helm,auth.password=yourpassword,auth.database=helm_db

2. Expose the PostgreSQL service within the cluster.

Configuring Helm to use the SQL backend

To use PostgreSQL as Helm's storage backend, set the following environment variables in your terminal:

export HELM_DRIVER=sql
export HELM_DRIVER_SQL_CONNECTION_STRING="postgresql://username:password@host:port/database"

Once set, Helm will store release metadata in the specified PostgreSQL database instead of Kubernetes Secrets or ConfigMaps.

Verifying the configuration

To confirm Helm is configured correctly, run:

helm list --all-namespaces

Expected output:

• A list of Helm releases managed by the PostgreSQL backend.
• If there are no releases, ensure the connection string is correct and PostgreSQL is reachable.

If configured correctly, Helm will use the SQL backend, and you will not encounter etcd size limitations.

Migrating existing releases (if applicable)

Helm does not automatically migrate releases between storage backends. Follow these steps to manually migrate existing releases:

Backup existing releases

• Run the following command to back up your existing releases:
  kubectl get secrets --all-namespaces -l "owner=helm" -o yaml > helm-backup.yaml

Reinstall releases

• Redeploy the releases using the SQL backend:
  helm upgrade --install <release-name> <chart-name> --namespace <namespace>

  Note

  Ensure you use the same release names and namespaces.

Verify releases

• Run the following command to verify your releases:
  helm list --all-namespaces

Clean up old secrets (after confirming everything works)

• Use the following command to delete your backup (after confirming everything works):
  kubectl delete -f helm-backup.yaml

Troubleshooting common issues and solutions

Need further assistance?

Learn more about configuring Helm with an SQL storage backend by visiting the official Helm documentation on SQL storage.

If you have further questions, feel free to engage with the Scaleway Slack community to exchange insights, share experiences, and discover practical solutions.