Clusters for Apache Kafka® FAQ
Overview
What is Clusters for Apache Kafka®?
Clusters for Apache Kafka® is Scaleway's product to provide a secure, fully-managed infrastructure for your Apache Kafka® deployments. With our clusters, you can create data pipelines and event-driven architectures that run natively on Apache Kafka®, without having to manage the underlying infrastructure. This product is currently in Public Beta.
Is Scaleway's Clusters for Apache Kafka® based on the open-source Kafka framework?
Yes, Scaleway's Clusters for Apache Kafka is built on the open-source Apache Kafka framework. We use the Kafka API to ensure compatibility and leverage the robust features of the open-source ecosystem.
How is Apache Kafka® different to RabbitMQ and NATS?
Apache Kafka, RabbitMQ, and NATS are all messaging systems but serve different purposes and have distinct architectures.
- Apache Kafka is designed for high-throughput, real-time data streaming, and is topic-based, using the pull model. It can process large amounts of data in real-time with high concurrency and fault tolerance, and is ideal for stream processing, event sourcing, and big data applications.
- RabbitMQ is a versatile message broker that supports multiple protocols, and offers strong support for message delivery guarantees and reliability. It uses the push model, and is well-suited for traditional message queuing and task distribution.
- NATS is a lightweight, high-performance subject-based messaging system focused on speed and simplicity, making it great for microservices.
Each system has its own strengths and is suited to different use cases depending on the requirements of the application.
Getting started
How do I use my cluster, and how can I connect to it?
To use your cluster, you first connect via the Kafka CLI or any Kafka-supported programming language. The connection is secured, and requires the appropriate credentials, certificate, and endpoint provided by Scaleway. All configuration is done through a Kafka client, allowing you to manage topics, producers, and consumers easily. This flexibility ensures seamless integration with your applications and workflows.
Pricing and billing
How am I billed for Clusters for Apache Kafka®?
See our dedicated pricing page for information on billing.
Quotas and limitations
What limitations apply during the Public Beta?
The following limitations apply to Clusters for Apache Kafka® in its current Public Beta phase:
- Only one Kafka version is supported
- Clusters can only be provisioned with either one or three nodes (brokers)
- The number of nodes cannot be modified after cluster creation
- Connectivity to the cluster is facilitated exclusively through Private Networks
- Only one set of credentials (username and password) can be created per cluster
We aim to improve and develop the product based on feedback received during the beta stage.