Content-based messaging systems are a subset of the publish/subscribe model, and contrast with topic-based systems in terms of the way messages are filtered/routed. In a content-based messaging system, the subscriber specifies the properties of the messages they want to receive, based on the message’s attributes or content. Message delivery is therefore selective, and messages are only delivered to a subscriber if the attributes or content match the constraints they set.
Content-based deduplication is a setting available for FIFO queues. Enable content-based deduplication if the message body is guaranteed to be unique for each message. A unique hash value is generated from the body of each message, which is used as its deduplication ID. This avoids the need to set a deduplication ID when sending messages. Read more in our dedicated documentation on creating queues.
Credentials give services and platforms access to Scaleway Messaging and Queuing, enabling them to connect to the host system. Credentials are protocol-specific: for SQS and SNS, different levels of permissions can be defined to write, read, or manage queues/topics, whereas NATS credentials give full read-write-manage access. Refer to our Messaging and Queuing additional content for more information.
A Dead Letter Queue (DLQ), or undelivered-message queue, receives and holds messages that cannot be delivered to their destination queues. This concept is not yet implemented for Scaleway Messaging and Queuing.
Fanout is a type of messaging pattern. A fanout exchange broadcasts messages to all queues/consumers it is aware of. This allows the same published message to be consumed by different consumers, who will process it in different ways. Each message is processed in the order in which it arrives.
FIFO stands for First In First Out, and represents a type of queue or topic where the exact order of messages is preserved, and duplicate messages are not tolerated. As well as these specificities, FIFO queues and topics support all the same features as the Standard type. Consider using FIFO queues and topics for any use cases where the order of messages is critical, such as e-commerce order management systems, systems where one action should not happen until another has been completed, or first-come-first-served ticketing systems.
In a topic-based system, where topics handle the logic, filtering is similar to routing. Messages are sent to defined topics, which can be thought of as filters in so far as subscribers can subscribe only to the topics they are interested in. In a content-based system, filtering is carried out more directly by subscribers, who define filters for messages based on the content/attributes they want to receive.
Long polling is a technology where the client requests information from the server without expecting an immediate response. In SQS, this enables clients to wait for the system to get messages that are not immediately available.
A message broker is a piece of software that allows applications, systems and services to communicate with each other and send/receive data. It facilitates the exchange of information by receiving messages from a producer, and transmitting them to a consumer. All communication with producers and consumers uses a protocol. There are two basic models of communication for message brokers: publish/subscribe and queuing.
The message retention period is a setting that can be configured for an SQS queue. It represents the length of time (in seconds) that messages are kept in the queue before being deleted. Setting a longer message retention period allows for a longer interval between a message being sent and it being received. Read more in our dedicated documentation on creating queues.
Scaleway’s Messaging and Queuing product is a message broker tool that allows you to transfer messages between different microservices and platforms. This enables them to “talk” to each other effectively even if they are not otherwise compatible. Messaging and Queuing enables and simplifies microservices application development and allows you to build highly scalable, reliable, distributed applications.
A messaging protocol defines a structured way for users / platforms / services / applications to exchange data and messages, even if normally they do not “speak the same language”. Protocols also describe how messages should be processed, prioritized, managed and routed. Messaging protocols currently supported by Scaleway Messaging and Queuing include NATS, SQS and SNS.
Messaging and Queuing namespaces are now deprecated. Previously, you had to create a namespace, set to either NATS or SQS/SNS protocol, to set a scope for your queues, topics, and credentials. Now, you simply activate SQS and/or SNS and your credentials and queues are scoped to the Project it is activated in. For NATS, namespaces have been replaced by NATS accounts, which set the scope for your queues and credentials.
The Neural Autonomic Transport System, or NATS, is an open-source messaging system written in Go. It is part of the Cloud Native Computing Foundation (CNCF) and has more than 40 client language implementations. The application has been designed with performance, scalability, and ease of use in mind.
Check our our NATS quickstart to get started with Scaleway Messaging and Queuing using NATS, or our tutorial on creating a serverless architecture to process large messages with NATS, to get an idea of how to go further.
A NATS account sets a scope for any NATS credentials, messages, queues and streams held within it. You can create one or multiple NATS accounts with Scaleway Messaging and Queuing. NATS accounts have replaced NATS namespaces.
Scaleway offers three protocols to use with Messaging and Queuing: NATS, SQS, and SNS.
To create a message broker configured for the NATS protocol, you must create a NATS account and then create credentials for that account. The account gives you an endpoint/URL for your NATS message broker.
For a message broker configured for the SQS or SNS protocols, you must activate the required protocol, and then create credentials for the protocol. Once activated, you get an endpoint/URL for your SQS or SNS message broker. You can activate both protocols: in this case, you must create separate credentials for each one, and they will each have a separate endpoint/URL.
Also known as “pub/sub”, the publish/subscribe model provides a pattern or framework for the exchange of messages between publishers and subscribers. It contrasts with the queuing model. The key feature of publish/subscribe is that messages are not sent to defined recipients. Instead, subscribers define the types of message they are interested in, and only receive messages matching their criteria. The publisher sends the message without knowing exactly who will receive it. The process of selecting which messages to receive is called filtering, which can be topic-based or content-based. The publish/subscribe model relies on a message broker to relay messages between publishers and subscribers.
Queues facilitate asynchronous communication between different microservices, applications, and platforms. You can create a queue, configure its delivery and message parameters, and then start sending messages to it. Messages are stored in the queue until they are processed and delivered, and deleted once consumed. Read more about creating and configuring queues, or check our tutorial on creating a serverless scraping architecture using an SQS queue to get an idea of what you can do with message queues.
When creating queues with Scaleway Messaging and Queuing SQS or SNS, two queue types are available. Standard queues provide at-least-once delivery, while FIFO queues offer first-in-first-out delivery, and (unlike Standard queues) guarantee that messages are delivered in order and without duplication. Content-based deduplication is only available for FIFO queue types. Find out more about creating queues with our dedicated documentation.
The message queuing model provides a pattern or framework for sending messages, which contrasts with the publish/subscribe model. Queuing is a form of asynchronous service-to-service communication. Whereas with the publish/subscribe model multiple subscribers can receive each message, with the queuing model, messages have just one destination. Messages are stored in the queue until they are processed and delivered, and they are deleted once consumed. This model is used in serverless and microservices architectures.
Queue volume is one of the factors affecting the billing of Scaleway Messaging and Queuing with SQS. Queue volume is calculated as the number of messages in a queue, multiplied by the message size. Or, the sum of the size of all messages in a queue.
Messaging and Queuing is available in multiple regions. A region designates the geographical area where the Messaging and Queuing service is hosted. Refer to the product availability table to check which regions are available for Messaging and Queuing.
When creating a NATS account or activating SQS or SNS, you need to do this on a region-by-region basis. The region drop-down in the Messaging and Queuing page of the console allows you to switch between available regions.
Similarly, when accessing your NATS or SQS/SNS resources in the console, be aware that they are organized by region. Use the region drop-down in the Messaging and Queuing page of the console to select the desired region, before going on to click on the relevant protocol to access resources for that protocol in that region.
In topic-based messaging, topics allow messages to be routed to the correct subscribers. Topics act as labels for each message, and the broker routes messages to subscribers who match the topic.
The Simple Notification Service, or SNS, is a publish/subscribe notification service for the mass delivery of messages. SNS acts as a single message bus that can be sent to a variety of devices and platforms through a single code interface. It is also possible to adapt message formats to the particular needs of each platform.
The Simple Queue Service, or SQS, is a distributed message queuing service that supports programmatic sending of messages via web service applications. The SQS protocol offers two types of message queues:
SQS/SNS was previously a namespace type available for Scaleway Messaging and Queuing. The two protocols have now been separated, and are now activated and deactivated separately.
Standard-type queues and topics represent the default queue/topic type, and offer an at-least-once message delivery system. Unlike FIFO queues and topics, standard queues provide only best-effort attempts to deliver messages in order. At-least-once delivery means that it is possible under rare circumstances that the same message may be received more than once.
Distinct from traditional message brokers where messages are deleted once received/consumed, streams retain records of their events. A streaming broker is therefore often likened to a distributed append-only logs file, where every new message is added at the end of the persistent log. Each message can be delivered to one or more consumers.
Stream volume is one of the factors affecting the billing of Scaleway Messaging and Queuing with NATS. Stream volume is calculated as the number of messages in a stream, multiplied by the message size. Or, the sum of the size of all messages in a stream.
Stream persistence is one of the factors affecting the billing of Scaleway Messaging and Queuing with NATS. Stream persistence is calculated as the total amount stored in a stream, multiplied by the duration it is stored for.
In publish/subscribe systems such as SNS, a subscriber is the entity (e.g. SQS queue, function, URL) that messages from topics are pushed to. Subscribers can filter messages based on their topic or content.
A subscription is a connection between a client or endpoint, and a topic. By creating a subscription, the subscribed endpoint receives messages and notifications published to the topic. You can create subscriptions for HTTP/S endpoints or Scaleway Serverless Functions and Containers.
A subscription protocol refers to the communication method used to deliver messages to a subscriber. Different types of subscriber require different protocols. When you create a new subscription to an SNS topic with Scaleway Messaging and Queueing, the following options are available:
| Protocol | Endpoint / Client | Note |
|---|---|---|
| HTTP | The URL of a service or web server that can receive notifications (HTTP POST requests) from SNS, e.g. http://example.fr | - For security reasons, we recommend using the HTTPS protocol rather than HTTP. - HTTP subscriptions must be confirmed after creation |
| HTTPS | The URL of a service or web server that can receive notifications (HTTPS POST requests) from SNS, e.g. https://example.fr | - HTTPS subscriptions must be confirmed after creation |
| Serverless Functions and Containers | A Scaleway Serverless Function or Container | - It must have a public privacy policy - It must be in a namespace from the same Project and region as the SNS topic |
A topic is a communication channel used to send messages and notifications to subscribed endpoints or clients. Publishers send messages to topics, and those messages are received by subscribers. Subscribers can include Serverless Functions and HTTP/HTTPS endpoints. As such, topics decouple the publishing and the receiving of messages, allowing for flexibility and scalabilty in building loosely-coupled systems.
When creating topics with Scaleway Messaging and Queuing SQS or SNS, only one topic type is currently available: Standard. Standard topics provide at-least-once delivery. Find out more about creating topics with our dedicated documentation.
It is planned to also offer the FIFO topic type in the future, for first-in-first-out delivery, guaranteeing that messages are delivered in order and without duplication. Join the #messaging-queuing channel on the Scaleway Slack Community for updates to new features and releases.
Topic-based messaging systems are a subset of the publish/subscribe model, and contrast with content-based systems. In a topic-based system, messages are published to “topics” or named logical channels. See topic for more information.
Visibility timeout is a setting that can be configured for an SQS queue. It represents the length of time (in seconds) during which, after a message is received, the queue hides it, so it cannot be received again by the same or other consumers. This is useful as the queue itself does not automatically delete messages once they are received, and so prevents consumers from receiving the same message many times before they have finished processing it. Read more in our dedicated documentation on creating queues.