Security and Reliability in Generative APIs
This page outlines key principles and best practices to help you ensure your applications’ security and reliability when using Generative APIs.
ResilienceLink to this anchor
Resilience ensures the continuity and availability of your applications and data, even in the face of disruptions or failures. In Generative APIs, you can promote resilience through three pillars: availability, durability and performance.
Availability and durabilityLink to this anchor
Generative APIs Service Level Agreements (SLAs) target the following Service Level Objectives (SLOs):
| Processing Type | Configuration Details | Availability |
|---|---|---|
| Standard | Standard synchronous HTTP calls to Generative APIs providing the generated content directly in HTTP response. These calls include stream and non-stream requests. | 99.9% |
| Batch | Asynchronous processing of files sent to Generative APIs providing the generated content through files. | 99.9% |
The detailed SLA measurements and guarantees can be found on the Service Level Agreement for Generative APIs page.
As we do not store any data with standard processing, durability requirements do not apply.
When processing data using batch processing, your input data is stored only during processing time (24 hours):
- As input data storage is only temporary, no specific durability guarantee applies.
- Output data (processing results) durability depends on the target storage system used. The storage system used by default is the Object Storage Standard Class
PerformanceLink to this anchor
Standard processing (synchronous HTTP calls):
- Generative APIs run models on mutualized infrastructure, and therefore ensure good performance in average usage. We monitor and respond quickly to any drops in token generation throughput, but cannot strictly guarantee performance, especially during customer peak loads. As a consequence, rate limits apply, to ensure fair use of synchronous HTTP calls. Bigger volumes of requests should be treated through batch processing.
- Guaranteed performance can be provided using dedicated resources on the Managed Inference product.
Batch processing (asynchronous file processing):
- When using batch processing, we handle scheduling of batch jobs to optimize both processing resource allocation and processing time. Processing time is therefore only guaranteed to be lower than 24 hours and rate limits (larger than Standard processing) still apply.
MonitoringLink to this anchor
Monitoring is an essential pillar to ensure the security and reliability of your services. It provides real-time insights into the performance, security, and resource consumption of your Generative APIs usage.
Metrics and logsLink to this anchor
Generative APIs metrics and logs are stored inside Scaleway Cockpit.
This includes:
- Metrics: Input and output tokens and API requests. Metrics are refreshed every minute (some dashboards may aggregate data by the hour for accuracy reasons, but metrics can be queried at a finer rate using Cockpit custom dashboards)
- Logs: No logs are currently stored inside Cockpit.
Configuration and version managementLink to this anchor
Configuration and version management are critical for maintaining reliability and performance across your services.
ConfigurationLink to this anchor
Currently, Generative APIs do not provide specific configuration properties stored within your account. All configuration parameters are the ones you send through each API HTTP call (such as temperature, top_p or seed) and you remain responsible for any change in outputs based on these parameters.
Since Generative AI models are by definition non-deterministic, we cannot guarantee the same input will provide the same output over time (for example when using two different HTTP calls). If you want deterministic processing, we encourage you to use Managed Inference with a specific model and set all randomness parameters to deterministic levels (for example using for instance temperature:0 and a specific seed value).
Version managementLink to this anchor
Supported models
Any changes to supported models and associated guarantees are detailed in our model lifecycle policy page.
API versions
Two types of API version updates may be performed:
| Upgrade Type | Description |
|---|---|
| Minor | These updates do not change the API’s current fields format and are backward compatible (no action is required on your side). New fields and features can however be added. |
| Major | These updates change the API’s current fields or paths. They may require action from your side. In this case, we will notify you with at least 3 months’ warning before deprecating significant features that might break your application. |
Compatibility with third party tools
By following industry standards (such as targeting OpenAI API compatibility), we aim to provide compatibility with most AI ecosystems and tools by default. However, as ecosystems evolve quickly, we cannot always guarantee compatibility with third party tools. We do provide extensive documentation:
- Current API supported features are available in our API Documentation
- Advanced errors and edge case workarounds are provided in our Troubleshooting section.
- Information about integration with third party tools is available in our dedicated documentation page
Data protectionLink to this anchor
Our data protection measures are detailed on our privacy policy page.
- Scaleway does not store sensitive data (such as the content of your prompt), unless we need it to provide the service (such as when using batch processing)
- When data is stored, it is protected using a state of the art method (such as encryption at rest)
- During transit, your data is encrypted via the HTTPS protocol
Scaleway accessLink to this anchor
In order to perform maintenance operations and guarantee the reliability of Generative APIs, or comply with local regulations, we need to access servers hosting the Generative APIs service.
Most of the time, any actions Scaleway carries out are automatic, e.g. updating configuration or upgrading software versions.
Manual interventions might be required occasionally for troubleshooting reasons (such as specific customer requests generating errors or carrying out malicious activity). We may temporarily complete the content of HTTP requests to identify a root cause issue or any security vulnerability. All Scaleway access is authenticated and traced following industry security standards.
Identity and access managementLink to this anchor
Identity and access management allows you to enable granular control over user permissions and to mitigate the risk of unauthorized access or data breaches.
All access to Generative APIs is authenticated and authorized, relying on Scaleway IAM permissions sets.
You are responsible for attributing these permissions to the relevant users or applications, and for reviewing these accesses frequently.
ComplianceLink to this anchor
Several regulations apply to us (Scaleway) directly, whereas others apply to your usage. Even in this case, we help you ease your compliance process by providing you with the information you need from your cloud provider.
AI ActLink to this anchor
We (Scaleway) ensure our compliance with the AI Act within our scope of responsibilities. We also ensure that you have the information needed to comply with the requirements that apply to you. This means concretely:
- Gathering information from AI Model Providers about their models (such as whether their training capacity is above 10²⁵ FLOPs, and falls into a specific category), and providing you with a link to these documents when they are made available.
- Providing you with links towards licensing required by the AI Model Providers.
Scaleway has no access to, nor knowledge of any inputs and outputs generated by the models. By using our AI products, you agree and acknowledge that you are (a) responsible for this use, including any content integrated into the models, and (b) required to use the AI products in compliance with our General Terms of Services.
As a client of our AI products, you are likely to be considered an AI System Provider or Deployer under the AI Act. As such, it is your responsibility to ensure you comply with requirements that apply to you.
Additional local regulationLink to this anchor
If you require additional information to comply with specific regulations, you can create a support ticket or contact your account manager.