How to use batch processing

Reviewed on February 17, 2026

Batch processing allows you to submit large volumes of requests to Generative APIs asynchronously, at a discounted price. Instead of sending individual requests, you upload an input file to Object Storage and create a batch job. The service processes the requests in the background and writes the results to an output file.

Before you start

To complete the actions presented below, you must have:

A Scaleway account logged into the console
Owner status or IAM permissions allowing you to perform actions in the intended Organization. These IAM permissions are GenerativeApisFullAccess, ObjectStorageObjectsRead,ObjectStorageObjectsWrite.
A valid API key for API authentication
An Object Storage bucket bucket in the same Project in which you want to run the batch job
Python 3.7+ installed on your system
Policies allowing the Scaleway-managed application principal scw-managed-genapi-batch to perform the s3:GetObject and s3:PutObject actions in your bucket. This application is auto-generated by Scaleway after you create your first batch.

How batch processing works

Upload a JSONL input file to an Object Storage bucket. This file contains all API queries to perform.
Create a batch job referencing this file.
The service processes each line as an individual request.
The results are written to an output file in the same bucket, and named {filename}_output.jsonl and {filename}_error.jsonl.
Retrieve the output file once the job status is completed.

Each line of the input file must be a valid JSON object representing a request to the Generative API.

Example input.jsonl:

Note

The JSONL file must adhere to the following requirements:

The custom_id field must be unique for each request.
The method, url, and model fields must remain consistent across all requests.
The body object must comply with the Generative API documentation.

{"custom_id": "request-1", "method": "POST", "url": "/v1/chat/completions", "body": {"model": "mistral-small-3.2-24b-instruct-2506", "messages": [{"role": "system", "content": "You are a helpful assistant."},{"role": "user", "content": "Translate this into French: Hello world!"}],"max_completion_tokens": 500}}
{"custom_id": "request-2", "method": "POST", "url": "/v1/chat/completions", "body": {"model": "mistral-small-3.2-24b-instruct-2506", "messages": [{"role": "system", "content": "You are an unhelpful assistant."},{"role": "user", "content": "Write a poem about the ocean."}],"max_completion_tokens": 500}}

Note

Batch processing can take up to 24 hours. After this timeframe, the batch job will be terminated. Any remaining requests will not be processed. You will only be billed for processed requests.

Object Storage bucket permissions

Batch processing relies on an Object Storage bucket. You can use any of your existing Object Storage buckets for batch processing or create a new one.

If your bucket has at least one bucket policy configured, you must edit this bucket policy or create a new one allowing the Scaleway-managed application principal (named scw-managed-genapi-batch) to perform the following actions:

s3:GetObject
s3:PutObject

Below is an example bucket policy:

{
  "Version": "2023-04-17",
  "Id": "allow-only-bucket",
  "Statement": [
    {
      "Sid": "scw-managed",
      "Effect": "Allow",
      "Principal": {
        "SCW": "application_id:example-4096-adea-b6e03b44a99d"
      },
      "Action": [
        "s3:GetObject",
        "s3:PutObject"
      ],
      "Resource": "my-readonly-bucket/*"
    },
    {
      "Sid": "Scaleway secure statement",
      "Effect": "Allow",
      "Principal": {
        "SCW": "user_id:example-9f9c-46b1-b2e0-ed883a6c54ce"
        },
      "Action": "*",
      "Resource": [
        "my-readonly-bucket",
        "my-readonly-bucket/*"
      ]
    }
  ]
}

Note that to configure this bucket policy, you must replace:

user_id:YOUR_IAM_USER_ID with your IAM user ID
application_id:SCW_MANAGED_GENAPI_BATCH_ID with the ID of the IAM application named scw-managed-genapi-batch. This application and its corresponding ID will be created automatically in your Organization after you initiate a first batch query. If you have bucket policies configured, you must create a first batch to obtain the scw-managed-genapi-batch IAM ID, even if the first batch fails.

Tip

The Scaleway secure statement ensures that you retain full access to your bucket and do not accidentally lock yourself out.

Creating a batch using the console

Creating the batch job

Click Generative APIs on the AI section of the side menu. The Generative APIs overview page appears.
Click the Batches tab.
Click Create batch. The batch creation wizard opens.

Important
The Object Storage bucket and the batch job must belong to the same Project.
Provide the following:
- Input file path (e.g., s3://scw-managed-genapi-batch/input.jsonl)
  
  Tip
  You can upload your input.jsonl file directly by clicking + Add file.
- Output prefix (optional)
Click Create batch. The batch job enters the Validating state.

Monitoring the job

You can monitor:

Status (Validating, In progress, Completed, Failed)
Number of processed requests
Error count

Once completed, you can download the generated output file from Object Storage.

Output format

Each line of the output file corresponds to one input request.

Example:

{"id":"93807e44-09fa-4a9f-baa3-574164651535","custom_id":"request-1","error":null,"response":{"request_id":"7a38ba8c-0fad-4923-9214-9b218a416b50","status_code":200,"body":{"id":"chatcmpl-7a38ba8c-0fad-4923-9214-9b218a416b50","object":"chat.completion","created":1771335694,"model":"mistral-small-3.2-24b-instruct-2506","choices":[{"index":0,"message":{"role":"assistant","content":"Bonjour le monde!","refusal":null,"annotations":null,"audio":null,"function_call":null,"tool_calls":[],"reasoning_content":null},"logprobs":null,"finish_reason":"stop","stop_reason":null,"token_ids":null}],"service_tier":null,"system_fingerprint":null,"usage":{"prompt_tokens":19,"total_tokens":25,"completion_tokens":6,"prompt_tokens_details":null},"prompt_logprobs":null,"prompt_token_ids":null,"kv_transfer_params":null}}}
{"id":"93807e44-09fa-4a9f-baa3-574164651535","custom_id":"request-2","error":null,"response":{"request_id":"01c2cc2c-b072-45fb-9031-2b10b75f1b2f","status_code":200,"body":{"id":"chatcmpl-01c2cc2c-b072-45fb-9031-2b10b75f1b2f","object":"chat.completion","created":1771335694,"model":"mistral-small-3.2-24b-instruct-2506","choices":[{"index":0,"message":{"role":"assistant","content":"Oh, the ocean, so vast and so wide,\nA shimmering expanse of blue tide.\nIt whispers and roars, in a rhythm so grand,\nA symphony of waves, in the ocean's band.\n\nIt's a mystery, deep and profound,\nWith secrets untold, in its watery ground.\nCreatures unseen, in its depths they reside,\nIn the ocean's embrace, they confide.\n\nIt's a force to be reckoned with, a power so great,\nA tempestuous beast, that can't be tamed, can't be sate.\nYet, it's also a cradle, a gentle, soothing song,\nA lullaby sung, to the weary and strong.\n\nSo here's to the ocean, in all its might and grace,\nA wonder of nature, in its own special place.\nMay it continue to inspire, to captivate and to amaze,\nThe ocean, the ocean, in all its ways.","refusal":null,"annotations":null,"audio":null,"function_call":null,"tool_calls":[],"reasoning_content":null},"logprobs":null,"finish_reason":"stop","stop_reason":null,"token_ids":null}],"service_tier":null,"system_fingerprint":null,"usage":{"prompt_tokens":20,"total_tokens":218,"completion_tokens":198,"prompt_tokens_details":null},"prompt_logprobs":null,"prompt_token_ids":null,"kv_transfer_params":null}}}

Creating a batch using the OpenAI Python SDK

You can also create and manage batches programmatically using the OpenAI-compatible SDK.

import os
import time
import logging
from openai import OpenAI

# Initialize client with staging base URL
client = OpenAI(
    api_key=os.getenv("SCW_SECRET_KEY"),
    base_url="https://api.scaleway.ai/v1"  # No "/batches" needed
)

# Your S3 input file URL (only path to the file without query parameters, hence not a pre-signed URL)
input_file_url = "https://<my-bucket-name>.s3.<region>.scw.cloud/input.jsonl"

# Submit the batch job
print("Submitting batch job...")
batch = client.batches.create(
    input_file_id=input_file_url,
    endpoint="/v1/chat/completions",
    completion_window="24h"
)

print(f"Batch created: {batch.id}")
print(f"Initial status: {batch.status}")

# Polling function
def wait_for_completion(batch_id, check_interval=10):
    print(f"Polling every {check_interval}s for completion...\n")

    while True:
        batch_job = client.batches.retrieve(batch_id)
        last_update_timestamp = batch_job.completed_at or batch_job.in_progress_at or batch_job.created_at
        print(f"Status: {batch_job.status.upper()} (Updated: {time.strftime('%Y-%m-%d %H:%M:%S', time.localtime(last_update_timestamp))})")

        if batch_job.status == "completed":
            print(f"\nBatch completed successfully!")
            print(f"Download results: {batch_job.output_file_id}")
            return batch_job.output_file_id

        elif batch_job.status == "failed":
            if hasattr(batch_job, 'errors') and batch_job.errors:
                print(f"Batch failed: {batch_job.errors}")
            else:
                print(f"Batch failed: Unknown error")
            return None

        elif batch_job.status in ["in_progress", "validating"]:
            time.sleep(check_interval)
        else:
            print(f"Unexpected status: {batch_job.status}")
            return None

# Start polling
wait_for_completion(batch.id)