LLM Inference

Serve Generative AI models and answer prompts from European end-consumers securely.

Choose among ready-to-be-served LLMs

What makes inference fast? Model optimization is one lever. To be served fast, a model must be optimized to the machines that run it.
This isn't always a piece of cake, and can turn into a time-consuming process. That's why Scaleway is providing an evolutionary Model Library, with curated and optimized LLMs.

Benefit from a dedicated H100-PCIe cluster

H100 PCIe GPU Instances excel in handling rigorous model serving tasks. Leveraging advanced data formats and its innovative transformer Engine, the H100 PCIe achieves a 30-fold improvement in inference speed over its predecessor, the NVIDIA A100 GPU.

Run on a fully secured European Cloud

Enjoy tailored security for your infrastructure: from highly secure VPC environments to accessible setups with internet and IAM tokens.
Maintain complete data control: no storage nor third-party access to your data (prompt & responses), ensuring it remains exclusively yours and within Europe.

Available zones:
Paris:PAR 2

State-of-the-art open weights LLMs


Trained on Scaleway's Nabuchodonosor 2023, Mixtral-8x7B is a state-of-the-art, pretrained generative model known as a Sparse Mixture of Experts. It has been benchmarked to surpass the performance of the Llama 2 70B model across a variety of tests.

Benefit from a secured European Cloud ecosystem

Virtual Private Cloud

Your LLM endpoints are accessible through low-latency and secure connection to your resources hosted at Scaleway, thanks to a resilient regional Private Network.

Learn more

Access Management

We make generative AI endpoints compatible with Scaleway's Identity and Access Management, so that your deployments are compliant with your enterprise architecture requirements.

Learn more


Identify bottlenecks on your deployments, view inference requests in real time and even report your energy consumption with a fully managed observability solution.

Learn more
  • Scaleway is a NVIDIA Elite Partner