How to use your Managed Inference deployment with a Private Network

Reviewed on July 16, 2025

In this tutorial, we guide you through the process of attaching a Private Network to your Managed Inference deployment. This can be done during the initial setup or added later to an existing deployment. Using a Private Network for communications between the Instances hosting your applications and your Managed Inference deployment ensures secure communication between resources, with low latency.

Before you start

To complete the actions presented below, you must have:

A Scaleway account logged into the console
Owner status or IAM permissions allowing you to perform actions in the intended Organization
A Managed Inference deployment

How to attach a Private Network to a Managed Inference deployment

Attaching a Private Network during model deployment

Click Managed Inference in the AI section of the Scaleway console side menu. A list of your deployments displays.
From the drop-down menu, select the geographical region where you want to deploy.
Navigate to the Deployments section and click Deploy a model. The setup wizard displays.
During the setup, in the step Configure network connectivity, check the Attach to Private Network box.
Choose one of the following options:
- Attach an existing Private Network: Select from the list of available networks.
- Create a new Private Network: Choose this option (at the bottom of the list) to create a new Private Network to attach the model to.
*Complete the deployment setup process and click Deploy model.

Your Managed Inference model will be deployed, and it will be attached to the selected Private Network.

Attaching a Private Network to an existing deployment

Click Managed Inference in the AI section of the Scaleway console side menu. A list of your deployments displays.
From the drop-down menu, select the geographical region you want to manage.
Click a deployment name or more icon > More info to access the deployment dashboard.
Go to the Overview tab and locate the Endpoints section.
Click Attach Private Network. Two options are available:
- Attach an existing Private Network: Select from the list of available networks.
- Create a new Private Network: Choose this option from the end of the list to create a new Private Network to attach the model to.
Tip
Alternatively, you can access the Security tab and attach a network from the Private Network section.
Click Attach to Private Network to confirm.

Verifying the Private Network connection

After attaching a Private Network, go to the Security tab.
You should see the Private Network connected to the deployment resource and its allocated IPs listed.

How to send inference requests in a Private Network

Note

For more information on managing access to deployments in a Private Network, see How to manage access to deployments.

Create an Instance which will host the inference application.

Important
Ensure the Instance is attached to a Private Network in the same VPC as your Managed Inference deployment.
Download the TLS certificate from your Managed Inference deployment, available from the Overview tab in the Endpoints section.
Transfer the TLS certificate to the Instance. You can use the scp (secure copy) command to securely transfer the certificate from your local machine to the Scaleway Instance.
- Example command:
```
scp -i ~/.ssh/id_ed25519 /home/user/certs/cert_file.pem root@51.115.xxx.yyy:/root
```
- Replace placeholders in the command above as follows:
  - -i ~/.ssh/id_ed25519: Path to your private SSH key.
  - /home/user/certs/cert_file.pem: Path to the certificate file on your local machine.
  - root: Your Scaleway Instance username (root for default configuration).
  - 51.115.xxx.yyy: Public IP address of your Scaleway Instance.
  - :/root: Destination directory on the Scaleway Instance.
Note
Ensure the destination directory exists and you have write permissions. Create the directory if necessary.
Connect to your Instance using SSH.
Open a text editor and create a file named inference.py using the following command:
```
nano inference.py
```

Paste the following Python code sample into your inference.py file:

Tip

This script takes an example of a conversational task with a request sent to an LLM through the Chat Completions API.

import requests

PAYLOAD = {
    "model": "<MODEL_DEPLOYED>", # EXAMPLE= meta/llama-3.1-8b-instruct:fp8
    "messages": [
        {"role": "system",
        "content": "You are a helpful, respectful and honest assistant."},
        {"role": "user",
        "content": "How can I use large language models to improve customer service?"}
    ],
    "max_tokens": 500,
    "temperature": 0.7,
    "stream": False
}

headers = {"Authorization": "Bearer " + "<SCW_SECRET_KEY>"} # ADD API KEY, IF API KEY AUTHENTICATION IS ENABLED FOR THE PRIVATE ENDPOINT

response = requests.post("<PRIVATE_ENDPOINT_URL>/v1/chat/completions",
                        headers=headers, json=PAYLOAD, stream=False, verify='<CERT_NAME>.pem')

if response.status_code == requests.codes.ok:
    # EXTRACT RESPONSE DATA
    data = response.json()
    content = data['choices'][0]['message']['content']
    print(content)
else:
    print("Error occurred:", response.text)

Edit the script as follows:

PAYLOAD: Update the model name and inference parameters.
headers: Add your IAM secret key if IAM authentication is enabled.
response: Update with your private endpoint URL and certificate file path.

Save your changes using CONTROL+O, then exit with CONTROL+X.
Make your script executable using the following command:
```
chmod +x inference.py
```
Run the script:
```
python3 inference.py
```

Detaching a Private Network from a Managed Inference deployment

Click Managed Inference in the AI section of the Scaleway console side menu. A list of your deployments displays.
Click a deployment name or more icon > More info to access the deployment dashboard.
Go to the Overview tab and locate the Endpoints section.
Click Detach Private Network. A pop-up displays.
Click Detach resource to confirm the removal of the private endpoint for your deployment.
Tip
Alternatively, you can access the Security tab and detach a network from the Private Network section.

Important
Managed Inference deployments must have at least one endpoint, either public or private. Consequently, users cannot detach their private endpoint without having a public one.