Generative APIs - Dedicated Deployment API

Deployments

Download schema

A deployment is a scalable pool of resources used to run inference models

List inference deployments

GET

https://api.scaleway.com

/inference/v1/regions/{region}/deployments

List all your inference deployments.

List inference deployments › path Parameters

region

string · enum · required

The region you want to target

Enum values:

fr-par

List inference deployments › query Parameters

page

integer · int32

Page number to return.

page_size

integer · uint32

Maximum number of deployments to return per page.

order_by

scaleway.inference.v1.ListDeploymentsRequest.OrderBy · enum

Order in which to return results.

Enum values:

created_at_desc

created_at_asc

name_asc

name_desc

Default: created_at_desc

project_id

string

Filter by Project ID. (UUID format)

organization_id

string

Filter by Organization ID. (UUID format)

name

string

Filter by deployment name.

tags

string[]

Filter by tags.

List inference deployments › Responses

200

scaleway.inference.v1.ListDeploymentsResponse

scaleway.inference.v1.Deployment[]

List of deployments on the current page.

total_count

integer · uint64

Total number of deployments.

GET/inference/v1/regions/{region}/deployments

curl -X GET \
  -H "X-Auth-Token: $SCW_SECRET_KEY" \
  "https://api.scaleway.com/inference/v1/regions/{region}/deployments"

Example Responses

{
  "deployments": [
    {
      "id": "6170692e-7363-616c-6577-61792e636f6d",
      "name": "name",
      "project_id": "6170692e-7363-616c-6577-61792e636f6d",
      "status": "unknown_status",
      "tags": [
        "string"
      ],
      "node_type_name": "node_type_name",
      "endpoints": [
        {
          "id": "6170692e-7363-616c-6577-61792e636f6d",
          "url": "url",
          "public_network": {},
          "private_network": {
            "private_network_id": "6170692e-7363-616c-6577-61792e636f6d"
          },
          "disable_auth": true
        }
      ],
      "size": 0,
      "min_size": 0,
      "max_size": 0,
      "error_message": "error_message",
      "model_id": "6170692e-7363-616c-6577-61792e636f6d",
      "quantization": {
        "bits": 0
      },
      "model_name": "model_name",
      "created_at": "2022-03-22T12:34:56.123456Z",
      "updated_at": "2022-03-22T12:34:56.123456Z",
      "region": "region"
    }
  ],
  "total_count": 0
}

json

application/json

Create a deployment

POST

https://api.scaleway.com

/inference/v1/regions/{region}/deployments

Create a new inference deployment related to a specific model.

Create a deployment › path Parameters

region

string · enum · required

The region you want to target

Enum values:

fr-par

Create a deployment › Request Body

name

string · required

Name of the deployment.

project_id

string · required

ID of the Project to create the deployment in. (UUID format)

model_id

string · required

ID of the model to use. (UUID format)

node_type_name

string · required

Name of the node type to use.

scaleway.inference.v1.EndpointSpec[] · required

List of endpoints to create.

accept_eula

boolean | null

Accept the model's End User License Agreement (EULA). If the model has an EULA, you must accept it before proceeding. The terms of the EULA can be retrieved using the GetModelEula API call.

tags

string[]

List of tags to apply to the deployment.

min_size

integer | null · uint32

Defines the minimum size of the pool.

max_size

integer | null · uint32

Defines the maximum size of the pool. Currently, autoscaling is not yet supported, and this value must be equal to min_size.

scaleway.inference.v1.DeploymentQuantization

Quantization settings to apply to this deployment.

Create a deployment › Responses

200

scaleway.inference.v1.Deployment

id

string

Unique identifier. (UUID format)

name

string

Name of the deployment.

project_id

string

Project ID. (UUID format)

status

scaleway.inference.v1.Deployment.Status · enum

Status of the deployment.

Enum values:

unknown_status

creating

deploying

ready

error

deleting

locked

scaling

Default: unknown_status

tags

string[]

List of tags applied to the deployment.

node_type_name

string

Node type of the deployment.

scaleway.inference.v1.Endpoint[]

List of endpoints.

size

integer · uint32

Current size of the pool.

min_size

integer · uint32

Defines the minimum size of the pool.

max_size

integer · uint32

Defines the maximum size of the pool. Currently, autoscaling is not yet supported, and this value must be equal to min_size.

error_message

string | null

Displays information if your deployment is in error state.

model_id

string

ID of the model used for the deployment. (UUID format)

scaleway.inference.v1.DeploymentQuantization

Quantization parameters for this deployment.

model_name

string

Name of the deployed model.

created_at

string | null · date-time

Creation date of the deployment. (RFC 3339 format)

updated_at

string | null · date-time

Last modification date of the deployment. (RFC 3339 format)

region

string

Region of the deployment.

POST/inference/v1/regions/{region}/deployments

curl -X POST \
  -H "X-Auth-Token: $SCW_SECRET_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "endpoints": [
        ""
    ],
    "model_id": "6170692e-7363-616c-6577-61792e636f6d",
    "name": "string",
    "node_type_name": "string",
    "project_id": "6170692e-7363-616c-6577-61792e636f6d"
  }' \
  "https://api.scaleway.com/inference/v1/regions/{region}/deployments"

Example Request Body

{
  "name": "name",
  "project_id": "6170692e-7363-616c-6577-61792e636f6d",
  "model_id": "6170692e-7363-616c-6577-61792e636f6d",
  "accept_eula": true,
  "node_type_name": "node_type_name",
  "tags": [
    "string"
  ],
  "min_size": 0,
  "max_size": 0,
  "endpoints": [
    {
      "public_network": {},
      "private_network": {
        "private_network_id": "6170692e-7363-616c-6577-61792e636f6d"
      },
      "disable_auth": true
    }
  ],
  "quantization": {
    "bits": 0
  }
}

json

Example Responses

{
  "id": "6170692e-7363-616c-6577-61792e636f6d",
  "name": "name",
  "project_id": "6170692e-7363-616c-6577-61792e636f6d",
  "status": "unknown_status",
  "tags": [
    "string"
  ],
  "node_type_name": "node_type_name",
  "endpoints": [
    {
      "id": "6170692e-7363-616c-6577-61792e636f6d",
      "url": "url",
      "public_network": {},
      "private_network": {
        "private_network_id": "6170692e-7363-616c-6577-61792e636f6d"
      },
      "disable_auth": true
    }
  ],
  "size": 0,
  "min_size": 0,
  "max_size": 0,
  "error_message": "error_message",
  "model_id": "6170692e-7363-616c-6577-61792e636f6d",
  "quantization": {
    "bits": 0
  },
  "model_name": "model_name",
  "created_at": "2022-03-22T12:34:56.123456Z",
  "updated_at": "2022-03-22T12:34:56.123456Z",
  "region": "region"
}

json

application/json

Get a deployment

GET

https://api.scaleway.com

/inference/v1/regions/{region}/deployments/{deployment_id}

Get the deployment for the given ID.

Get a deployment › path Parameters

region

string · enum · required

The region you want to target

Enum values:

fr-par

deployment_id

string · required

ID of the deployment to get. (UUID format)

Get a deployment › Responses

200

scaleway.inference.v1.Deployment

id

string

Unique identifier. (UUID format)

name

string

Name of the deployment.

project_id

string

Project ID. (UUID format)

status

scaleway.inference.v1.Deployment.Status · enum

Status of the deployment.

Enum values:

unknown_status

creating

deploying

ready

error

deleting

locked

scaling

Default: unknown_status

tags

string[]

List of tags applied to the deployment.

node_type_name

string

Node type of the deployment.

scaleway.inference.v1.Endpoint[]

List of endpoints.

size

integer · uint32

Current size of the pool.

min_size

integer · uint32

Defines the minimum size of the pool.

max_size

integer · uint32

Defines the maximum size of the pool. Currently, autoscaling is not yet supported, and this value must be equal to min_size.

error_message

string | null

Displays information if your deployment is in error state.

model_id

string

ID of the model used for the deployment. (UUID format)

scaleway.inference.v1.DeploymentQuantization

Quantization parameters for this deployment.

model_name

string

Name of the deployed model.

created_at

string | null · date-time

Creation date of the deployment. (RFC 3339 format)

updated_at

string | null · date-time

Last modification date of the deployment. (RFC 3339 format)

region

string

Region of the deployment.

GET/inference/v1/regions/{region}/deployments/{deployment_id}

curl -X GET \
  -H "X-Auth-Token: $SCW_SECRET_KEY" \
  "https://api.scaleway.com/inference/v1/regions/{region}/deployments/{deployment_id}"

Example Responses

{
  "id": "6170692e-7363-616c-6577-61792e636f6d",
  "name": "name",
  "project_id": "6170692e-7363-616c-6577-61792e636f6d",
  "status": "unknown_status",
  "tags": [
    "string"
  ],
  "node_type_name": "node_type_name",
  "endpoints": [
    {
      "id": "6170692e-7363-616c-6577-61792e636f6d",
      "url": "url",
      "public_network": {},
      "private_network": {
        "private_network_id": "6170692e-7363-616c-6577-61792e636f6d"
      },
      "disable_auth": true
    }
  ],
  "size": 0,
  "min_size": 0,
  "max_size": 0,
  "error_message": "error_message",
  "model_id": "6170692e-7363-616c-6577-61792e636f6d",
  "quantization": {
    "bits": 0
  },
  "model_name": "model_name",
  "created_at": "2022-03-22T12:34:56.123456Z",
  "updated_at": "2022-03-22T12:34:56.123456Z",
  "region": "region"
}

json

application/json

Delete a deployment

DELETE

https://api.scaleway.com

/inference/v1/regions/{region}/deployments/{deployment_id}

Delete an existing inference deployment.

Delete a deployment › path Parameters

region

string · enum · required

The region you want to target

Enum values:

fr-par

deployment_id

string · required

ID of the deployment to delete. (UUID format)

Delete a deployment › Responses

200

scaleway.inference.v1.Deployment

id

string

Unique identifier. (UUID format)

name

string

Name of the deployment.

project_id

string

Project ID. (UUID format)

status

scaleway.inference.v1.Deployment.Status · enum

Status of the deployment.

Enum values:

unknown_status

creating

deploying

ready

error

deleting

locked

scaling

Default: unknown_status

tags

string[]

List of tags applied to the deployment.

node_type_name

string

Node type of the deployment.

scaleway.inference.v1.Endpoint[]

List of endpoints.

size

integer · uint32

Current size of the pool.

min_size

integer · uint32

Defines the minimum size of the pool.

max_size

integer · uint32

Defines the maximum size of the pool. Currently, autoscaling is not yet supported, and this value must be equal to min_size.

error_message

string | null

Displays information if your deployment is in error state.

model_id

string

ID of the model used for the deployment. (UUID format)

scaleway.inference.v1.DeploymentQuantization

Quantization parameters for this deployment.

model_name

string

Name of the deployed model.

created_at

string | null · date-time

Creation date of the deployment. (RFC 3339 format)

updated_at

string | null · date-time

Last modification date of the deployment. (RFC 3339 format)

region

string

Region of the deployment.

DELETE/inference/v1/regions/{region}/deployments/{deployment_id}

curl -X DELETE \
  -H "X-Auth-Token: $SCW_SECRET_KEY" \
  "https://api.scaleway.com/inference/v1/regions/{region}/deployments/{deployment_id}"

Example Responses

{
  "id": "6170692e-7363-616c-6577-61792e636f6d",
  "name": "name",
  "project_id": "6170692e-7363-616c-6577-61792e636f6d",
  "status": "unknown_status",
  "tags": [
    "string"
  ],
  "node_type_name": "node_type_name",
  "endpoints": [
    {
      "id": "6170692e-7363-616c-6577-61792e636f6d",
      "url": "url",
      "public_network": {},
      "private_network": {
        "private_network_id": "6170692e-7363-616c-6577-61792e636f6d"
      },
      "disable_auth": true
    }
  ],
  "size": 0,
  "min_size": 0,
  "max_size": 0,
  "error_message": "error_message",
  "model_id": "6170692e-7363-616c-6577-61792e636f6d",
  "quantization": {
    "bits": 0
  },
  "model_name": "model_name",
  "created_at": "2022-03-22T12:34:56.123456Z",
  "updated_at": "2022-03-22T12:34:56.123456Z",
  "region": "region"
}

json

application/json

Update a deployment

PATCH

https://api.scaleway.com

/inference/v1/regions/{region}/deployments/{deployment_id}

Update an existing inference deployment.

Update a deployment › path Parameters

region

string · enum · required

The region you want to target

Enum values:

fr-par

deployment_id

string · required

ID of the deployment to update. (UUID format)

Update a deployment › Request Body

name

string | null

Name of the deployment.

tags

array | null

List of tags to apply to the deployment.

min_size

integer | null · uint32

Defines the new minimum size of the pool.

max_size

integer | null · uint32

Defines the maximum size of the pool. Currently, autoscaling is not yet supported, and this value must be equal to min_size.

model_id

string | null

Id of the model to set to the deployment.

scaleway.inference.v1.DeploymentQuantization

Quantization to use to the deployment.

Update a deployment › Responses

200

scaleway.inference.v1.Deployment

id

string

Unique identifier. (UUID format)

name

string

Name of the deployment.

project_id

string

Project ID. (UUID format)

status

scaleway.inference.v1.Deployment.Status · enum

Status of the deployment.

Enum values:

unknown_status

creating

deploying

ready

error

deleting

locked

scaling

Default: unknown_status

tags

string[]

List of tags applied to the deployment.

node_type_name

string

Node type of the deployment.

scaleway.inference.v1.Endpoint[]

List of endpoints.

size

integer · uint32

Current size of the pool.

min_size

integer · uint32

Defines the minimum size of the pool.

max_size

integer · uint32

Defines the maximum size of the pool. Currently, autoscaling is not yet supported, and this value must be equal to min_size.

error_message

string | null

Displays information if your deployment is in error state.

model_id

string

ID of the model used for the deployment. (UUID format)

scaleway.inference.v1.DeploymentQuantization

Quantization parameters for this deployment.

model_name

string

Name of the deployed model.

created_at

string | null · date-time

Creation date of the deployment. (RFC 3339 format)

updated_at

string | null · date-time

Last modification date of the deployment. (RFC 3339 format)

region

string

Region of the deployment.

PATCH/inference/v1/regions/{region}/deployments/{deployment_id}

curl -X PATCH \
  -H "X-Auth-Token: $SCW_SECRET_KEY" \
  -H "Content-Type: application/json" \
  -d '{}' \
  "https://api.scaleway.com/inference/v1/regions/{region}/deployments/{deployment_id}"

Example Request Body

{
  "name": "name",
  "tags": [],
  "min_size": 0,
  "max_size": 0,
  "model_id": "model_id",
  "quantization": {
    "bits": 0
  }
}

json

Example Responses

{
  "id": "6170692e-7363-616c-6577-61792e636f6d",
  "name": "name",
  "project_id": "6170692e-7363-616c-6577-61792e636f6d",
  "status": "unknown_status",
  "tags": [
    "string"
  ],
  "node_type_name": "node_type_name",
  "endpoints": [
    {
      "id": "6170692e-7363-616c-6577-61792e636f6d",
      "url": "url",
      "public_network": {},
      "private_network": {
        "private_network_id": "6170692e-7363-616c-6577-61792e636f6d"
      },
      "disable_auth": true
    }
  ],
  "size": 0,
  "min_size": 0,
  "max_size": 0,
  "error_message": "error_message",
  "model_id": "6170692e-7363-616c-6577-61792e636f6d",
  "quantization": {
    "bits": 0
  },
  "model_name": "model_name",
  "created_at": "2022-03-22T12:34:56.123456Z",
  "updated_at": "2022-03-22T12:34:56.123456Z",
  "region": "region"
}

json

application/json

Get the CA certificate

GET

https://api.scaleway.com

/inference/v1/regions/{region}/deployments/{deployment_id}/certificate

Get the CA certificate used for the deployment of private endpoints. The CA certificate will be returned as a PEM file.

Get the CA certificate › path Parameters

region

string · enum · required

The region you want to target

Enum values:

fr-par

deployment_id

string · required

(UUID format)

Get the CA certificate › Responses

200

scaleway.std.File

name

string

content_type

string

content

string

Models Node types