Generative APIs - Dedicated Deployment API

Models

A model represents a pre-trained machine learning model that can be deployed on the Generative APIs - Dedicated Deployment service.

They are used to define the inference model, its source, and its compatibility with the available nodes. Some models may be available in multiple parameters sizes, which will affect the performance and the accuracy of the model.

List models

GET

https://api.scaleway.com

/inference/v1/regions/{region}/models

List all available models.

List models › path Parameters

region

string · enum · required

The region you want to target

Enum values:

fr-par

List models › query Parameters

order_by

string · enum

Order in which to return results.

Enum values:

display_rank_asc

created_at_asc

created_at_desc

name_asc

name_desc

Default: display_rank_asc

page

integer · int32

Page number to return.

page_size

integer · uint32

Maximum number of models to return per page.

project_id

string

Filter by Project ID. (UUID format)

organization_id

string

Filter by Organization ID. (UUID format)

name

string

Filter by model name.

tags

string[]

Filter by tags.

List models › Responses

200

scaleway.inference.v1.ListModelsResponse

scaleway.inference.v1.Model[]

List of models on the current page.

total_count

integer · uint64

Total number of models.

GET/inference/v1/regions/{region}/models

curl -X GET \
  -H "X-Auth-Token: $SCW_SECRET_KEY" \
  "https://api.scaleway.com/inference/v1/regions/{region}/models"

Example Responses

{
  "models": [
    {
      "id": "6170692e-7363-616c-6577-61792e636f6d",
      "name": "name",
      "project_id": "6170692e-7363-616c-6577-61792e636f6d",
      "tags": [
        "string"
      ],
      "status": "unknown_status",
      "description": "description",
      "error_message": "error_message",
      "has_eula": true,
      "created_at": "2022-03-22T12:34:56.123456Z",
      "updated_at": "2022-03-22T12:34:56.123456Z",
      "region": "region",
      "nodes_support": [
        {
          "nodes": [
            {
              "node_type_name": "node_type_name",
              "quantizations": [
                {
                  "quantization_bits": 0,
                  "allowed": true,
                  "max_context_size": 0
                }
              ]
            }
          ]
        }
      ],
      "parameter_size_bits": 0,
      "size_bytes": 0
    }
  ],
  "total_count": 0
}

json

application/json

Import a model

POST

https://api.scaleway.com

/inference/v1/regions/{region}/models

Import a new model to your model library.

Import a model › path Parameters

region

string · enum · required

The region you want to target

Enum values:

fr-par

Import a model › Request Body

name

string · required

Name of the model.

project_id

string · required

ID of the Project to import the model in. (UUID format)

object · required

Where to import the model from.

Import a model › Responses

200

scaleway.inference.v1.Model

id

string

Unique identifier. (UUID format)

name

string

Unique Name identifier.

project_id

string

Project ID. (UUID format)

tags

string[]

List of tags applied to the model.

status

string · enum

Status of the model.

Enum values:

unknown_status

preparing

downloading

ready

error

Default: unknown_status

description

string

Purpose of the model.

error_message

string

Displays information if your model is in error state.

has_eula

boolean

Defines whether the model has an end user license agreement.

created_at

string · date-time

Creation date of the model. (RFC 3339 format)

updated_at

string · date-time

Last modification date of the model. (RFC 3339 format)

region

string

Region of the model.

scaleway.inference.v1.ModelSupportInfo[]

Supported nodes types with quantization options and context lengths.

parameter_size_bits

integer · uint32

Size, in bits, of the model parameters.

size_bytes

integer · uint64

Total size, in bytes, of the model files.

POST/inference/v1/regions/{region}/models

curl -X POST \
  -H "X-Auth-Token: $SCW_SECRET_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "string",
    "project_id": "6170692e-7363-616c-6577-61792e636f6d",
    "source": {
        "secret": "string",
        "url": "string"
    }
  }' \
  "https://api.scaleway.com/inference/v1/regions/{region}/models"

Example Request Body

{
  "name": "name",
  "project_id": "6170692e-7363-616c-6577-61792e636f6d",
  "source": {
    "url": "url",
    "secret": "secret"
  }
}

json

Example Responses

{
  "id": "6170692e-7363-616c-6577-61792e636f6d",
  "name": "name",
  "project_id": "6170692e-7363-616c-6577-61792e636f6d",
  "tags": [
    "string"
  ],
  "status": "unknown_status",
  "description": "description",
  "error_message": "error_message",
  "has_eula": true,
  "created_at": "2022-03-22T12:34:56.123456Z",
  "updated_at": "2022-03-22T12:34:56.123456Z",
  "region": "region",
  "nodes_support": [
    {
      "nodes": [
        {
          "node_type_name": "node_type_name",
          "quantizations": [
            {
              "quantization_bits": 0,
              "allowed": true,
              "max_context_size": 0
            }
          ]
        }
      ]
    }
  ],
  "parameter_size_bits": 0,
  "size_bytes": 0
}

json

application/json

Get a model

GET

https://api.scaleway.com

/inference/v1/regions/{region}/models/{model_id}

Get the model for the given ID.

Get a model › path Parameters

region

string · enum · required

The region you want to target

Enum values:

fr-par

model_id

string · required

ID of the model to get. (UUID format)

Get a model › Responses

200

scaleway.inference.v1.Model

id

string

Unique identifier. (UUID format)

name

string

Unique Name identifier.

project_id

string

Project ID. (UUID format)

tags

string[]

List of tags applied to the model.

status

string · enum

Status of the model.

Enum values:

unknown_status

preparing

downloading

ready

error

Default: unknown_status

description

string

Purpose of the model.

error_message

string

Displays information if your model is in error state.

has_eula

boolean

Defines whether the model has an end user license agreement.

created_at

string · date-time

Creation date of the model. (RFC 3339 format)

updated_at

string · date-time

Last modification date of the model. (RFC 3339 format)

region

string

Region of the model.

scaleway.inference.v1.ModelSupportInfo[]

Supported nodes types with quantization options and context lengths.

parameter_size_bits

integer · uint32

Size, in bits, of the model parameters.

size_bytes

integer · uint64

Total size, in bytes, of the model files.

GET/inference/v1/regions/{region}/models/{model_id}

curl -X GET \
  -H "X-Auth-Token: $SCW_SECRET_KEY" \
  "https://api.scaleway.com/inference/v1/regions/{region}/models/{model_id}"

Example Responses

{
  "id": "6170692e-7363-616c-6577-61792e636f6d",
  "name": "name",
  "project_id": "6170692e-7363-616c-6577-61792e636f6d",
  "tags": [
    "string"
  ],
  "status": "unknown_status",
  "description": "description",
  "error_message": "error_message",
  "has_eula": true,
  "created_at": "2022-03-22T12:34:56.123456Z",
  "updated_at": "2022-03-22T12:34:56.123456Z",
  "region": "region",
  "nodes_support": [
    {
      "nodes": [
        {
          "node_type_name": "node_type_name",
          "quantizations": [
            {
              "quantization_bits": 0,
              "allowed": true,
              "max_context_size": 0
            }
          ]
        }
      ]
    }
  ],
  "parameter_size_bits": 0,
  "size_bytes": 0
}

json

application/json

Delete a model

DELETE

https://api.scaleway.com

/inference/v1/regions/{region}/models/{model_id}

Delete an existing model from your model library.

Delete a model › path Parameters

region

string · enum · required

The region you want to target

Enum values:

fr-par

model_id

string · required

ID of the model to delete. (UUID format)

Delete a model › Responses

No data returned

Get a model EULA

GET

https://api.scaleway.com

/inference/v1/regions/{region}/models/{model_id}/eula

Get the EULA for the given model ID.

Get a model EULA › path Parameters

region

string · enum · required

The region you want to target

Enum values:

fr-par

model_id

string · required

ID of the model to get the Eula for. (UUID format)

Get a model EULA › Responses

200

scaleway.inference.v1.Eula

content

string

Content of the end user license agreement.

Verify a model

POST

https://api.scaleway.com

/inference/v1/regions/{region}/verify-model

Verify a model should be good to be deployed on Generative APIs - Dedicated Deployment.

Verify a model › path Parameters

region

string · enum · required

The region you want to target

Enum values:

fr-par

Verify a model › Request Body

object · required

Where to verify the model from.

Verify a model › Responses

200

scaleway.inference.v1.VerifyModelResponse

scaleway.inference.v1.ModelSupportedNode[]

size_bytes

integer · uint64

Deployments

List models

GET

https://api.scaleway.com

/inference/v1/regions/{region}/models

List all available models.

List models › path Parameters

region

string · enum · required

The region you want to target

Enum values:

fr-par

List models › query Parameters

order_by

string · enum

Order in which to return results.

Enum values:

display_rank_asc

created_at_asc

created_at_desc

name_asc

name_desc

Default: display_rank_asc

page

integer · int32

Page number to return.

page_size

integer · uint32

Maximum number of models to return per page.

project_id

string

Filter by Project ID. (UUID format)

organization_id

string

Filter by Organization ID. (UUID format)

name

string

Filter by model name.

tags

string[]

Filter by tags.

List models › Responses

200

scaleway.inference.v1.ListModelsResponse

scaleway.inference.v1.Model[]

List of models on the current page.

total_count

integer · uint64

Total number of models.

Import a model

POST

https://api.scaleway.com

/inference/v1/regions/{region}/models

Import a new model to your model library.

Import a model › path Parameters

region

string · enum · required

The region you want to target

Enum values:

fr-par

Import a model › Request Body

name

string · required

Name of the model.

project_id

string · required

ID of the Project to import the model in. (UUID format)

object · required

Where to import the model from.

Import a model › Responses

200

scaleway.inference.v1.Model

id

string

Unique identifier. (UUID format)

name

string

Unique Name identifier.

project_id

string

Project ID. (UUID format)

tags

string[]

List of tags applied to the model.

status

string · enum

Status of the model.

Enum values:

unknown_status

preparing

downloading

ready

error

Default: unknown_status

description

string

Purpose of the model.

error_message

string

Displays information if your model is in error state.

has_eula

boolean

Defines whether the model has an end user license agreement.

created_at

string · date-time

Creation date of the model. (RFC 3339 format)

updated_at

string · date-time

Last modification date of the model. (RFC 3339 format)

region

string

Region of the model.

scaleway.inference.v1.ModelSupportInfo[]

Supported nodes types with quantization options and context lengths.

parameter_size_bits

integer · uint32

Size, in bits, of the model parameters.

size_bytes

integer · uint64

Total size, in bytes, of the model files.

Get a model

GET

https://api.scaleway.com

/inference/v1/regions/{region}/models/{model_id}

Get the model for the given ID.

Get a model › path Parameters

region

string · enum · required

The region you want to target

Enum values:

fr-par

model_id

string · required

ID of the model to get. (UUID format)

Get a model › Responses

200

scaleway.inference.v1.Model

id

string

Unique identifier. (UUID format)

name

string

Unique Name identifier.

project_id

string

Project ID. (UUID format)

tags

string[]

List of tags applied to the model.

status

string · enum

Status of the model.

Enum values:

unknown_status

preparing

downloading

ready

error

Default: unknown_status

description

string

Purpose of the model.

error_message

string

Displays information if your model is in error state.

has_eula

boolean

Defines whether the model has an end user license agreement.

created_at

string · date-time

Creation date of the model. (RFC 3339 format)

updated_at

string · date-time

Last modification date of the model. (RFC 3339 format)

region

string

Region of the model.

scaleway.inference.v1.ModelSupportInfo[]

Supported nodes types with quantization options and context lengths.

parameter_size_bits

integer · uint32

Size, in bits, of the model parameters.

size_bytes

integer · uint64

Total size, in bytes, of the model files.

Delete a model

DELETE

https://api.scaleway.com

/inference/v1/regions/{region}/models/{model_id}

Delete an existing model from your model library.

Delete a model › path Parameters

region

string · enum · required

The region you want to target

Enum values:

fr-par

model_id

string · required

ID of the model to delete. (UUID format)

Delete a model › Responses

No data returned

Get a model EULA

GET

https://api.scaleway.com

/inference/v1/regions/{region}/models/{model_id}/eula

Get the EULA for the given model ID.

Get a model EULA › path Parameters

region

string · enum · required

The region you want to target

Enum values:

fr-par

model_id

string · required

ID of the model to get the Eula for. (UUID format)

Get a model EULA › Responses

200

scaleway.inference.v1.Eula

content

string

Content of the end user license agreement.

Verify a model

POST

https://api.scaleway.com

/inference/v1/regions/{region}/verify-model

Verify a model should be good to be deployed on Generative APIs - Dedicated Deployment.

Verify a model › path Parameters

region

string · enum · required

The region you want to target

Enum values:

fr-par

Verify a model › Request Body

object · required

Where to verify the model from.

Verify a model › Responses

200

scaleway.inference.v1.VerifyModelResponse

scaleway.inference.v1.ModelSupportedNode[]

size_bytes

integer · uint64