Copy page

Commands

Documentation for `scw inference`

This API allows you to handle your Generative APIs - Dedicated Deployment services.

Deployment commands

Deployment commands.

Create a deployment

Create a new inference deployment related to a specific model.

Usage:

Code
scw inference deployment create [arg=value ...]

Arguments:

Name	Description	Argument Specifications
name	Name of the deployment	Required Default: <generated>
project-id	Project ID to use. If none is passed the default project ID will be used
model-id	ID of the model to use	Required
accept-eula	Accept the model's End User License Agreement (EULA).
node-type-name	Name of the node type to use	Required
tags.{index}	List of tags to apply to the deployment
min-size	Defines the minimum size of the pool
max-size	Defines the maximum size of the pool. Currently, autoscaling is not yet supported, and this value must be equal to min_size
endpoints.{index}.public-network	Set the endpoint as public
endpoints.{index}.is-public	Will configure your public endpoint if true	Default: true
endpoints.{index}.private-network.private-network-id
endpoints.{index}.disable-auth	Disable the authentication on the endpoint.	Default: false
quantization.bits	The number of bits each model parameter should be quantized to. The quantization method is chosen based on this value.
region	Region to target. If none is passed will use default region from the config	Default: fr-par One of: fr-par

Delete a deployment

Delete an existing inference deployment.

Usage:

Code
scw inference deployment delete <deployment-id ...> [arg=value ...]

Arguments:

Name	Description	Argument Specifications
deployment-id	ID of the deployment to delete	Required
region	Region to target. If none is passed will use default region from the config	Default: fr-par One of: fr-par

Get a deployment

Get the deployment for the given ID.

Usage:

Code
scw inference deployment get <deployment-id ...> [arg=value ...]

Arguments:

Name	Description	Argument Specifications
deployment-id	ID of the deployment to get	Required
region	Region to target. If none is passed will use default region from the config	Default: fr-par One of: fr-par

Get the CA certificate

Get the CA certificate used for the deployment of private endpoints. The CA certificate will be returned as a PEM file.

Usage:

Code
scw inference deployment get-certificate [arg=value ...]

Arguments:

Name	Description	Argument Specifications
deployment-id		Required
region	Region to target. If none is passed will use default region from the config	Default: fr-par One of: fr-par

List inference deployments

List all your inference deployments.

Usage:

Code
scw inference deployment list [arg=value ...]

Arguments:

Name	Description	Argument Specifications
order-by	Order in which to return results	One of: created_at_desc, created_at_asc, name_asc, name_desc
project-id	Filter by Project ID
name	Filter by deployment name
tags.{index}	Filter by tags
organization-id	Filter by Organization ID
region	Region to target. If none is passed will use default region from the config	Default: fr-par One of: fr-par, all

Update a deployment

Update an existing inference deployment.

Usage:

Code
scw inference deployment update <deployment-id ...> [arg=value ...]

Arguments:

Name	Description	Argument Specifications
deployment-id	ID of the deployment to update	Required
name	Name of the deployment
tags.{index}	List of tags to apply to the deployment
min-size	Defines the new minimum size of the pool
max-size	Defines the maximum size of the pool. Currently, autoscaling is not yet supported, and this value must be equal to min_size
model-id	Id of the model to set to the deployment
quantization.bits	The number of bits each model parameter should be quantized to. The quantization method is chosen based on this value.
region	Region to target. If none is passed will use default region from the config	Default: fr-par One of: fr-par

Endpoint management commands

Endpoint management commands.

Create an endpoint

Create a new Endpoint related to a specific deployment.

Usage:

Code
scw inference endpoint create <deployment-id ...> [arg=value ...]

Arguments:

Name	Description	Argument Specifications
deployment-id	ID of the deployment to create the endpoint for	Required
endpoint.public-network	Set the endpoint as public
endpoint.private-network.private-network-id
endpoint.disable-auth	Disable the authentication on the endpoint.	Default: false
region	Region to target. If none is passed will use default region from the config	Default: fr-par One of: fr-par

Delete an endpoint

Delete an existing Endpoint.

Usage:

Code
scw inference endpoint delete <endpoint-id ...> [arg=value ...]

Arguments:

Name	Description	Argument Specifications
endpoint-id	ID of the endpoint to delete	Required
region	Region to target. If none is passed will use default region from the config	Default: fr-par One of: fr-par

Update an endpoint

Update an existing Endpoint.

Usage:

Code
scw inference endpoint update <endpoint-id ...> [arg=value ...]

Arguments:

Name	Description	Argument Specifications
endpoint-id	ID of the endpoint to update	Required
disable-auth	Disable the authentication on the endpoint.
region	Region to target. If none is passed will use default region from the config	Default: fr-par One of: fr-par

Models commands

Models commands.

Delete a model

Delete an existing model from your model library.

Usage:

Code
scw inference model delete <model-id ...> [arg=value ...]

Arguments:

Name	Description	Argument Specifications
model-id	ID of the model to delete	Required
region	Region to target. If none is passed will use default region from the config	Default: fr-par One of: fr-par

Get a model

Get the model for the given ID.

Usage:

Code
scw inference model get <model-id ...> [arg=value ...]

Arguments:

Name	Description	Argument Specifications
model-id	ID of the model to get	Required
region	Region to target. If none is passed will use default region from the config	Default: fr-par One of: fr-par

Import a model

Import a new model to your model library.

Usage:

Code
scw inference model import [arg=value ...]

Arguments:

Name	Description	Argument Specifications
name	Name of the model	Required Default: <generated>
project-id	Project ID to use. If none is passed the default project ID will be used
source.url
source.secret
region	Region to target. If none is passed will use default region from the config	Default: fr-par One of: fr-par

List models

List all available models.

Usage:

Code
scw inference model list [arg=value ...]

Arguments:

Name	Description	Argument Specifications
order-by	Order in which to return results	One of: display_rank_asc, created_at_asc, created_at_desc, name_asc, name_desc
project-id	Filter by Project ID
name	Filter by model name
tags.{index}	Filter by tags
organization-id	Filter by Organization ID
region	Region to target. If none is passed will use default region from the config	Default: fr-par One of: fr-par, all

List available serverless models

List all available serverless models using the OpenAI-compatible API endpoint. Prices per million of tokens.

Usage:

Code
scw inference model list-serverless

Node types management commands

Node types management commands.

List available node types

List all available node types. By default, the node types returned in the list are ordered by creation date in ascending order, though this can be modified via the order_by field.

Usage:

Code
scw inference node-type list [arg=value ...]

Arguments:

Name	Description	Argument Specifications
include-disabled-types	Include disabled node types in the response
region	Region to target. If none is passed will use default region from the config	Default: fr-par One of: fr-par, all