How to change the model of a Managed Inference deployment
You can change the model used by your Managed Inference deployment at any time, as long as the new model is compatible with the existing deployment node. If you want to change to a model that is too big for your current node type, you must delete your existing deployment and create a new one with a compatible node type.
Follow the steps below to change the model using the Scaleway console:
Before you start
To complete the actions presented below, you must have:
- A Scaleway account logged into the console
- A Managed Inference deployment
- Owner status or IAM permissions allowing you to perform actions in the intended Organization
- Click Managed Inference in the AI section of the Scaleway console side menu. A list of your deployments displays.
- From the drop-down menu, select the geographical region you want to manage.
- Choose a deployment either by clicking its name, or selecting More info from the drop-down menu represented by the icon more icon to access the deployment dashboard.
- Click the Settings tab of your deployment to display additional settings.
- In the Model panel, click Change model. A pop-up displays, showing your current model.
- From the drop-down menu, select the model you want to change to, and click Change model.
The change of model is initiated. Note that while the model is changed, your deployment will not be available for 15 - 30 minutes.
Remember to update the model string in your client configuration, to reflect the new model. You can find the model string in the code sample available in the Playground tab of your deployment's dashboard (use the View code button).
If you have also changed to a different type of model (e.g. from a chat model to an embedding model), you will also need to update the client code itself, as well as the model string.