---
title: How to query language models
description: Learn how to interact with powerful language models using Scaleway's Generative APIs service.
tags: generative-apis ai-data language-models chat-completions-api
dates:
validation: 2025-08-22
posted: 2024-08-28
---
import Requirements from '@macros/iam/requirements.mdx'
import ChatCompVsResponsesApi from '@macros/ai/chat-comp-vs-responses-api.mdx'
Scaleway's Generative APIs service allows users to interact with powerful language models hosted on the platform.
There are several ways to interact with language models:
- The Scaleway [console](https://console.scaleway.com) provides complete [playground](/generative-apis/how-to/query-language-models/#accessing-the-playground), aiming to test models, adapt parameters, and observe how these changes affect the output in real-time
- Via the [Chat Completions API](https://www.scaleway.com/en/developers/api/generative-apis/chat-completions) or the [Responses API](https://www.scaleway.com/en/developers/api/generative-apis/responses)
- Via your own [dedicated deployment](/generative-apis/how-to/create-deployment/) of a chosen model
- A Scaleway account logged into the [console](https://console.scaleway.com)
- [Owner](/iam/concepts/#owner) status or [IAM permissions](/iam/concepts/#permission) allowing you to perform actions in the intended Organization
- A valid [API key](/iam/how-to/create-api-keys/) for API authentication
- Python 3.7+ installed on your system
## Accessing the playground
Scaleway provides a web playground for instruct-based models hosted on Generative APIs.
1. Navigate to **Generative APIs** under the **AI** section of the [Scaleway console](https://console.scaleway.com/) side menu. The list of models you can query displays.
2. Click the name of the chat model you want to try. Alternatively, click **Try** next to the model's name.
The web playground displays.
## Using the playground
1. Enter a prompt at the bottom of the page, or use one of the suggested prompts in the conversation area.
2. Edit the hyperparameters listed on the right column, for example the default temperature for more or less randomness on the outputs.
3. Switch models at the top of the page, to observe the capabilities of chat models offered via Generative APIs.
4. Click **Deploy**, then select the **Serverless** option to get code snippets configured according to your settings in the playground.
You can also choose to deploy a model on your own dedicated Instance by selecting the **Dedicated** option. In this case, you can access the playground after completing the steps in the deployment wizard. Once in the playground of your deployment, click **View code** to get code snippets that match your settings in the playground.
## Querying language models via API
You can query the models programmatically using your favorite tools or languages.
In the example that follows, we will use the OpenAI Python client.
### Chat Completions API or Responses API?
### Installing the OpenAI SDK
Install the OpenAI SDK using pip:
```bash
pip install openai
```
### Initializing the client
Initialize the OpenAI client with your base URL and API key:
In the case of a dedicated Generative APIs deployment, the `base_url` value is the **Public Endpoint URL** displayed on the Overview tab of the deployment's dashboard.
```python
from openai import OpenAI
# Initialize the client with your base URL and API key
client = OpenAI(
base_url="https://api.scaleway.ai/v1", # Scaleway's Generative APIs service URL
api_key="" # Your unique API secret key from Scaleway
)
```
### Generating a chat completion
You can now create a chat completion using either the Chat Completions or Responses API, as shown in the following examples:
```python
# Create a chat completion using the 'llama-3.1-8b-instruct' model
response = client.chat.completions.create(
model="llama-3.1-8b-instruct",
messages=[{"role": "user", "content": "Describe a futuristic city with advanced technology and green energy solutions."}],
temperature=0.2, # Adjusts creativity
max_completion_tokens=100, # Limits the length of the output
top_p=0.7 # Controls diversity through nucleus sampling. You usually only need to use temperature.
)
# Print the generated response
print(response.choices[0].message.content)
```
This code sends a message to the model and returns an answer based on your input. The `temperature`, `max_completion_tokens`, and `top_p` parameters control the response's creativity, length, and diversity, respectively.
```python
# Create a chat completion using the 'gpt-oss-120b' model
response = client.responses.create(
model="gpt-oss-120b",
input=[{"role": "user", "content": "Briefly describe a futuristic city with advanced technology and green energy solutions."}],
temperature=0.2, # Adjusts creativity
max_output_tokens=100, # Limits the length of the output
top_p=0.7 # Controls diversity through nucleus sampling. You usually only need to use temperature.
)
# Print the generated response. Here, the last output message will contain the final content.
# Previous outputs will contain reasoning content.
print(response.output[-1].content[0].text)
```
A conversation style may include a default system prompt. You may set this prompt by setting the first message with the role system. For example:
```python
[
{
"role": "system",
"content": "You are Xavier Niel."
},
{
"role": "user",
"content": "Hello, what is your name?"
}
]
```
Adding such a system prompt can also help resolve issues if you receive responses such as `I'm not sure what tools are available to me. Can you please provide a library of tools that I can use to generate a response?`.
### Model parameters and their effects
The following parameters will influence the output of the model:
- **`messages`**: A list of message objects that represent the conversation history. Each message should have a `role` (e.g., "system", "user", "assistant") and `content`.
- **`temperature`**: Controls the output's randomness. Lower values (e.g., 0.2) make the output more deterministic, while higher values (e.g., 0.8) make it more creative.
- **`max_completion_tokens`**: The maximum number of tokens (words or parts of words) in the generated output.
- **`top_p`**: Recommended for advanced use cases only. You usually only need to use temperature. `top_p` controls the diversity of the output, using nucleus sampling, where the model considers the tokens with top probabilities until the cumulative probability reaches `top_p`.
- **`stop`**: A string or list of strings where the model will stop generating further tokens. This is useful for controlling the end of the output.
See the [dedicated API documentation](https://www.scaleway.com/en/developers/api/generative-apis/chat-completions) for a full list of all available parameters.
- **`input`**: A single text string, or an array of string/multi-modal inputs to provide to the model to generate a response. When using the array option, you can define a `role` and list of `content` inputs of different types (texts, files, images etc.)
- **`max_output_tokens`**: A maximum number of output tokens that can be generated for a completion. Different default maximum values
are enforced for each model, to avoid edge cases where tokens are generated indefinitely.
- **`temperature`**: Controls the output's randomness. Lower values (e.g., 0.2) make the output more deterministic, while higher values (e.g., 0.8) make it more creative.
- **`top_p`**: Recommended for advanced use cases only. You usually only need to use temperature. `top_p` controls the diversity of the output, using nucleus sampling, where the model considers the tokens with top probabilities until the cumulative probability reaches `top_p`.
See the [dedicated API documentation](https://www.scaleway.com/en/developers/api/generative-apis/responses) for a full list of all available parameters.
If you encounter an error such as "Forbidden 403" refer to the [API documentation](/generative-apis/api-cli/understanding-errors) for troubleshooting tips.
## Streaming
By default, the outputs are returned to the client only after the generation process is complete. However, a common alternative is to stream the results back to the client as they are generated. This is particularly useful in chat applications, where it allows the client to view the results incrementally as each token is produced.
The example below shows you how to use the Chat Completions API.
```python
from openai import OpenAI
client = OpenAI(
base_url="https://api.scaleway.ai/v1", # Scaleway's Generative APIs service URL
api_key="" # Your Scaleway API secret key
)
response = client.chat.completions.create(
model="llama-3.1-8b-instruct",
messages=[{
"role": "user",
"content": "Sing me a song",
}],
stream=True,
)
for chunk in response:
if chunk.choices and chunk.choices[0].delta.content:
print(chunk.choices[0].delta.content, end="")
```
The example below shows you how to use the Responses API.
```python
from openai import OpenAI
client = OpenAI(
base_url="https://api.scaleway.ai/v1", # Scaleway's Generative APIs service URL
api_key="" # Your Scaleway API secret key
)
response = client.responses.create(
model="gpt-oss-120b",
input=[{"role": "user", "content": "Sing me a song"}],
stream=True
)
for chunk in response:
if chunk.type in ("response.reasoning_text.delta","response.output_text.delta"):
print(chunk.delta, end="")
```
## Async
The service also supports asynchronous mode for any chat completion.
```python
import asyncio
from openai import AsyncOpenAI
client = AsyncOpenAI(
base_url="https://api.scaleway.ai/v1", # Scaleway's Generative APIs service URL
api_key="" # Your unique API key from Scaleway
)
async def main():
stream = await client.chat.completions.create(
model="llama-3.1-8b-instruct",
messages=[{
"role": "user",
"content": "Sing me a song",
}],
stream=True,
)
async for chunk in stream:
if chunk.choices and chunk.choices[0].delta.content:
print(chunk.choices[0].delta.content, end="")
asyncio.run(main())
```
```python
import asyncio
from openai import AsyncOpenAI
client = AsyncOpenAI(
base_url="https://api.scaleway.ai/v1", # Scaleway's Generative APIs service URL
api_key="" # Your unique API key from Scaleway
)
async def main():
stream = await client.responses.create(
model="gpt-oss-120b",
input=[{
"role": "user",
"content": "Sing me a song"
}],
stream=True,
)
async for event in stream:
if event.type == "response.output_text.delta":
print(event.delta, end="")
elif event.type == "response.completed":
continue
asyncio.run(main())
```