Skip to navigationSkip to main contentSkip to footerScaleway DocsAsk our AI
Ask our AI

How to execute complex startup commands using Serverless Jobs

Scaleway Serverless Jobs allows you to execute specific startup commands when running a job. Due to technical specifications, complex commands (e.g. piped commands, xargs commands) may fail occasionally, preventing jobs from running successfully.

You can bypass this limitation by passing complex commands and scripts to a Serverless Job via a secret reference. You can then inject this secret as a file in your job, and call it at startup to execute the commands it contains.

Before you start

To complete the actions presented below, you must have:

How to create a secret containing your command

  1. Click Secret Manager in the Security & Identity section of the Scaleway console side menu.

  2. In the Region drop-down, select the region in which you want to store your secret. Secrets cannot be moved from one region to another after creation.

  3. Click + Create secret.

  4. Select Import secret, choose the Opaque secret type, then drag and drop your file to the dedicated area. The maximum file size for your secret is 64 KiB.

    Note

    We recommend using a shell script (.sh) file containing your command. Refer to the section below for example commands.

  5. Choose the Scaleway-managed encryption key, as it requires no configuration on your side.

  6. Choose a path for your secret.

  7. Enter a name for your secret, and, optionally, add a description and tags.

  8. Click Create secret to confirm.

Your file can now be passed to your Serverless Job as a secret reference.

Create a Serverless Job referencing your command as a secret

  1. Click Jobs in the Serverless section of the side menu. The jobs page displays.

  2. Click Create job. The job creation wizard displays.

  3. Select the external container registry.

  4. Enter the following image URL:

    scaleway/cli:latest
  5. Enter a name, select the desired region, and choose the smallest resources available.

  6. From the Data tab, add your command file as a secrets reference. Refer to the dedicated documentation on secrets for more information.

  7. From the Execution tab, add the following startup command to call your file:

    bash /complex_command.sh start
  8. Click Create job to finish.

You job is now ready to run.

Complex commands examples

Below are examples of commands that must be passed via a secret referenced as a file in Serverless Jobs. You can find more complex command examples on the Scaleway CLI repository.

Retrieve a specific field from the output using jq

## Retrieve all available instance type names

# Using jq
scw -o json instance server-type list | jq -r '.[].name'
# Using CLI templates
scw instance server-type list -o template="{{ .Name }}"

Filter command output using jq

# Retrieve all available instance type with GPUs
scw -o json instance server-type list | jq '.[] | select (.gpu > 0)'

Parallelize actions using xargs

## Reboot all listed servers at the same time, 8 max concurrent actions

# Using jq
scw -o json instance server list | jq -r '.[].id' | xargs -L1 -P8 scw instance server reboot
# Using CLI templates
scw instance server list -o template="{{ .ID }}" | xargs -L1 -P8 scw instance server reboot

Create arguments for a CLI command and use it in xargs

## List private-nics of all listed servers

# Using jq
scw -o json instance server list | jq -r '.[] | "server-id=\(.id)"' | xargs -L1 scw instance private-nic list
# Using CLI templates
scw instance server list -o template="server-id={{ .ID }}" | xargs -L1 scw instance private-nic list
Still need help?

Create a support ticket
No Results