Skip to navigationSkip to main contentSkip to footerScaleway Docs HomepageAsk our AI
Ask our AI

Data Orchestrator - Quickstart

Note

Data Orchestrator is currently in Public Beta. During this Beta period:

  • Graph view supports the visualization of the logic tasks fork and try-catch.
  • Workflows run Scaleway Serverless Jobs, Serverless Functions, Serverless Containers, and HTTP calls.

Data Orchestrator is designed to help the user automate, schedule, and manage data workflows across various systems and environments. Follow this guide to create your first workflow and take it through its full lifecycle.

Before you start

To complete the actions presented below, you must have:

Create a workflow

  1. In the Scaleway console (External link) side menu, go to Data & Analytics > Data Orchestrator. The list of your workflows appears.
  2. Click Create workflow. The Data Orchestrator workflow creation wizard opens.
  3. Select the region for your workflow.
  4. Enter a name or use the automatically generated one.
  5. Click Create workflow and add tasks.
  6. Click Choose a job to create the first task. The graph view opens with an empty job task created.
  7. Click the empty job task, then:
    1. Select the region of the job.
    2. Select the job you want to use.
  8. In the tasks menu on the left, click Serverless Jobs to create a new empty job task. Note that a new task always appears to the right of all other tasks.
  9. Click the new task and select the second job.
  10. Drag the connector on the right side of the first task to the connector on the left side of the second task to link them.
  11. Click Save changes to finish, or click Save and run to save the workflow and run it immediately.

Run a workflow

  1. In the Scaleway console (External link) side menu, go to Data & Analytics > Data Orchestrator. The list of your workflows appears.
  2. Click the name of the workflow you want to run. The workflow Overview tab opens.
  3. Click the Actions button, then click Run in the drop-down menu. Your workflow starts, and a new entry appears in the Runs tab.
Tip

On the Data Orchestrator page, you can also click the run icon next to the workflow you want to run.

Monitor a running workflow

  1. Click the name of the workflow you want to monitor. The workflow Overview tab opens.
  2. Make sure the last run status is Running or Queued.
  3. Click View run next to the last run status. The run information appears.

Schedule a workflow

  1. Click the name of the workflow you want to manage. The workflow Overview tab opens.
  2. Click the Settings tab.
  3. Check the Enable run scheduling box.
  4. Enter the cron expression and the time zone.
  5. Click Save changes.

Delete a workflow

  1. Click the name of the workflow you want to delete. The workflow Overview tab opens.
  2. Navigate to the Settings tab.
  3. Click Delete Workflow.
  4. Type DELETE to confirm.
  5. Click Delete Workflow.
Still need help?

Create a support ticket
No Results