You can also download the work files by cloning our Scaleway Serverless examples repository.
Create snapshots of an Instance with Serverless Jobs
- serverless
- jobs
- instance
- snapshot
- backup
- image
- disk
- storage
Scaleway Serverless Jobs allows you to create and automate recurring tasks. This tutorial will guide you through the process of creating snapshots of a Scaleway Instance on a recurring schedule using a Serverless Job.
Serverless Jobs are perfectly adapted for these autonomous tasks, as we do not need autoscaling or exposure via a web server. Refer to the differences between jobs, containers and functions for more information.
Before you start
To complete the actions presented below, you must have:
- A Scaleway account logged into the console
- Owner status or IAM permissions allowing you to perform actions in the intended Organization
- Created a Container Registry namespace.
- Created an Instance
- Installed and started the Docker daemon to build the image.
Creating the snapshot generator files
-
Create a file named
main.go
, and add the code below to it:package mainimport ("fmt""os""github.com/scaleway/scaleway-sdk-go/api/instance/v1""github.com/scaleway/scaleway-sdk-go/scw")func main() {fmt.Println("creating snapshot of instance...")// Create a Scaleway client with credentials from environment variables.client, err := scw.NewClient(// Get your organization ID at https://console.scaleway.com/organization/settingsscw.WithDefaultOrganizationID(os.Getenv("SCW_DEFAULT_ORGANIZATION_ID")),// Get your credentials at https://console.scaleway.com/iam/api-keysscw.WithAuth(os.Getenv("SCW_ACCESS_KEY"), os.Getenv("SCW_SECRET_KEY")),// Get more about our availability zones at https://www.scaleway.com/en/docs/console/my-account/reference-content/products-availability/scw.WithDefaultRegion(scw.RegionFrPar),)if err != nil {panic(err)}// Create SDK objects for Scaleway Instance productinstanceAPI := instance.NewAPI(client)if err := createSnapshots(instanceAPI); err != nil {panic(err)}}func createSnapshots(instanceAPI *instance.API) error {gotInstance, err := instanceAPI.GetServer(&instance.GetServerRequest{ServerID: os.Getenv("INSTANCE_ID"),Zone: scw.Zone(os.Getenv("INSTANCE_ZONE")),})if err != nil {return fmt.Errorf("error while getting instance %w", err)}for _, volume := range gotInstance.Server.Volumes {snapshotResp, err := instanceAPI.CreateSnapshot(&instance.CreateSnapshotRequest{Name: volume.Name + RandomString(4),VolumeID: &volume.ID,VolumeType: instance.SnapshotVolumeTypeBSSD,Zone: scw.Zone(os.Getenv("INSTANCE_ZONE")),})if err != nil {return fmt.Errorf("error while creating snapshopt %w", err)}fmt.Println("created snapshot ", snapshotResp.Snapshot.ID)}return nil}func init() {if os.Getenv("SCW_DEFAULT_ORGANIZATION_ID") == "" {panic("missing SCW_DEFAULT_ORGANIZATION_ID")}if os.Getenv("SCW_ACCESS_KEY") == "" {panic("missing SCW_ACCESS_KEY")}if os.Getenv("SCW_SECRET_KEY") == "" {panic("missing SCW_SECRET_KEY")}if os.Getenv("INSTANCE_ID") == "" {panic("missing INSTANCE_ID")}if os.Getenv("INSTANCE_ZONE") == "" {panic("missing INSTANCE_ZONE")}} -
Create a file called
go.mod
, and add the code below to it:module github.com/scaleway/serverless-examples/jobs/instances-snapshotgo 1.22.2require github.com/scaleway/scaleway-sdk-go v1.0.0-beta.21require gopkg.in/yaml.v2 v2.4.0 // indirect -
Run the follwing command to download the required dependencies:
go get -
Create a file named
string.go
, and add the code below to it:package mainimport "crypto/rand"// RandomString is used to generate a random string containing upper and lower characters// + number, of size n.func RandomString(n int) string {const letters = "0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz"bytes := make([]byte, n)if _, err := rand.Read(bytes); err != nil {panic(err)}for i, b := range bytes {bytes[i] = letters[b%byte(len(letters))]}return string(bytes)}
Building and pushing the image to Container Registry
Serverless Jobs rely on containers to run in the cloud, and therefore require a container image hosted in the cloud using Scaleway Container Registry.
-
Create a
Dockerfile
, and add the following code to it:# Using apline/golang imageFROM golang:1.22-alpine# Set destination for COPYWORKDIR /app# Copy required filesCOPY go.mod ./COPY go.sum ./COPY *.go ./# Build the executableRUN go build -o /jobs-snapshot# Run the executableCMD [ "/jobs-snapshot" ] -
Run the following command in a terminal to connect to your Container Registry namespace. Do not forget to edit the command with your namespace name.
docker login rg.fr-par.scw.cloud/your-namespace-name -u nologin --password-stdin <<< "$SCW_SECRET_KEY" -
Run the following command to build the container image locally:
docker build -t rg.fr-par.scw.cloud/your-namespace-name/jobs-snapshot:v1 . -
Run the following command to push the container image to the registry:
docker push rg.fr-par.scw.cloud/your-namespace-name/jobs-snapshot:v1
Your image and its tag now appear in the Container Registry in the Scaleway console.
Creating the Job Definition
-
In the Scaleway console, click Jobs in the Serverless section of the side menu. The jobs page displays.
-
Click Create job. The job creation wizard displays.
-
Select the Scaleway Container Registry, then select your Container Registry namespace from the drop-down list, and the container image and tag.
-
Enter a name or use the automatically generated one.
-
Select the region in which your job will be created.
-
Keep the default resources values, as this job requires little compute capabilities.
-
Set the cron schedule to
0 2 * * *
and select the relevant time zone to run the job every day at 2:00 a.m. Refer to the cron schedules documentation for more information. -
Define the following environment variables:
INSTANCE_ID
: the ID of the instance you want to snapshotINSTANCE_ZONE
: the Availabilitiy Zone of your Instance (e.g.fr-par-2
)SCW_ACCESS_KEY
: your API access keySCW_SECRET_KEY
: your API secret keySCW_DEFAULT_ORGANIZATION_ID
: your Oganization ID
-
Click Create job.
Running the job
From the Overview tab of the Serverless job you just created, click, Actions, then select Run job from the contextual menu.
The execution appears in the Job runs section. You can access the logs of your job by clicking «See more Icon» next to the job run ID, and selecting See on cockpit.
Possible improvements
This tutorial is a lightweight example of how to create recurring snapshots of an Instance. You can go further by:
- Using it to manage all your Instances snapshots
- Create backups of your storage disks
- Set up an alerting system in case of unextpected behavior