Deploying and managing Instances with Terraform and Functions

• serverless
• python
• terraform

HashiCorp Terraform is an open-source software tool to deploy Infrastructure as Code (IaC). It allows you to automate the lifecycle of your Instances by using declarative configuration files. In this tutorial you will discover an example of Instance automation using Python and Terraform (automatically shut down / start Instances).

Before you start

To complete the actions presented below, you must have:

• A Scaleway account logged into the console
• Owner status or IAM permissions allowing you to perform actions in the intended Organization
• A valid API key
• Installed Python on your machine
• Installed Terraform on your machine

Context

This tutorial will simulate a project with a production environment running all the time and a development environment that will be switched off on weekends to reduce costs.

Initialize a Terraform project

1. Create a new folder, called Terraform, to store your configuration. Then enter the folder:

   mkdir Terraform && cd Terraform
   

2. Create five files to configure your infrastructure:

   • main.tf: will contain the main set of configurations for your project. Here, it will be our Instance
   • provider.tf: Terraform relies on plugins called ”providers” to interact with remote systems
   • backend.tf: each Terraform configuration can specify a backend, which defines where the state file of the current infrastructure will be stored. Terraform uses this file to keep track of the managed resources. This state can be stored locally or remotely. Configuring a remote backend allows multiple people to work on the same infrastructure
   • variables.tf: will contain the variable definitions for your project. Since all Terraform values must be defined, any variables that are not given a default value will become required arguments
   • terraform.tfvars: allows you to set the actual value of the variables

3. Create the following folders:

   • function: to store your function code
   • files: to temporary store your zip function code Your folder structure should look like:

   Terraform
   | -- files
   | -- function
   -- main.tf
   -- backend.tf
   -- provider.tf
   -- terraform.tfvars
   -- variables.tf
   

4. Edit the backend.tf file to enable remote configuration backup:

   terraform {
   backend "s3" {
       bucket                      = "XXXXXXXXX"
       key                         = "terraform.tfstate"
       region                      = "fr-par"
       endpoint                    = "https://s3.fr-par.scw.cloud"
       skip_credentials_validation = true
       skip_region_validation      = true
   }
   }
       /*
       For the credentials part:
       ==> Create a ~/.aws/credentials:
       [default]
       aws_access_key_id=<SCW_ACCESS_KEY>
       aws_secret_access_key=<SCW_SECRET_KEY>
       region=fr-par
       */
   

5. Edit the provider.tf file and add Scaleway as a provider:

   terraform {
     required_providers {
       scaleway = {
         source  = "scaleway/scaleway"
         version = "2.41.1"
       }
     }
     required_version = ">= 0.13"
   }
   

6. Specify the following variables in the variables.tf file:

variable "zone" {
  type = string
}
variable "region" {
  type = string
}
variable "env" {
  type = string
}
variable "project_id" {
  type        = string
  description = "Your project ID"
}
variable "auth_token" {
    type = string
    description = "Scaleway authentication token used in the function"
}

7. Add the variable values to terraform.tfvars:

   zone                           = "fr-par-1"
   region                         = "fr-par"
   env                            = "dev"
   project_id                     = "Your project ID"
   auth_token                     = "Your Scaleway Secret key"
   

Writing the code

For this example we will use the native python library urllib, which allows us to keep the package slim.

Inside the function folder create a handler.py file with following contents:

import os
from urllib import request, parse, error
import json
auth_token = os.environ["X-AUTH-TOKEN"]
def handle(event, context):
  ## get information from cron
  event_body = eval(event["body"])
  zone = event_body["zone"]
  server_id = event_body["server_id"]
  action = event_body["action"]  # action should be "poweron" or "poweroff"
  # create request
  url = f"https://api.scaleway.com/instance/v1/zones/{zone}/servers/{server_id}/action"
  data = json.dumps({"action": action}).encode("ascii")
  req = request.Request(url, data=data, method="POST")
  req.add_header("Content-Type", "application/json")
  req.add_header("X-Auth-Token", auth_token)
  # Sending request to Instance API
  try:
      res = request.urlopen(req).read().decode()
  except error.HTTPError as e:
      res = e.read().decode()
  return {
      "body": json.loads(res),
      "statusCode": 200,
  }

Configuring your infrastructure

1. Edit the file main.tf to add a production Instance using a GP1-S named “Prod”:

   ## Configuring Producion environment
   resource "scaleway_instance_ip" "public_ip-prod" {
       project_id = var.project_id
   }
   resource "scaleway_instance_server" "scw-instance-prod" {
     name="prod"
     project_id = var.project_id
     type  = "GP1-S"
     image = "ubuntu_focal"
     tags = ["terraform instance", "scw-instance", "production"]
     ip_id = scaleway_instance_ip.public_ip-prod.id
     root_volume {
       # The local storage of a DEV1-L Instance is 80 GB, subtract 30 GB from the additional l_ssd volume, then the root volume needs to be 50 GB.
       size_in_gb = 200
     }
   }
   

2. Add a development Instance using a DEV1-L named “Dev”:

   ## Configuring Development environment that will be automatically turn off on week-ends and turn on monday mornings
   resource "scaleway_instance_ip" "public_ip-dev" {
       project_id = var.project_id
   }
   resource "scaleway_instance_server" "scw-instance-dev" {
     name="dev"
     project_id = var.project_id
     type  = "DEV1-L"
     image = "ubuntu_focal"
     tags = ["terraform instance", "scw-instance", "dev"]
     ip_id = scaleway_instance_ip.public_ip-dev.id
     root_volume {
       size_in_gb = 80
     }
   }
   

3. Write a function that will run the code you have just written:

   # Creating function code archive that will then be updated
   data "archive_file" "source_zip" {
     type             = "zip"
     source_dir       = "${path.module}/function"
     output_path      = "${path.module}/files/function.zip"
   }
   # Creating the function namespace
   resource "scaleway_function_namespace" "main"  {
     name        = "instance-management"
     description = "namespace to gather all functions dedicated to Instance management"
     project_id = var.project_id
   }
   # Creating the function
   resource "scaleway_function" "main" {
     namespace_id = scaleway_function_namespace.main.id
     name         = "instancewake"
     runtime      = "python310"
     handler      = "handler.handle"
     privacy      = "public"
     zip_file = data.archive_file.source_zip.output_path # this enable to automatically zip your code and each time you edit it
     zip_hash = filesha256(data.archive_file.source_zip.output_path)
     deploy = true
     max_scale    = "5"
     environment_variables = {
         "X-AUTH-TOKEN" = var.auth_token
     }
   }
   

4. Add a cronjob attached to the function to turn your function off every Friday evening:

   # Adding a first cron to turn off the Instance every friday evening (11:30 pm)
   resource "scaleway_function_cron" "turn-off" {
       function_id = scaleway_function.main.id
       schedule = "30 23 * * 5"
       args = jsonencode({
           "zone": scaleway_instance_server.scw-instance-dev.zone,
           "server_id": regex("/([^/]+$)", scaleway_instance_server.scw-instance-dev.id)[0], # We use the dev Instance id and strip it from the region
           "action":"poweroff"
           }
       )
   }
   

5. Create a cronjob attached to the function to turn your function on every Monday morning:

   # Adding a second cron to turn on the Instance every monday morning (7:00 am)
   resource "scaleway_function_cron" "turn-on" {
       function_id = scaleway_function.main.id
       schedule = "0 7 * * 1"
       args = jsonencode({
           "zone": scaleway_instance_server.scw-instance-dev.zone,
           "server_id": regex("/([^/]+$)", scaleway_instance_server.scw-instance-dev.id)[0],
           "action":"poweron"
           }
       )
   }
   

Deploying your infrastructure

Everything is configured now, so you can deploy your infrastructure using Terraform.

1. Add your Scaleway credentials to the environment variables:

   export SCW_ACCESS_KEY="my-access-key"
   export SCW_SECRET_KEY="my-secret-key"
   

2. Initialize Terraform:

   terraform init
   

3. Let Terraform verify your configuration:

   terraform plan
   

4. Deploy your infrastructure:

   terraform apply