How to connect a Serverless Function to Object Storage

Because Serverless functions are stateless, you will likely need to connect to services such as Object Storage or Databases to store your data. This page shows an example of how to connect to Scaleway’s Object Storage using the AWS SDK and Python or Node.

Environment variables

Both our Python and Node code examples below require you to add the following environment variables:

• ACCESS_KEY_ID (the access key of your Scaleway API key)
• ACCESS_KEY (the secret key of your Scaleway API key
• BUCKET_NAME (the name for your bucket)
• S3_ENDPOINT_URL (the bucket endpoint URL)

Python

We are going to use the AWS SDK for Python (boto3) in the following steps. You can run these steps on your local computer or any Linux Instance. Learn more about boto3.

1. Download boto3 in the package folder:

   pip install boto3 --target ./package
   Copy code

   Note:

   Alternatively you can use a requirements.txt file:

   pip install -r requirements.txt --target ./package
   Copy code

2. Write your code.

   • Read a bucket object:

   import boto3
   from botocore.exceptions import ClientError
   import os
   
   # Defining bucket from environment variables
   bucket = os.environ.get('BUCKET_NAME')
   region_name=os.environ.get('REGION')
   use_ssl=True
   endpoint_url=os.environ.get('S3_ENDPOINT_URL')
   aws_access_key_id=os.environ.get('ACCESS_KEY_ID')
   aws_secret_access_key=os.environ.get('ACCESS_KEY')
   
   # Create S3 client
   s3 = boto3.resource(
       's3',
       region_name=region_name,
       use_ssl=True,
       endpoint_url=endpoint_url,
       aws_access_key_id=aws_access_key_id,
       aws_secret_access_key=aws_secret_access_key
   )
   
   def handle(event, context):
       # Get object from S3
   obj = s3.Object(bucket, <file)
   # Read object content
   body = obj.get()['Body'].read().decode('utf-8') #be careful to choose the right encoding
   return {
       "body": { "Object Body": body
       },
       "statusCode": 200,
   }
   Copy code

   • Write a new bucket object:

   import boto3
   from botocore.exceptions import ClientError
   import os
   
   # Defining bucket form environment variables
   bucket = os.environ.get('BUCKET_NAME')
   region_name=os.environ.get('REGION')
   use_ssl=True
   endpoint_url=os.environ.get('S3_ENDPOINT_URL')
   aws_access_key_id=os.environ.get('ACCESS_KEY_ID')
   aws_secret_access_key=os.environ.get('ACCESS_KEY')
   
   print(bucket, ' ', region_name, ' ', endpoint_url, ' ', aws_access_key_id)
   
   # Create S3 client
   s3 = boto3.resource(
       's3',
       region_name=region_name,
       use_ssl=True,
       endpoint_url=endpoint_url,
       aws_access_key_id=aws_access_key_id,
       aws_secret_access_key=aws_secret_access_key
   )
   
   def handle(event, context):
       txt_data = b'Hello World!'
       try:
           # Create object on S3 and write data
           obj = s3.Object(bucket, <File Name>)
           obj.put(
               Body=txt_data
           )
           return {
               "body": {"result": "new file created"
                       },
               "statusCode": 200,
           }
       # Handling errors
       except ClientError as e:
           print("Error while writing file : ", e)
           return {
               "body": {"result": "Error couldn't connect to Object Storage"
                       },
               "statusCode": 500,
           }
   Copy code

3. Push your code to Serverless Function using a zip-file or the serverless framework (the latter is recommended).

   • Do not forget to add the following environment variables:
     • ACCESS_KEY_ID
     • ACCESS_KEY
     • BUCKET_NAME
     • REGION
     • S3_ENDPOINT_URL For more information on boto3, consult the official documentation.

NodeJS

We use the NodeJS aws-sdk library to push a file to an Object storage bucket using the S3 protocol. You can run these steps on your local computer or any Linux Instance.

1. Download the aws-sdk:

   npm i aws-sdk
   Copy code

2. Write your code.
   • Read a bucket object:

   const AWS = require ('aws-sdk');
   
   const BUCKET_NAME = process.env.BUCKET_NAME;
   const S3_ENDPOINT_URL= process.env.S3_ENDPOINT_URL;
   const ID = process.env.ACCESS_KEY_ID;
   const SECRET = process.env.ACCESS_KEY;
   
   // Create S3 service object
   const s3 = new AWS.S3({
   endpoint: S3_ENDPOINT_URL,
   credentials: {
       accessKeyId: ID,
       secretAccessKey: SECRET,
   }
   });
   
   //Create handler
   module.exports.handle = async (event, context, callback) => {
   const params = {
       Bucket: BUCKET_NAME,
       Key: 'functions.txt'
       };
   try {   
   const data = await s3.getObject(params).promise();
   console.log(`Successfully read file from ${BUCKET_NAME}`) ;
   const objectData = data.Body.toString('utf-8');
   return {
       statusCode: 200,
       body: JSON.stringify({
           message: objectData
       }),
       headers: {
           "Content-Type": "application/json",
       }
       };
   } catch (e) {
       return {
       statusCode: 500,
       body: JSON.stringify({
           message: `Could not retrieve file from S3: ${e.message}`
       }),
       headers: {
           "Content-Type": "application/json",
       }
       };
   };
   
   };
   Copy code

   • Write to a bucket object:

   
   const AWS = require ('aws-sdk');
   
   const BUCKET_NAME = process.env.BUCKET_NAME;
   const S3_ENDPOINT_URL= process.env.S3_ENDPOINT_URL;
   const ID = process.env.ACCESS_KEY_ID;
   const SECRET = process.env.ACCESS_KEY;
   
   // Create S3 service object
   const s3 = new AWS.S3({
   endpoint: S3_ENDPOINT_URL,
   credentials: {
       accessKeyId: ID,
       secretAccessKey: SECRET,
   }
   });
   
   module.exports.handle = async  (event, context, callback) => { 
   const params = {
       Bucket: BUCKET_NAME,
       Key: 'nodejs14file.txt',
       Body: "Hello World from NodeJs 14"
       };
       try {
       const putResult = await s3.putObject(params).promise();
       message=`File uploaded successfully at ${BUCKET_NAME}`;
       } catch (error) { 
           console.log(error);
           return {
           statusCode: 500,
           body: JSON.stringify({
               message: "Error while uploading file, check logs for more information",
           }),
           headers: {
               "Content-Type": "application/json",
           },
       };
       }
   return {
       statusCode: 200,
       body: JSON.stringify({
           message: message,
       }),
       headers: {
           "Content-Type": "application/json",
       },
   }
   }
   Copy code

3. Push your code to Serverless Function using a zip-file or the serverless framework (the latter is recommended).

For more information on aws-sdk, consult the official documentation.