Scaleway DocumentationComputeFunctionsHow to
Connect a function to Object Storage

Jump toUpdate content

How to connect a Serverless Function to Object Storage

Reviewed on 26 May 2021Published on 26 May 2021

Because Serverless functions are stateless, you will likely need to connect to services such as Object Storage or Managed Database. This page shows an example of how to connect to Scaleway’s Object Storage using the AWS SDK and Python or Node.

Requirements:

Python

We are going to use the AWS SDK for Python (boto3) in the following steps. You can run these steps on your local computer or any Linux Instance. Learn more about boto3.

  1. Download boto3 in the package folder:

    pip install boto3 --target ./package
    Note:

    Alternatively you can use a requirements.txt file:

    pip install -r requirements.txt --target ./package
  2. Write your code.

    • Read a bucket object:
    import boto3
    from botocore.exceptions import ClientError
    import os

    # Defining bucket from environment variables
    bucket = os.environ.get('BUCKET_NAME')
    region_name=os.environ.get('REGION')
    use_ssl=True
    endpoint_url=os.environ.get('S3_ENDPOINT_URL')
    aws_access_key_id=os.environ.get('ACCESS_KEY_ID')
    aws_secret_access_key=os.environ.get('ACCESS_KEY')

    # Create S3 client
    s3 = boto3.resource(
    's3',
    region_name=region_name,
    use_ssl=True,
    endpoint_url=endpoint_url,
    aws_access_key_id=aws_access_key_id,
    aws_secret_access_key=aws_secret_access_key
    )

    def handle(event, context):
    # Get object from S3
    obj = s3.Object(bucket, <file)
    # Read object content
    body = obj.get()['Body'].read().decode('utf-8') #be careful to choose the right encoding
    return {
    "body": { "Object Body": body
    },
    "statusCode": 200,
    }
    • Write a new bucket object:
    import boto3
    from botocore.exceptions import ClientError
    import os

    # Defining bucket form environment variables
    bucket = os.environ.get('BUCKET_NAME')
    region_name=os.environ.get('REGION')
    use_ssl=True
    endpoint_url=os.environ.get('S3_ENDPOINT_URL')
    aws_access_key_id=os.environ.get('ACCESS_KEY_ID')
    aws_secret_access_key=os.environ.get('ACCESS_KEY')

    print(bucket, ' ', region_name, ' ', endpoint_url, ' ', aws_access_key_id)

    # Create S3 client
    s3 = boto3.resource(
    's3',
    region_name=region_name,
    use_ssl=True,
    endpoint_url=endpoint_url,
    aws_access_key_id=aws_access_key_id,
    aws_secret_access_key=aws_secret_access_key
    )

    def handle(event, context):
    txt_data = b'Hello World!'
    try:
    # Create object on S3 and write data
    obj = s3.Object(bucket, <File Name>)
    obj.put(
    Body=txt_data
    )
    return {
    "body": {"result": "new file created"
    },
    "statusCode": 200,
    }
    # Handling errors
    except ClientError as e:
    print("Error while writing file : ", e)
    return {
    "body": {"result": "Error couldn't connect to Object Storage"
    },
    "statusCode": 500,
    }
  3. Push your code to Serverless Function using a zip-file or the serverless framework (the latter is recommended).

    • Do not forget to add the following environment variables:
      • ACCESS_KEY_ID
      • ACCESS_KEY
      • BUCKET_NAME
      • REGION
      • S3_ENDPOINT_URL For more information on boto3, consult the official documentation.

NodeJS

We use the NodeJS aws-sdk library to push a file to an Object storage bucket using the S3 protocol. You can run these steps on your local computer or any Linux Instance.

  1. Download the aws-sdk:

    npm i aws-sdk
  2. Write your code.

    • Read a bucket object:
    const AWS = require ('aws-sdk');

    const BUCKET_NAME = process.env.BUCKET_NAME;
    const S3_ENDPOINT_URL= process.env.S3_ENDPOINT_URL;
    const ID = process.env.ACCESS_KEY_ID;
    const SECRET = process.env.ACCESS_KEY;

    // Create S3 service object
    const s3 = new AWS.S3({
    endpoint: S3_ENDPOINT_URL,
    credentials: {
    accessKeyId: ID,
    secretAccessKey: SECRET,
    }
    });

    //Create handler
    module.exports.handle = async (event, context, callback) => {
    const params = {
    Bucket: BUCKET_NAME,
    Key: 'functions.txt'
    };
    try {
    const data = await s3.getObject(params).promise();
    console.log(`Successfully read file from ${BUCKET_NAME}`) ;
    const objectData = data.Body.toString('utf-8');
    return {
    statusCode: 200,
    body: JSON.stringify({
    message: objectData
    }),
    headers: {
    "Content-Type": "application/json",
    }
    };
    } catch (e) {
    return {
    statusCode: 500,
    body: JSON.stringify({
    message: `Could not retrieve file from S3: ${e.message}`
    }),
    headers: {
    "Content-Type": "application/json",
    }
    };
    };

    };
    • Write to a bucket object:

    const AWS = require ('aws-sdk');

    const BUCKET_NAME = process.env.BUCKET_NAME;
    const S3_ENDPOINT_URL= process.env.S3_ENDPOINT_URL;
    const ID = process.env.ACCESS_KEY_ID;
    const SECRET = process.env.ACCESS_KEY;

    // Create S3 service object
    const s3 = new AWS.S3({
    endpoint: S3_ENDPOINT_URL,
    credentials: {
    accessKeyId: ID,
    secretAccessKey: SECRET,
    }
    });

    module.exports.handle = async (event, context, callback) => {
    const params = {
    Bucket: BUCKET_NAME,
    Key: 'nodejs14file.txt',
    Body: "Hello World from NodeJs 14"
    };
    try {
    const putResult = await s3.putObject(params).promise();
    message=`File uploaded successfully at ${BUCKET_NAME}`;
    } catch (error) {
    console.log(error);
    return {
    statusCode: 500,
    body: JSON.stringify({
    message: "Error while uploading file, check logs for more information",
    }),
    headers: {
    "Content-Type": "application/json",
    },
    };
    }
    return {
    statusCode: 200,
    body: JSON.stringify({
    message: message,
    }),
    headers: {
    "Content-Type": "application/json",
    },
    }
    }
  3. Push your code to Serverless Function using a zip-file or the serverless framework (the latter is recommended).

    • Do not forget to add the following environment variables:
      • ACCESS_KEY_ID
      • ACCESS_KEY
      • BUCKET_NAME
      • S3_ENDPOINT_URL For more information on aws-sdk, consult the official documentation.
See Also