Jump toUpdate content
How to connect a Serverless Function to Object Storage
Because Serverless functions are stateless, you will likely need to connect to services such as Object Storage or Databases to store your data. This page shows an example of how to connect to Scaleway’s Object Storage using the AWS SDK and Python or Node.
If you have activated IAM, you may need certain IAM permissions to carry out some actions described on this page. This means:
- you are the Owner of the Scaleway Organization in which the actions will be carried out, or
- you are an IAM user of the Organization, with a policy granting you the necessary permission sets
- You have an account and are logged into the Scaleway console
- You have created a functions namespace
- You have generated an API key
Environment variables
Both our Python and Node code examples below require you to add the following environment variables:
ACCESS_KEY_ID
(the access key of your Scaleway API key)ACCESS_KEY
(the secret key of your Scaleway API keyBUCKET_NAME
(the name for your bucket)S3_ENDPOINT_URL
(the bucket endpoint URL)
If you have activated IAM, you can use a Scaleway API key attached to an IAM application in the <ACCESS_KEY_ID>
and <ACCESS_KEY>
fields. Make sure that the IAM application has a policy giving it permissions to read and create Object Storage buckets, and that the API key’s preferred project for Object Storage is set to the correct Project.
Python
We are going to use the AWS SDK for Python (boto3) in the following steps. You can run these steps on your local computer or any Linux Instance. Learn more about boto3.
-
Download boto3 in the package folder:
pip install boto3 --target ./package
Note:Alternatively you can use a requirements.txt file:
pip install -r requirements.txt --target ./package
-
Write your code.
- Read a bucket object:
import boto3
from botocore.exceptions import ClientError
import os
# Defining bucket from environment variables
bucket = os.environ.get('BUCKET_NAME')
region_name=os.environ.get('REGION')
use_ssl=True
endpoint_url=os.environ.get('S3_ENDPOINT_URL')
aws_access_key_id=os.environ.get('ACCESS_KEY_ID')
aws_secret_access_key=os.environ.get('ACCESS_KEY')
# Create S3 client
s3 = boto3.resource(
's3',
region_name=region_name,
use_ssl=True,
endpoint_url=endpoint_url,
aws_access_key_id=aws_access_key_id,
aws_secret_access_key=aws_secret_access_key
)
def handle(event, context):
# Get object from S3
obj = s3.Object(bucket, <file)
# Read object content
body = obj.get()['Body'].read().decode('utf-8') #be careful to choose the right encoding
return {
"body": { "Object Body": body
},
"statusCode": 200,
}- Write a new bucket object:
import boto3
from botocore.exceptions import ClientError
import os
# Defining bucket form environment variables
bucket = os.environ.get('BUCKET_NAME')
region_name=os.environ.get('REGION')
use_ssl=True
endpoint_url=os.environ.get('S3_ENDPOINT_URL')
aws_access_key_id=os.environ.get('ACCESS_KEY_ID')
aws_secret_access_key=os.environ.get('ACCESS_KEY')
print(bucket, ' ', region_name, ' ', endpoint_url, ' ', aws_access_key_id)
# Create S3 client
s3 = boto3.resource(
's3',
region_name=region_name,
use_ssl=True,
endpoint_url=endpoint_url,
aws_access_key_id=aws_access_key_id,
aws_secret_access_key=aws_secret_access_key
)
def handle(event, context):
txt_data = b'Hello World!'
try:
# Create object on S3 and write data
obj = s3.Object(bucket, <File Name>)
obj.put(
Body=txt_data
)
return {
"body": {"result": "new file created"
},
"statusCode": 200,
}
# Handling errors
except ClientError as e:
print("Error while writing file : ", e)
return {
"body": {"result": "Error couldn't connect to Object Storage"
},
"statusCode": 500,
} -
Push your code to Serverless Function using a zip-file or the serverless framework (the latter is recommended).
- Do not forget to add the following environment variables:
ACCESS_KEY_ID
ACCESS_KEY
BUCKET_NAME
REGION
S3_ENDPOINT_URL
For more information on boto3, consult the official documentation.
- Do not forget to add the following environment variables:
NodeJS
We use the NodeJS aws-sdk library to push a file to an Object storage bucket using the S3 protocol. You can run these steps on your local computer or any Linux Instance.
- Download the aws-sdk:
npm i aws-sdk
- Write your code.
- Read a bucket object:
const AWS = require ('aws-sdk');
const BUCKET_NAME = process.env.BUCKET_NAME;
const S3_ENDPOINT_URL= process.env.S3_ENDPOINT_URL;
const ID = process.env.ACCESS_KEY_ID;
const SECRET = process.env.ACCESS_KEY;
// Create S3 service object
const s3 = new AWS.S3({
endpoint: S3_ENDPOINT_URL,
credentials: {
accessKeyId: ID,
secretAccessKey: SECRET,
}
});
//Create handler
module.exports.handle = async (event, context, callback) => {
const params = {
Bucket: BUCKET_NAME,
Key: 'functions.txt'
};
try {
const data = await s3.getObject(params).promise();
console.log(`Successfully read file from ${BUCKET_NAME}`) ;
const objectData = data.Body.toString('utf-8');
return {
statusCode: 200,
body: JSON.stringify({
message: objectData
}),
headers: {
"Content-Type": "application/json",
}
};
} catch (e) {
return {
statusCode: 500,
body: JSON.stringify({
message: `Could not retrieve file from S3: ${e.message}`
}),
headers: {
"Content-Type": "application/json",
}
};
};
};- Write to a bucket object:
const AWS = require ('aws-sdk');
const BUCKET_NAME = process.env.BUCKET_NAME;
const S3_ENDPOINT_URL= process.env.S3_ENDPOINT_URL;
const ID = process.env.ACCESS_KEY_ID;
const SECRET = process.env.ACCESS_KEY;
// Create S3 service object
const s3 = new AWS.S3({
endpoint: S3_ENDPOINT_URL,
credentials: {
accessKeyId: ID,
secretAccessKey: SECRET,
}
});
module.exports.handle = async (event, context, callback) => {
const params = {
Bucket: BUCKET_NAME,
Key: 'nodejs14file.txt',
Body: "Hello World from NodeJs 14"
};
try {
const putResult = await s3.putObject(params).promise();
message=`File uploaded successfully at ${BUCKET_NAME}`;
} catch (error) {
console.log(error);
return {
statusCode: 500,
body: JSON.stringify({
message: "Error while uploading file, check logs for more information",
}),
headers: {
"Content-Type": "application/json",
},
};
}
return {
statusCode: 200,
body: JSON.stringify({
message: message,
}),
headers: {
"Content-Type": "application/json",
},
}
} - Push your code to Serverless Function using a zip-file or the serverless framework (the latter is recommended).
For more information on aws-sdk, consult the official documentation.