For illustration purposes, this script encodes a video using the x264 video codec and the AAC audio codec. Encoding settings can be modified using command-line parameters to FFMPEG.
Encode videos using Serverless Jobs and FFMPEG
- serverless
- jobs
- ffmpeg
- video
- encoding
This tutorial demonstrates the process of encoding videos retrieved from Object Storage using Serverless Jobs: media encoding is a resource-intensive task over prolonged durations, making it suitable for Serverless Jobs. The job takes a video file as its input, encodes it using a Docker image based on FFMPEG, then uploads the encoded video back to the S3 bucket.
Before you start
To complete the actions presented below, you must have:
- A Scaleway account logged into the console
- Owner status or IAM permissions allowing you to perform actions in the intended Organization
- An Object Storage bucket
- A valid API key
- Installed Docker engine
Creating the job image
The initial step involves defining a Docker image for interacting with the S3 Object Storage using MinIO and performing a video encoding task using FFMPEG.
-
Create a bash script
encode.sh
with the following content:#!/bin/shset -eecho "Configuring S3 access for MinIO"mc config host add scw "https://$JOB_S3_ENDPOINT/" "$JOB_S3_ACCESS_KEY" "$JOB_S3_SECRET_KEY"echo "Downloading the file from S3"mc cp "scw/$JOB_INPUT_PATH/$JOB_INPUT_FILENAME" "/tmp/$JOB_INPUT_FILENAME"echo "Encoding the file"ffmpeg -i "/tmp/$JOB_INPUT_FILENAME" -vcodec libx264 -acodec aac "/tmp/$JOB_OUTPUT_FILENAME"echo "Uploading the encoded file to the S3"mc cp "/tmp/$JOB_OUTPUT_FILENAME" "scw/$JOB_OUTPUT_PATH/$JOB_OUTPUT_FILENAME"That bash script downloads a video from an S3 bucket, encodes that video using FFMPEG, and then uploads the encoded video into the bucket, by leveraging a couple of environment variables which will be detailed in the following sections.
Note -
Use that script as an entrypoint for a new Docker image, by creating the following Dockerfile:
FROM linuxserver/ffmpeg:amd64-latest# Install the MinIO S3 clientRUN curl https://dl.min.io/client/mc/release/linux-amd64/mc -o /usr/local/bin/mcRUN chmod +x /usr/local/bin/mc# Define the image entrypointCOPY encode.sh /encode.shRUN chmod +x /encode.shENTRYPOINT /encode.shThis Dockerfile uses
linuxserver/ffmpeg
as a base image bundled with FFMPEG along with a variety of encoding codecs and installs MinIO as a command-line S3 client to copy files over Object Storage. -
Build and push the image to your Container Registry:
docker build . -t <registry and image name>docker push <registry and image name>NoteYou can find the name and endpoint of your Container Registry from the Scaleway console
Creating the serverless job
-
In the Scaleway Console, click Jobs in the Serverless section of the side menu, then Create job to display the job creation wizard.
-
Select the Scaleway Container Registry and your Container Registry namespace from the drop-down list, then select the image from the previous section.
-
Choose your job name, description, and region. In the Resources section, pick a reasonable amount of resources.
NoteThe allocated CPU defines the speed of the encoding process. The allocated memory depends on the encoding codec and bitrate. For instance, encoding a Full HD video with the x264 codec takes about 3 GB of memory.
-
Toggle the Advanced options section and add 3 environment variables:
JOB_S3_ENDPOINT
is your S3 endpoint (e.g.s3.nl-ams.scw.cloud
).JOB_S3_ACCESS_KEY
is your API access key.JOB_S3_SECRET_KEY
is your API secret key.
-
Leave the other sections empty, then click
Create job
.
Triggering the serverless job
Ensure that your S3 bucket contains at least one video that can be encoded.
-
In the Scaleway Console, go to Serverless Jobs and click on the name of your job. The job Overview tab displays.
-
Click the Actions button, then click Run job with options in the drop-down menu.
-
Add 4 environment variables:
JOB_INPUT_PATH
is the folder containing the video to encode, including your S3 bucket name.JOB_INPUT_FILENAME
is the file name of the video to encode, including the file extension.JOB_OUTPUT_PATH
is the folder containing the encoded video that will be uploaded, including your S3 bucket name.JOB_OUTPUT_FILENAME
is the file name of the encoded video that will be uploaded.
-
Click Run job.
The progress and details for your Job run can be viewed in the Job runs section of the job Overview tab in the Scaleway console. You can also access the detailed logs of your job in Cockpit.
Once the run status is Succeeded, the encoded video can be found in your S3 bucket under the folder and file name specified above in the environment variables.
Your job can also be triggered through the Scaleway API using the same environment variables:
curl -X POST \-H "X-Auth-Token: <API Key>" \-H "Content-Type: application/json" \-d '{ "environment_variables": { "JOB_INPUT_FILENAME": "...", "JOB_INPUT_PATH" : "...", "JOB_OUTPUT_FILENAME" : "...", "JOB_OUTPUT_PATH" : "..." } }' \"https://api.scaleway.com/serverless-jobs/v1alpha1/regions/<Region>/job-definitions/<Job ID>/start"