Multipart Uploads allows you to upload large files (up to 5 TB) to the Object Storage platform in multiple parts. This allows faster, more flexible uploads.
If you do not complete a multipart upload, all the uploaded parts will still be stored and counted as part of your storage usage. Multipart uploads can be aborted manually via the API and CLI or automatically using a Lifecycle rule.
If you use the API or the AWS CLI, you will have to abort each incomplete multipart upload independently. However there is an easier and faster way to abort multipart uploads, using the open-source S3-compatible client mc, from MinIO. In this tutorial, we show you how to use mc to abort and clean up all your incomplete multipart uploads at once.
1 . Follow the instructions given in the official MinIO documentation here to install the MinIO client (
mc) for your OS.
For example, on Linux:
wget https://dl.min.io/client/mc/release/linux-amd64/mc chmod +x mc
Note: Make sure that you download the binary somewhere that is in your
2 . Run
mc --help to check it has installed.
3 . From the same page as before, follow the instructions to add a cloud storage service:
mc alias set <ALIAS> <YOUR-S3-ENDPOINT> <YOUR-ACCESS-KEY> <YOUR-SECRET-KEY> --api <API-SIGNATURE>
For example, if your Scaleway Object Storage endpoint is in the fr-par region, you can use the following command (replace the two fields in pointy brackets with your access key and secret key):
mc alias set s3 https://s3.fr-par.scw.cloud <YOUR-ACCESS-KEY> <YOUR-SECRET-KEY> --api S3v4
4 . You should now be able to run some basic commands, eg:
mc ls s3
mc ls s3/<your-bucket-name>
mc rm s3/<your-bucket-name>/1.pdf
…plus all other commands detailed via
mc --help .
1 . Use the basic
mc ls command we saw above but with the added
-I (incomplete) flag to list all incomplete multipart uploads:
mc ls s3/<mybucketname> -I
2 . Use the basic
mc rm command we saw above, but with the added
r (recursive) and
--force flags to abort and clean up all incomplete multipart uploads:
mc rm s3/<mybucketname> -I -r --force
Your incomplete multipart uploads are now aborted and all the parts cleaned up, in one simple step!