
Cold Storage
What is the cold storage Glacier storage class?
The cold storage “Glacier” class of Object Storage gets its name in opposition to “hot data”. Hot data is the type that is frequently accessed. Cold storage is accessed less frequently and does not require fast access like hot data.
Scaleway’s cold storage service Glacier is engineered on specific hardware:
- SMR disks that increase storage density and overall per-drive storage capacity
- motherboards manufactured internally that can power disks on/off on-demand with a SATA bus tree matrix.
This particular hardware configuration allows us to offer competitive pricing for use-cases that do not need the performance of other Object storage classes. The counterpart is that data stored on a cold storage service such as Scaleway Glacier has limitations in relation to data retrieval.
When should I use Glacier class instead of Standard class?
Scaleway Glacier is engineered for use-cases that can be described as “deep archiving”. It is a durable Object Storage class designed for data archiving, for example legal documents or long-term backups that do not require immediate access.
We recommend you use Glacier to store average sized files (larger than 1 MB), as opposed to several small files.
Standard class is ideal for storing frequently accessed data and files, such as static assets, on-demand content such as streaming, Content Delivery Networks (CDN) or work related documents.
I have started the restoration of an object, why is it not available instantaneously?
An object stored in Glacier class is listed but cannot be downloaded instantly. It needs to be restored to Standard class first. We recommend that you use Glacier to archive data that you may need only once or twice within a decade.
The time it takes to restore an object depends on the size of the object and on if multipart is configured. It can take anywhere from a few minutes to 24 hours for restore to start (for objects larger than 1 MB). Unfortunately, we are unable to provide estimates of how long data retrieval takes for each use case.
To facilitate the steps of restoration and ensure a fast restitution of your data, we recommend that you use average sized files (larger than 1 MB).
Find out more about how to restore an object from glacier in our dedicated page.
How much do I pay to archive objects to Glacier or to restore them to Standard class?
Nothing. Switching the object class from STANDARD
to GLACIER
is free.
Restoring objects from GLACIER
to STANDARD
is also free of charge.
Can I upload an object directly to Glacier?
Yes, you can select the storage class (STANDARD
or GLACIER
) when uploading objects using the Scaleway console. It is also possible to do so using the API. For more information, refer to our feature documentation.
Why are Glacier objects stored in Paris DC4? Is it accessible from buckets in all locations?
Our Paris underground databunker (DC4) provides an increased layer of security by being 25 meters deep below the surface in a bunker where the Glacier storage class cluster is available. This storage cluster is available from any bucket in Paris and Amsterdam and all objects can change the storage class to Glacier.
It is not yet possible to transfer files to Glacier in Warsaw, Poland.
When are lifecycle management rules applied?
The expiration & transition rules are implemented every day at midnight UTC, but the actual action might take between one minute and twenty-four hours* after implementation.
For example, a user wants to configure a lifecycle rule that moves an object after two days to Glacier:
- lifecycle rules are checked every day at midnight.
- the user uploaded a file on April 6th, at 2:00 pm UTC.
- the rule is triggered on April 8th, at 11:59 pm UTC.
- the application of the rule can take up to 24 hours ⇒ the object will be transferred to Glacier no later than 11:59 pm UTC on April 9th.
When a lifecycle rule is set for a very large amount of objects (several 100000s) simultaneously, the rule might take longer to be applied.
How do I encrypt my data?
For increased security, we recommend you encrypt your data before sending it to Scaleway Glacier. Encryption should be done on the client-side to guarantee that there is no evidence of your private key stored on the cluster. Data can be encrypted using third-party solutions, like GPG. It is possible to automatize your backups and to encrypt them using a solution like Duplicity or Rclone using the crypt backend.
Can I manage my storage and archiving constraints related to the GDPR with Scaleway Glacier?
The GDPR requires that the life span of data be defined in two distinct cases:
- the deletion of data after a certain period of time
- the archiving of data that must legally be kept
Lifecycle management features meet both of these requirements by allowing you to set a timeframe beyond which your data is archived or deleted.
What is the maximum data volume and object size supported by Scaleway Glacier?
There is no maximum data size, Glacier accepts an unlimited number of objects up to 5 TB each. Large objects can be uploaded using multipart uploads.
Why is Scaleway Glacier technology much more reliable than tape storage technologies?
Access to tape storage platforms requires a tape library to identify the cartridge on which the data is stored. Furthermore, retrieving data stored using tape storage requires reading several kilometers of tape.
Scaleway Glacier offers extra benefits compared to tape storage:
- Scaleway Glacier’s hard disk drive-based technology offers much better random data access time
- Scaleway Glacier enables parallel writing to multiple disks simultaneously, providing very high read and write speeds.
- redundancy is achieved by advanced parity mechanisms (6+3) on many media, whereas tape technology does not allow this and is limited to CRC.
- there is no risk of mechanical part failure (ex: robot) or wear and tear (tape drive).
Can I change the storage class of an entire bucket from Standard to Glacier?
Yes, a best practice is to create a transition lifecycle rule with an empty prefix. In this case, all files contained within the selected bucket will have their storage class altered automatically on the dates stipulated by you. However, due to S3 Protocol restrictions, a lifecycle rule cannot be created to modify the storage class from Glacier to Standard.