Setting up a Nginx reverse proxy with Object Storage

Object Storage Proxy Overview

The Scaleway Object Storge service lets you store infinite data in buckets. You can either access the data directly via your bucket. This consumes data transfer and will be deducted from your monthly package or be billed in case you exceed your included data transfer.

It is possible to access your bucket via a compute instance as a proxy. Transfer of data between your bucket and the compute platform is free of charge and you can use the bandwidth included with your instance to distribute the files in your bucket.

We use a pre-build and containerized Nginx server running on Docker to access the bucket.


Installing Docker

We use a pre-build Docker container with Nginx to run the application. Therefore it is required that Docker Community Edition is installed on the instance.

Configuring The Proxy

Important: The docker image runs a recompiled version of Nginx. The standard build of Nginx available in the distribution repository does not support this feature.

1 . The image requires a config file in the container at: /nginx.conf. Use the -v option to mount one from your host.

docker run -p 8000:8000 -v /path/to/nginx.conf:/nginx.conf coopernurse/nginx-s3-proxy

The Nginx configration file should look like the following example:

Important: The following configuration makes all your bucket readable, regardless of S3 rights. Make sure not to store any sensitive data if you want to expose it using the Nginx proxy.

worker_processes 2;
pid /run/;
daemon off;

events {
	worker_connections 768;

http {
	sendfile on;
	tcp_nopush on;
	tcp_nodelay on;
	keepalive_timeout 65;
	types_hash_max_size 2048;
	server_names_hash_bucket_size 64;

	include /usr/local/nginx/conf/mime.types;
	default_type application/octet-stream;

	access_log /usr/local/nginx/logs/access.log;
	error_log  /usr/local/nginx/logs/error.log;

	gzip on;
	gzip_disable "msie6";
	gzip_http_version 1.1;
	gzip_types text/plain text/css application/json application/x-javascript text/xml application/xml application/xml+rss text/javascript;

    proxy_cache_lock on;
    proxy_cache_lock_timeout 60s;
    proxy_cache_path /data/cache levels=1:2 keys_zone=s3cache:10m max_size=30g;

    server {
        listen     8000;

        location / {
            # Limit all HTTP requests except for GET: 
            limit_except GET { deny  all; }

            # Configure the URL of your bucket:

            # Set the Access key:
            aws_access_key scw-access-key;

            # Set the Secret key: 
            aws_secret_key scw-secret-key;

            # Configure the name of your bucket: 
            s3_bucket scw-bucket;

            proxy_set_header Authorization $s3_auth_token;
            proxy_set_header x-amz-date $aws_date;

            proxy_cache        s3cache;
            proxy_cache_valid  200 302  24h;

Replace the following parameters in the file:

  • scw-bucket: The name of your Scaleway Object Storage bucket
  • scw-access-key: Your Access key
  • scw-secret-key: Your Secret key

2 . You can now access your bucket via a web browser by typing : http://your_server_ip:8000

3 . (Optional) If you want to cache data locally, bind a path to /data/cache:

docker run -p 8000:8000 -v /path/to/nginx.conf:/nginx.conf -v /my/cache/path:/data/cache coopernurse/nginx-s3-proxy

Discover a New Cloud Experience

Deploy SSD Cloud Servers in seconds.