Scaleway

Migrating Object Storage data with Minio Client

2019-03-20T00:00:00+00:00

Minio Client Overview

Minio Client provides a modern alternative to UNIX commands like ls, cat, cp, mirror, diff etc. It is able to communicate with any S3 compatible cloud storage provider and can be used to migrate data from one region to another.

Requirements

You have an account and are logged into console.scaleway.com

You have configured your SSH Key

You have object storage buckets in two different regions

Installing the Minio Client

1 . Connect to your server as root via SSH.

2 . Update the APT packet cache and the software already installed on the instance:

apt update && apt upgrade -y

3 . Download the Minio Client:

wget https://dl.minio.io/client/mc/release/linux-amd64/mc -P /usr/bin/

4 . Make the file executable:

chmod +x /usr/bin/mc

Configuring the Minio Client and Migrating Data

1 . Create a directory to store the configuration file of Minio client:

mkdir ~/.mc/

2 . Create a configuration file for Minio Client:

touch ~/.mc/config.json

3 . Open the file ~/.mc/config.json in a text editor, and edit it as follows:

{
	"version": "8",
	"hosts": {
		"fr-par": {
			"url": "https://s3.fr-par.scw.cloud",
			"accessKey": "<ACCESS_KEY>",
			"secretKey": "<SECRET_KEY>",
			"api": "S3v2"
		},
		"nl-ams": {
			"url": "http://s3.nl-ams.scw.cloud",
			"accessKey": "<ACCESS_KEY>",
			"secretKey": "<SECRET_KEY>",
			"api": "S3v2"
		}
  }
}

Important: Replace <ACCESS_KEY> and <SECRET_KEY> with the credentials of your API token.

3 . Start the migration of your data:

mc cp --recursive nl-ams/oldbucket fr-par/newbucket

The above command migrates the content from the bucket oldbucket in the region NL-AMS to the bucket newbucket in the region FR-PAR. Edit the command towards your needs before running it.

Minio Client displays a status bar during the transfer, allowing to observe the status of the migration:

Ubuntu ML

2019-03-14T00:00:00+00:00

Ubuntu is a Debian-based Linux operating system, with Unity as its default desktop environment. It is based on free software and named after the Southern African philosophy of ubuntu (literally, “human-ness”), which often is translated as “humanity towards others” or “the belief in a universal bond of sharing that connects all humanity”.

We’ve included everything you need to make full use of your GPU, and it comes with conda pre-installed.

Configuring NGINX with Let’s Encrypt

2019-02-28T00:00:00+00:00

Let's Encrypt Overview

Let’s Encrypt is a Certificate Authority (CA) that provides free TLS/SSL certificates to enable HTTPS on web servers. They provide a software client called Certbot to automatize most of the steps required to obtain a certificate and to configure it within the Nginx web server.

Requirements

You have an account and are logged into console.scaleway.com

You have configured your SSH Key

You have a registered domain name pointed to your web server

You have a virtual cloud server running on Ubuntu Bionic Beaver (18.04)

Installing the NGINX web server

1 . Connect to your server as root via SSH.

2 . Update the APT packet cache and the software already installed on the instance:

apt update && apt upgrade -y

3 . Install the Nginx web server via APT:

apt install nginx -y

Configuring a Server Block

The default installation of Nginx on Ubuntu Bionic Beaver comes with one pre-defined server block that listens on port 80. While it is possible to host a single site by putting the content into the directory /var/www/html, it would not be possible to host multiple sites one the same instance. To avoid this problem, server blocks can be configured. These specify a directory for the content that will be served when requesting a specific site. The content of /var/www/html will be served as the default directory, if a request does not match any other site configured.

Important: In this tutorial the domain name example.com will be used. You should replace it with your own domain name while setting up your instance.

1 . Create the directory for your domain name. Using the -p flag will create any required parent directory in case they do not exist:

mkdir -p /var/www/example.com/html

2 . Create a place holder page that will be displayed when accessing your domain:

nano /var/www/example.com/html/index.html

3 . Put some content like the following into the file which will be displayed to a user when requesting your site. Save and quit nano once you have edited the file:

<html>
    <head>
        <title>Welcome to example.com</title>
    </head>
    <body>
        <h1>Hello World!</h1>
        <p>You have accessed the example.com website.</p>
    </body>
</html>

4 . To serve the site, a server block is required. Create the block in the directory /etc/nginx/sites-available/:

nano /etc/nginx/sites-available/example.com

And put the following content into it:

server {
        listen 80;
        listen [::]:80;

        root /var/www/example.com/html;
        index index.html index.htm;

        server_name example.com www.example.com;

        location / {
                try_files $uri $uri/ =404;
        }
}

Important: Edit the lines root and server_name according to your domain name.

5 . Enable the file by linking it to the sites-enabled directory, to enable the server block during Nginx startup:

ln -s /etc/nginx/sites-available/example.com /etc/nginx/sites-enabled/

6 . Verify if there are no errors in the configuration file before restarting Nginx:

nginx -t

7 . Restart the Nginx web server:

systemctl restart nginx

8 . When typing http://example.com in your browser, you should see your newly created place holder page:

Installing Cerbot and Obtaining a Certificate

1 . Certbot is in active development and it is possible, that the packages included in Ubuntu are already Outdated. Therefore add the Certbot repository to apt to download the latest release of the software:

add-apt-repository ppa:certbot/certbot

Press Enter when asked to confirm the action.

2 . Install Certbot for Nginx:

apt install python-certbot-nginx -y

3 . Launch the certificate generation:

certbot --nginx -d example.com -d www.example.com

Important: The parameter -d specifies the domains for which you want to request a certificate. Make sure to replace it with your own domain name. Also keep in mind that if you want to have a certificate for example.com and for www.example.com you have to specify both.

When running Certbot for the first time, you will be asked to enter your email address. Confirm it by pressing Enter on your keyboard.

4 . Once confirmed Certbot will run a challenge and request the certificate. When asked to redirect all traffic to HTTPS, press 2, then Enter on your keyboard:

Please choose whether or not to redirect HTTP traffic to HTTPS, removing HTTP access.
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
1: No redirect - Make no further changes to the webserver configuration.
2: Redirect - Make all requests redirect to secure HTTPS access. Choose this for
new sites, or if you're confident your site works on HTTPS. You can undo this
change by editing your web server's configuration.
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Select the appropriate number [1-2] then [enter] (press 'c' to cancel):

Certbot will now reconfigure Nginx and once you see the following message your certificate is successfully installed:

IMPORTANT NOTES:
 - Congratulations! Your certificate and chain have been saved at:
   /etc/letsencrypt/live/example.com/fullchain.pem
   Your key file has been saved at:
   /etc/letsencrypt/live/example.com/privkey.pem
   Your cert will expire on 2019-04-15. To obtain a new or tweaked
   version of this certificate in the future, simply run certbot again
   with the "certonly" option. To non-interactively renew *all* of
   your certificates, run "certbot renew"
 - Your account credentials have been saved in your Certbot
   configuration directory at /etc/letsencrypt. You should make a
   secure backup of this folder now. This configuration directory will
   also contain certificates and private keys obtained by Certbot so
   making regular backups of this folder is ideal.
 - If you like Certbot, please consider supporting our work by:

   Donating to ISRG / Let's Encrypt:   https://letsencrypt.org/donate
   Donating to EFF:                    https://eff.org/donate-le

You can now open your web browser and type https://example.com to verify that your connection is secure:

Installing Mattermost Messaging on Ubuntu 18.04

2019-01-22T00:00:00+00:00

Mattermost Overview

Mattermost is an open-source messaging tool for inter-team communication, released under the MIT license and available both in a free team and paid enterprise edition. This tutorial will show you how to install a Mattermost instance with a NGINX frontend proxy and a MariaDB database.

Requirements

You have an account and are logged into console.scaleway.com

You have configured your SSH Key

You have a cloud compute instance running on Ubuntu Bionic Beaver

Installing Mattermost

1 . Connect to your compute instance via SSH.

2 . Update the APT packet cache and the software already installed on the instance:

apt update && apt upgrade -y

3 . Install the MariaDB database server:

apt install mariadb-server

4 . Initialize the database server:

mysql_secure_installation

5 . Log into mysql as root:

mysql

6 . Create a database for Mattermost:

create user 'mattermost'@'localhost' identified by 'a-secure-password';

Important: Replace a-secure-password with a strong password, as it will be used to login to MariaDB

7 . Create a database for Mattermost whilst being logged into the MariaDB shell:

create database mattermost;

8 . Grant access privileges to the user mattermost:

grant all privileges on mattermost.* to 'mattermost'@'localhost';

9 . Exit the MariaDB shell:

exit

10 . Download the latest version of the Mattermost server to your instance. At the time writing this tutorial it is version 5.7:

wget https://releases.mattermost.com/5.7.0/mattermost-5.7.0-linux-amd64.tar.gz

11 . Extract the Mattermost server files:

tar -xvzf mattermost*.gz

12 . Move the extracted files to the /opt directory:

mv mattermost /opt

13 . Create the storage directory for Mattermost:

mkdir /opt/mattermost/data

Important: This directory will contain all data sent to your Mattermost instance (Messages, Images, Files etc.). Make sure that it has enough space available for the anticipated number of contents sent to your instance.

14 . Set up a system user and group to be in charge of running the service and set the ownership and permissions as following:

Create a system user and group called mattermost:
```
useradd --system --user-group mattermost
```
Set the ownership of the Mattermost files to the previously created user and group:
```
chown -R mattermost:mattermost /opt/mattermost
```
Give write permissions to the mattermost group:
```
chmod -R g+w /opt/mattermost
```

15 . Open the file /opt/mattermost/config/config.json with your favourite text editor and edit the database settings as following:

Set the "DriverName" to mysql
Configure "DataSource" as following, replacing with the password you have set for your database:

"mattermost:<a-secret-password>@tcp(localhost:3306)/mattermost?charset=utf8mb4,utf8&readTimeout=30s&writeTimeout=30s"

16 . Start the Mattermost server manually to check everything is working:

sudo -u mattermost ./bin/mattermost

During startup the database structure of Mattermost will be configured and you will see some log messages as well as a notice: Server is listening on :8065. This means your Mattermost server is now ready to be used. Stop the process by pushing Control+C on your keyboard.

17 . Open your favourite text editor and create the file /lib/systemd/system/mattermost.service with the following content to create a systemd script to run the application as a service on your instance:

[Unit]
Description=Mattermost
After=network.target
After=mysql.service
Requires=mysql.service

[Service]
Type=notify
ExecStart=/opt/mattermost/bin/mattermost
TimeoutStartSec=3600
Restart=always
RestartSec=10
WorkingDirectory=/opt/mattermost
User=mattermost
Group=mattermost
LimitNOFILE=49152

[Install]
WantedBy=mysql.service

18 . Reload the systemd configuration:

systemctl daemon-reload

19 . Start the service:

systemctl start mattermost.service

20 . Enable the service to start Mattermost during the startup of the instance:

systemctl enable mattermost.service

You can now connect to your Mattermost server at http://your_servers_ip:8065 and create your first user and do some initial configuration and setup.

Installing a Nginx Proxy with Let's Encrypt

For security reasons it is recommended to put the application behind a proxy. Nginx is being used as frontend for the application. To provide an encrypted connection, certbot is used to generate a free Let’s Encrypt SSL certificate to encrypt the connection to the instance.

1 . Install Nginx and Certbot:

apt install nginx python-certbot-nginx -y

2 . Create a new server block for Mattermost (replace mattermost.example.com with the FQDN of your instance):

touch /etc/nginx/sites-available/mattermost.example.com

and put the following content into the file:

upstream backend {
   server 127.0.0.1:8065;
   keepalive 32;
}

proxy_cache_path /var/cache/nginx levels=1:2 keys_zone=mattermost_cache:10m max_size=3g inactive=120m use_temp_path=off;

server {
   listen 80;
   server_name    mattermost.example.com;

   location ~ /api/v[0-9]+/(users/)?websocket$ {
       proxy_set_header Upgrade $http_upgrade;
       proxy_set_header Connection "upgrade";
       client_max_body_size 50M;
       proxy_set_header Host $http_host;
       proxy_set_header X-Real-IP $remote_addr;
       proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
       proxy_set_header X-Forwarded-Proto $scheme;
       proxy_set_header X-Frame-Options SAMEORIGIN;
       proxy_buffers 256 16k;
       proxy_buffer_size 16k;
       client_body_timeout 60;
       send_timeout 300;
       lingering_timeout 5;
       proxy_connect_timeout 90;
       proxy_send_timeout 300;
       proxy_read_timeout 90s;
       proxy_pass http://backend;
   }

   location / {
       client_max_body_size 50M;
       proxy_set_header Connection "";
       proxy_set_header Host $http_host;
       proxy_set_header X-Real-IP $remote_addr;
       proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
       proxy_set_header X-Forwarded-Proto $scheme;
       proxy_set_header X-Frame-Options SAMEORIGIN;
       proxy_buffers 256 16k;
       proxy_buffer_size 16k;
       proxy_read_timeout 600s;
       proxy_cache mattermost_cache;
       proxy_cache_revalidate on;
       proxy_cache_min_uses 2;
       proxy_cache_use_stale timeout;
       proxy_cache_lock on;
       proxy_http_version 1.1;
       proxy_pass http://backend;
   }
}

3 . Remove the default server block of Nginx:

rm /etc/nginx/sites-enabled/default

4 . Enable the Mattermost server block by linking it to the sites-enabled directory:

ln -s /etc/nginx/sites-available/mattermost.example.com /etc/nginx/sites-enabled/mattermost.example.com

5 . Restart Nginx:

sudo service nginx restart

6 . Run Certbot:

certbot

Important: The application will ask you several questions. You have to confirm the steps as described. For more information refer to our documentation about certbot.

7 . Connect to your instance by going to https://your_servers_ip. The Mattermost login screen will appear. Connect yourself with the user you have created during the initial setup of the application.

8 . Once logged in click on the menu button, then on System Console to enter the configuration interface of Mattermost.

9 . Inside the general settings section, click on Configuration.

10 . Configure the Listen Address as 127.0.0.1:8065:

This prevents direct connections to Mattermost from other machines.

11 . Restart the Mattermost service:

systemctl restart mattermost.service

12 . Reload the Mattermost application in your browser. The basic configuration has been done now, you can start chatting:

You may configure advanced settings, like email notifications and other customizations of your Mattermost application. For more information, refer to the official documentation.

How to Create your first GPU Instance

2019-01-21T00:00:00+00:00

GPU Overview

GPU servers are designed for artificial intelligence, machine learning and complex modeling. They are equipped with high-end GPUs and huge quantity of cores, memory and storage. In short, GPUs are optimized for taking huge batches of data and performing the same operation over and over very quickly.

There are six steps to provision a new GPU server:

Choose an image
Choose a region
Choose a GPU Instance
Configure storage
Configure advanced options
Name and tag the instance

Important: Your SSH public keys are fetched during the boot process.

If you upload them to the management console after your server is booted, they will not be added to your authorized_keys file.

If you do not want the keys to be downloaded during the next boot, execute the following command on your server:

root@c1-X-Y-Z-T:~# echo manual > /etc/init/ssh-keys.override

Requirements

You have an account and are logged into console.scaleway.com

You have configured your SSH Key

Creating a GPU Server

In the Compute section of the side menu, click Compute. If you do not have an instance already created, the product presentation is displayed.

Click Create a server. The creation page displays.

Note: GPU instances are currently available on request. Please open a support ticket to request a GPU instance quota for your account.

Choosing an Image

Start by choosing an image for the cloud instance.

You can choose an image from five sources:

OS Images: Choose your favourite basic Linux distribution, ready to be configured by you.
GPU OS: Choose an Ubuntu Bionic (18.04) optimized for Machine Learning. It comes preinstalled with everything you need to fully use your GPU and Anaconda, the most complete python distribution.
InstantApps: We provide an up-to-date list different pre-installed applications.
My images: You can populate your own list of server templates. See also Create your own image
Snapshots: You can recover a server from a previously saved state. See also Backup your data with snapshots

Choosing a Region

You can choose a region in which your instance will be deployed geographically.

Currently we provide the following locations for GPU instances:

PAR1: Paris, France

Choosing the GPU Instance

Click on the GPU tab to deploy a GPU instance:

Currently RENDER-S instances are available, featuring a Nvidia Tesla P100 16GB, 10 vCores and 400GB SSD storage space.

Configuring Volumes

Each RENDER-S instance comes with a fixed volume size of 400GB.

Configuring Advanced Options

You can configure advanced options of your server, including:

IPv6 connectivity
Cloud Init

Naming and tagging the instance

Edit the following information about the instance:

The instances name
The tag you want to assign to it (optional). Tags let you organize your servers, you can assign any tag to each server.

Starting the Instance

Click the “Create a new Server” button. This action launches the create server action. The instance will be ready soon after.

Logging to your GPU Instance

On a Mac, Windows or Linux computer, you can follow the tutorial on Logging into the Instance.

Going Further

Installing OpenVPN on Ubuntu Bionic Beaver

2019-01-16T00:00:00+00:00

OpenVPN Overview

OpenVPN is an open-source software to run a virtual private network (VPN) to create secure point-to-point or site-to-site connections in routed or bridged configurations. The software uses a proprietary security protocol that uses SSL/TLS for key exchange.

Requirements

You have an account and are logged into console.scaleway.com

You have configured your SSH Key

You have a virtual cloud server running on Ubuntu Bionic Beaver

Installing Easy-RSA

The first step in building an OpenVPN configuration is to establish a PKI (Public Key Infrastructure). It is composed of the following elements:

a public and private key for the server and each client
the certification authority (CA) and the key used to identify servers as well as client certificate

Important: Important: For security reasons, it is recommended to run the CA on a different computer from the OpenVPN server.

OpenVPN supports two-way certificate-based authentication, this means that the client must authenticate the server certificate and the server must authenticate the client certificate before mutual trust is established.

Both the server and the client will authenticate each other. First, the certificate needs to be signed by the certification authority (CA) then, the information in the header (common name of the certificate or the certificate type) of the authenticated certificate can be tested.

1 . Connect to your server via SSH.

2 . Update the APT packet cache and the software already installed.

apt update && apt upgrade -y

3 . Install easy-RSA:

apt install easy-rsa -y

4 . Copy the easy-RSA directory into /etc/openvpn:

mkdir -p /etc/openvpn/easy-rsa/
cp -r /usr/share/easy-rsa/* /etc/openvpn/easy-rsa/

5 . Modify /etc/openvpn/easy-rsa/vars by adjusting the following values so they match your configuration:

# These are the default values for fields
# which will be placed in the certificate.
# Don't leave any of these fields blank.
export KEY_COUNTRY="US"
export KEY_PROVINCE="CA"
export KEY_CITY="SanFrancisco"
export KEY_ORG="Fort-Funston"
export KEY_EMAIL="me@myhost.mydomain"
export KEY_OU="MyOrganizationalUnit"

6 . Run the following commands to generate the Certificate Authority and a key:

cd /etc/openvpn/easy-rsa/
cp openssl-1.0.0.cnf openssl.cnf
source vars
./clean-all
./build-ca

7 . Generate a certificate and a private key for the server:

./build-key-server openvpn-server

As in the previous step, most of the default settings can be kept. Two requests require positive answers: “Sign the certificate? [y/n]” and “1 out of 1 certificate requests certified, commit? [y/n]”.

8 . Generate a Diffie Hellman key for the OpenVPN server:

./build-dh

9 . All certificates and keys have been generated in the keys/ subdirectory. It is recommended to copy them into the directory /etc/openvpn/.

cd keys/
cp ca.crt  dh2048.pem  openvpn-server.crt  openvpn-server.key /etc/openvpn/

10 . The VPN client also needs a certificate to authenticate against the server. It is recommended to create a certificate for each client by running the following commands:

cd /etc/openvpn/easy-rsa/
source vars
./build-key openvpn-client01

When prompt, confirm the follwoing questions: “Sign the certificate? [y/n]” and “1 out of 1 certificate requests certified, commit? [y/n]”.

Copy the following files to the client:

/etc/openvpn/ca.crt
/etc/openvpn/easy-rsa/keys/openvpn-client01.crt
/etc/openvpn/easy-rsa/keys/openvpn-client01.key

Important: Since client certificates and keys are only required on the client machine, it is recommended to delete them from the server for security reasons.

Installing and Configuring OpenVPN

As you have generated both, the servers and the client certificate, start configuring the OpenVPN server.

1 . Start by installing the OpenVPN server:

apt install openvpn -y

2 . Create a configuration file /etc/openvpn/openvpn-server.conf for the server:

#################################################
# Sample OpenVPN 2.0 config file for            #
# multi-client server.                          #
#                                               #
# This file is for the server side              #
# of a many-clients <-> one-server              #
# OpenVPN configuration.                        #
#                                               #
# OpenVPN also supports                         #
# single-machine <-> single-machine             #
# configurations (See the Examples page         #
# on the web site for more info).               #
#                                               #
# This config should work on Windows            #
# or Linux/BSD systems.  Remember on            #
# Windows to quote pathnames and use            #
# double backslashes, e.g.:                     #
# "C:\\Program Files\\OpenVPN\\config\\foo.key" #
#                                               #
# Comments are preceded with '#' or ';'         #
#################################################

# Which local IP address should OpenVPN
# listen on? (optional)
;local a.b.c.d

# Which TCP/UDP port should OpenVPN listen on?
# If you want to run multiple OpenVPN instances
# on the same machine, use a different port
# number for each one.  You will need to
# open up this port on your firewall.
port 1194

# TCP or UDP server?
;proto tcp
proto udp

# "dev tun" will create a routed IP tunnel,
# "dev tap" will create an ethernet tunnel.
# Use "dev tap0" if you are ethernet bridging
# and have precreated a tap0 virtual interface
# and bridged it with your ethernet interface.
# If you want to control access policies
# over the VPN, you must create firewall
# rules for the the TUN/TAP interface.
# On non-Windows systems, you can give
# an explicit unit number, such as tun0.
# On Windows, use "dev-node" for this.
# On most systems, the VPN will not function
# unless you partially or fully disable
# the firewall for the TUN/TAP interface.
;dev tap
dev tun

# Windows needs the TAP-Win32 adapter name
# from the Network Connections panel if you
# have more than one.  On XP SP2 or higher,
# you may need to selectively disable the
# Windows firewall for the TAP adapter.
# Non-Windows systems usually don't need this.
;dev-node MyTap

# SSL/TLS root certificate (ca), certificate
# (cert), and private key (key).  Each client
# and the server must have their own cert and
# key file.  The server and all clients will
# use the same ca file.
#
# See the "easy-rsa" directory for a series
# of scripts for generating RSA certificates
# and private keys.  Remember to use
# a unique Common Name for the server
# and each of the client certificates.
#
# Any X509 key management system can be used.
# OpenVPN can also use a PKCS #12 formatted key file
# (see "pkcs12" directive in man page).
ca ca.crt
cert openvpn-server.crt
key openvpn-server.key  # This file should be kept secret

# Diffie hellman parameters.
# Generate your own with:
#   openssl dhparam -out dh2048.pem 2048
dh dh2048.pem

# Network topology
# Should be subnet (addressing via IP)
# unless Windows clients v2.0.9 and lower have to
# be supported (then net30, i.e. a /30 per client)
# Defaults to net30 (not recommended)
;topology subnet

# Configure server mode and supply a VPN subnet
# for OpenVPN to draw client addresses from.
# The server will take 10.8.0.1 for itself,
# the rest will be made available to clients.
# Each client will be able to reach the server
# on 10.8.0.1. Comment this line out if you are
# ethernet bridging. See the man page for more info.
server 10.8.0.0 255.255.255.0

# Maintain a record of client <-> virtual IP address
# associations in this file.  If OpenVPN goes down or
# is restarted, reconnecting clients can be assigned
# the same virtual IP address from the pool that was
# previously assigned.
ifconfig-pool-persist ipp.txt

# Configure server mode for ethernet bridging.
# You must first use your OS's bridging capability
# to bridge the TAP interface with the ethernet
# NIC interface.  Then you must manually set the
# IP/netmask on the bridge interface, here we
# assume 10.8.0.4/255.255.255.0.  Finally we
# must set aside an IP range in this subnet
# (start=10.8.0.50 end=10.8.0.100) to allocate
# to connecting clients.  Leave this line commented
# out unless you are ethernet bridging.
;server-bridge 10.8.0.4 255.255.255.0 10.8.0.50 10.8.0.100

# Configure server mode for ethernet bridging
# using a DHCP-proxy, where clients talk
# to the OpenVPN server-side DHCP server
# to receive their IP address allocation
# and DNS server addresses.  You must first use
# your OS's bridging capability to bridge the TAP
# interface with the ethernet NIC interface.
# Note: this mode only works on clients (such as
# Windows), where the client-side TAP adapter is
# bound to a DHCP client.
;server-bridge

# Push routes to the client to allow it
# to reach other private subnets behind
# the server.  Remember that these
# private subnets will also need
# to know to route the OpenVPN client
# address pool (10.8.0.0/255.255.255.0)
# back to the OpenVPN server.
;push "route 192.168.10.0 255.255.255.0"
;push "route 192.168.20.0 255.255.255.0"

# To assign specific IP addresses to specific
# clients or if a connecting client has a private
# subnet behind it that should also have VPN access,
# use the subdirectory "ccd" for client-specific
# configuration files (see man page for more info).

# EXAMPLE: Suppose the client
# having the certificate common name "Thelonious"
# also has a small subnet behind his connecting
# machine, such as 192.168.40.128/255.255.255.248.
# First, uncomment out these lines:
;client-config-dir ccd
;route 192.168.40.128 255.255.255.248
# Then create a file ccd/Thelonious with this line:
#   iroute 192.168.40.128 255.255.255.248
# This will allow Thelonious' private subnet to
# access the VPN.  This example will only work
# if you are routing, not bridging, i.e. you are
# using "dev tun" and "server" directives.

# EXAMPLE: Suppose you want to give
# Thelonious a fixed VPN IP address of 10.9.0.1.
# First uncomment out these lines:
;client-config-dir ccd
;route 10.9.0.0 255.255.255.252
# Then add this line to ccd/Thelonious:
#   ifconfig-push 10.9.0.1 10.9.0.2

# Suppose that you want to enable different
# firewall access policies for different groups
# of clients.  There are two methods:
# (1) Run multiple OpenVPN daemons, one for each
#     group, and firewall the TUN/TAP interface
#     for each group/daemon appropriately.
# (2) (Advanced) Create a script to dynamically
#     modify the firewall in response to access
#     from different clients.  See man
#     page for more info on learn-address script.
;learn-address ./script

# If enabled, this directive will configure
# all clients to redirect their default
# network gateway through the VPN, causing
# all IP traffic such as web browsing and
# and DNS lookups to go through the VPN
# (The OpenVPN server machine may need to NAT
# or bridge the TUN/TAP interface to the internet
# in order for this to work properly).
;push "redirect-gateway def1 bypass-dhcp"

# Certain Windows-specific network settings
# can be pushed to clients, such as DNS
# or WINS server addresses.  CAVEAT:
# http://openvpn.net/faq.html#dhcpcaveats
# The addresses below refer to the public
# DNS servers provided by opendns.com.
;push "dhcp-option DNS 208.67.222.222"
;push "dhcp-option DNS 208.67.220.220"

# Uncomment this directive to allow different
# clients to be able to "see" each other.
# By default, clients will only see the server.
# To force clients to only see the server, you
# will also need to appropriately firewall the
# server's TUN/TAP interface.
;client-to-client

# Uncomment this directive if multiple clients
# might connect with the same certificate/key
# files or common names.  This is recommended
# only for testing purposes.  For production use,
# each client should have its own certificate/key
# pair.
#
# IF YOU HAVE NOT GENERATED INDIVIDUAL
# CERTIFICATE/KEY PAIRS FOR EACH CLIENT,
# EACH HAVING ITS OWN UNIQUE "COMMON NAME",
# UNCOMMENT THIS LINE OUT.
;duplicate-cn

# The keepalive directive causes ping-like
# messages to be sent back and forth over
# the link so that each side knows when
# the other side has gone down.
# Ping every 10 seconds, assume that remote
# peer is down if no ping received during
# a 120 second time period.
keepalive 10 120

# For extra security beyond that provided
# by SSL/TLS, create an "HMAC firewall"
# to help block DoS attacks and UDP port flooding.
#
# Generate with:
#   openvpn --genkey --secret ta.key
#
# The server and each client must have
# a copy of this key.
# The second parameter should be '0'
# on the server and '1' on the clients.
tls-auth ta.key 0 # This file is secret

# Select a cryptographic cipher.
# This config item must be copied to
# the client config file as well.
# Note that v2.4 client/server will automatically
# negotiate AES-256-GCM in TLS mode.
# See also the ncp-cipher option in the manpage
cipher AES-256-CBC

# Enable compression on the VPN link and push the
# option to the client (v2.4+ only, for earlier
# versions see below)
;compress lz4-v2
;push "compress lz4-v2"

# For compression compatible with older clients use comp-lzo
# If you enable it here, you must also
# enable it in the client config file.
;comp-lzo

# The maximum number of concurrently connected
# clients we want to allow.
;max-clients 100

# It's a good idea to reduce the OpenVPN
# daemon's privileges after initialization.
#
# You can uncomment this out on
# non-Windows systems.
;user nobody
;group nobody

# The persist options will try to avoid
# accessing certain resources on restart
# that may no longer be accessible because
# of the privilege downgrade.
persist-key
persist-tun

# Output a short status file showing
# current connections, truncated
# and rewritten every minute.
status openvpn-status.log

# By default, log messages will go to the syslog (or
# on Windows, if running as a service, they will go to
# the "\Program Files\OpenVPN\log" directory).
# Use log or log-append to override this default.
# "log" will truncate the log file on OpenVPN startup,
# while "log-append" will append to it.  Use one
# or the other (but not both).
;log         openvpn.log
;log-append  openvpn.log

# Set the appropriate level of log
# file verbosity.
#
# 0 is silent, except for fatal errors
# 4 is reasonable for general usage
# 5 and 6 can help to debug connection problems
# 9 is extremely verbose
verb 3

# Silence repeating messages.  At most 20
# sequential messages of the same message
# category will be output to the log.
;mute 20

# Notify the client that when the server restarts so it
# can automatically reconnect.
explicit-exit-notify 1

Important: Make sure that you mention the files that you have created previously:

ca ca.crt

cert openvpn-server.crt

key openvpn-server.key

dh dh2048.pem

3 . Edit the file /etc/sysctl.conf and uncomment the following line:

#net.ipv4.ip_forward=1

4 . Reload sysctl:

sysctl -p /etc/sysctl.conf

5 . Create a ta.key:

openvpn --genkey --secret ta.key

Important: Download a copy of this key also on your client to establish the VPN connection.

6 . Start the OpenVPN service. It is required to specify the configration file as an instance variable. The configuration file you created previously is called /etc/openvpn/openvpn-server.conf, this means you have to add @openvpn-server to end of your unit when launching it:

systemctl start openvpn@openvpn-server

7 . Check that the service is running:

 systemctl status openvpn@openvpn-server

If everything went well, a message similar to the following appears:

● openvpn@openvpn-server.service - OpenVPN connection to openvpn-server
   Loaded: loaded (/lib/systemd/system/openvpn@.service; disabled; vendor preset: enabled)
   Active: active (running) since Mon 2019-01-21 14:43:35 UTC; 9s ago
     Docs: man:openvpn(8)
           https://community.openvpn.net/openvpn/wiki/Openvpn24ManPage
           https://community.openvpn.net/openvpn/wiki/HOWTO
 Main PID: 30023 (openvpn)
   Status: "Initialization Sequence Completed"
    Tasks: 1 (limit: 2295)
   CGroup: /system.slice/system-openvpn.slice/openvpn@openvpn-server.service
           └─30023 /usr/sbin/openvpn --daemon ovpn-openvpn-server --status /run/openvpn/openvpn-server.status 10 --cd /etc

Jan 21 14:43:35 openvpn ovpn-openvpn-server[30023]: /sbin/ip addr add dev tun0 local 10.8.0.1 peer 10.8.0.2
Jan 21 14:43:35 openvpn ovpn-openvpn-server[30023]: /sbin/ip route add 10.8.0.0/24 via 10.8.0.2
Jan 21 14:43:35 openvpn ovpn-openvpn-server[30023]: Could not determine IPv4/IPv6 protocol. Using AF_INET
Jan 21 14:43:35 openvpn ovpn-openvpn-server[30023]: Socket Buffers: R=[212992->212992] S=[212992->212992]
Jan 21 14:43:35 openvpn ovpn-openvpn-server[30023]: UDPv4 link local (bound): [AF_INET][undef]:1194
Jan 21 14:43:35 openvpn ovpn-openvpn-server[30023]: UDPv4 link remote: [AF_UNSPEC]
Jan 21 14:43:35 openvpn ovpn-openvpn-server[30023]: MULTI: multi_init called, r=256 v=256
Jan 21 14:43:35 openvpn ovpn-openvpn-server[30023]: IFCONFIG POOL: base=10.8.0.4 size=62, ipv6=0
Jan 21 14:43:35 openvpn ovpn-openvpn-server[30023]: IFCONFIG POOL LIST
Jan 21 14:43:35 openvpn ovpn-openvpn-server[30023]: Initialization Sequence Completed

8 . If everything went well, enable the service to start it automatically during the boot of your instance:

systemctl enable openvpn@openvpn-server

Your OpenVPN server is now ready to accept connections.

Configuring the Client

1 . To connect to your server you have to install the OpenVPN client. On Ubuntu Linux, you can install it easily with apt:

apt install openvpn

If your computer is running a Windows Operating system, you can download the OpenVPN client for Windows. On MacOS you can use a tool like Tunnelblick to establish the VPN connection.

Make sure that you have copied the following files on your local computer before continuing:

/etc/openvpn/ca.crt
/etc/openvpn/ta.key
/etc/openvpn/easy-rsa/keys/openvpn-client01.crt
/etc/openvpn/easy-rsa/keys/openvpn-client01.key

You can download them for example via SFTP with Filezilla.

2 . Create a configuration file openvpn-client01.ovpn for the client and put the following information in it:

##############################################
# Sample client-side OpenVPN 2.0 config file #
# for connecting to multi-client server.     #
#                                            #
# This configuration can be used by multiple #
# clients, however each client should have   #
# its own cert and key files.                #
#                                            #
# On Windows, you might want to rename this  #
# file so it has a .ovpn extension           #
##############################################

# Specify that we are a client and that we
# will be pulling certain config file directives
# from the server.
client

# Use the same setting as you are using on
# the server.
# On most systems, the VPN will not function
# unless you partially or fully disable
# the firewall for the TUN/TAP interface.
;dev tap
dev tun

# Windows needs the TAP-Win32 adapter name
# from the Network Connections panel
# if you have more than one.  On XP SP2,
# you may need to disable the firewall
# for the TAP adapter.
;dev-node MyTap

# Are we connecting to a TCP or
# UDP server?  Use the same setting as
# on the server.
;proto tcp
proto udp

# The hostname/IP and port of the server.
# You can have multiple remote entries
# to load balance between the servers.
remote YOUR_SERVER_IP 1194
;remote my-server-2 1194

# Choose a random host from the remote
# list for load-balancing.  Otherwise
# try hosts in the order specified.
;remote-random

# Keep trying indefinitely to resolve the
# host name of the OpenVPN server.  Very useful
# on machines which are not permanently connected
# to the internet such as laptops.
resolv-retry infinite

# Most clients don't need to bind to
# a specific local port number.
nobind

# Downgrade privileges after initialization (non-Windows only)
;user nobody
;group nobody

# Try to preserve some state across restarts.
persist-key
persist-tun

# If you are connecting through an
# HTTP proxy to reach the actual OpenVPN
# server, put the proxy server/IP and
# port number here.  See the man page
# if your proxy server requires
# authentication.
;http-proxy-retry # retry on connection failures
;http-proxy [proxy server] [proxy port #]

# Wireless networks often produce a lot
# of duplicate packets.  Set this flag
# to silence duplicate packet warnings.
;mute-replay-warnings

# SSL/TLS parms.
# See the server config file for more
# description.  It's best to use
# a separate .crt/.key file pair
# for each client.  A single ca
# file can be used for all clients.
ca ca.crt
cert openvpn-client01.crt
key openvpn-client01.key

# Verify server certificate by checking that the
# certicate has the correct key usage set.
# This is an important precaution to protect against
# a potential attack discussed here:
#  http://openvpn.net/howto.html#mitm
#
# To use this feature, you will need to generate
# your server certificates with the keyUsage set to
#   digitalSignature, keyEncipherment
# and the extendedKeyUsage to
#   serverAuth
# EasyRSA can do this for you.
remote-cert-tls server

# If a tls-auth key is used on the server
# then every client must also have the key.
tls-auth ta.key 1

# Select a cryptographic cipher.
# If the cipher option is used on the server
# then you must also specify it here.
# Note that v2.4 client/server will automatically
# negotiate AES-256-GCM in TLS mode.
# See also the ncp-cipher option in the manpage
cipher AES-256-CBC

# Enable compression on the VPN link.
# Don't enable this unless it is also
# enabled in the server config file.
#comp-lzo

# Set log file verbosity.
verb 3

# Silence repeating messages
;mute 20

Add your OpenVPN servers IP at remote YOUR_SERVER_IP 1194 and make sure all file names correspond to your files.

3 . Restart the openvpn client. On Ubuntu:

systemctl restart openvpn@openvpn-client01

On Windows or MacOS, double click on the openvpn-client01.ovpn file to import the configuration settings into your client.

You can now connect to the Internet via your VPN.

For more configuration options, refer to the official documentation of OpenVPN. To save time, you may deploy your OpenVPN also with a simple click and the Scaleway OpenVPN InstantApp.

Installing PowerDNS

2018-12-03T00:00:00+00:00

PowerDNS Overview

PowerDNS is an open source DNS server written in C++. A DNS server is a server that contains a database of public IP addresses and their associated domain names. Its purpose is to resolve, or translate, those common names to IP addresses as requested. PowerDNS runs on most Linux and all other Unix derivatives. The software comes with the PowerDNS-Admin web interface, simplifying the management of your DNS zones.

Requirements

You have an account and are logged into console.scaleway.com

You have configured your SSH Key

You have at least two virtual cloud servers running on Ubuntu Bionic Beaver

Installing PowerDNS

The goal of this guide is to have PowerDNS configured with a MariaDB/MySQL backend with SQL replication to sync information between slave servers.

Installing and Configuring MariaDB (All Servers)

PowerDNS needs a database to store DNS zones. MariaDB is being used to store the information, alternatively a zone file configuration is possible.

1 . Start by installing MariaDB on your install as described here.

2 . Once the database server is installed, connect to it and create a database called powerdns:

mysql -u root -p
CREATE DATABASE powerdns;

3 . Create a user for PowerDNS and assign privileges to the powerdns database:

GRANT ALL ON powerdns.* TO 'powerdns'@'localhost' \
IDENTIFIED BY '<SECRET_PASSWORD>';

4 . Update the settings by flushing the privileges:

FLUSH PRIVILEGES;

5 . Switch to the powerdns database to create the required tables:

USE powerdns;

6 . Create the required tables:

CREATE TABLE domains (
  id                    INT AUTO_INCREMENT,
  name                  VARCHAR(255) NOT NULL,
  master                VARCHAR(128) DEFAULT NULL,
  last_check            INT DEFAULT NULL,
  type                  VARCHAR(6) NOT NULL,
  notified_serial       INT UNSIGNED DEFAULT NULL,
  account               VARCHAR(40) CHARACTER SET 'utf8' DEFAULT NULL,
  PRIMARY KEY (id)
) Engine=InnoDB CHARACTER SET 'latin1';

CREATE UNIQUE INDEX name_index ON domains(name);


CREATE TABLE records (
  id                    BIGINT AUTO_INCREMENT,
  domain_id             INT DEFAULT NULL,
  name                  VARCHAR(255) DEFAULT NULL,
  type                  VARCHAR(10) DEFAULT NULL,
  content               VARCHAR(64000) DEFAULT NULL,
  ttl                   INT DEFAULT NULL,
  prio                  INT DEFAULT NULL,
  change_date           INT DEFAULT NULL,
  disabled              TINYINT(1) DEFAULT 0,
  ordername             VARCHAR(255) BINARY DEFAULT NULL,
  auth                  TINYINT(1) DEFAULT 1,
  PRIMARY KEY (id)
) Engine=InnoDB CHARACTER SET 'latin1';

CREATE INDEX nametype_index ON records(name,type);
CREATE INDEX domain_id ON records(domain_id);
CREATE INDEX ordername ON records (ordername);


CREATE TABLE supermasters (
  ip                    VARCHAR(64) NOT NULL,
  nameserver            VARCHAR(255) NOT NULL,
  account               VARCHAR(40) CHARACTER SET 'utf8' NOT NULL,
  PRIMARY KEY (ip, nameserver)
) Engine=InnoDB CHARACTER SET 'latin1';


CREATE TABLE comments (
  id                    INT AUTO_INCREMENT,
  domain_id             INT NOT NULL,
  name                  VARCHAR(255) NOT NULL,
  type                  VARCHAR(10) NOT NULL,
  modified_at           INT NOT NULL,
  account               VARCHAR(40) CHARACTER SET 'utf8' DEFAULT NULL,
  comment               TEXT CHARACTER SET 'utf8' NOT NULL,
  PRIMARY KEY (id)
) Engine=InnoDB CHARACTER SET 'latin1';

CREATE INDEX comments_name_type_idx ON comments (name, type);
CREATE INDEX comments_order_idx ON comments (domain_id, modified_at);


CREATE TABLE domainmetadata (
  id                    INT AUTO_INCREMENT,
  domain_id             INT NOT NULL,
  kind                  VARCHAR(32),
  content               TEXT,
  PRIMARY KEY (id)
) Engine=InnoDB CHARACTER SET 'latin1';

CREATE INDEX domainmetadata_idx ON domainmetadata (domain_id, kind);


CREATE TABLE cryptokeys (
  id                    INT AUTO_INCREMENT,
  domain_id             INT NOT NULL,
  flags                 INT NOT NULL,
  active                BOOL,
  content               TEXT,
  PRIMARY KEY(id)
) Engine=InnoDB CHARACTER SET 'latin1';

CREATE INDEX domainidindex ON cryptokeys(domain_id);


CREATE TABLE tsigkeys (
  id                    INT AUTO_INCREMENT,
  name                  VARCHAR(255),
  algorithm             VARCHAR(50),
  secret                VARCHAR(255),
  PRIMARY KEY (id)
) Engine=InnoDB CHARACTER SET 'latin1';

CREATE UNIQUE INDEX namealgoindex ON tsigkeys(name, algorithm);

Useful tip: You can verify that the tables have been created, by running show tables;.

7 . Disconnect from the MariaDB server:

QUIT;

Disabling systemd-resolve

Ubuntu Bionic Beaver comes with systemd-resolve preinstalled, which needs to be disabled as it binds to port 53 which will conflict with the ports required for PowerDNS.

1 . Disable the resolvd service by running the following commands:

systemctl disable systemd-resolved
systemctl stop systemd-resolved

2 . Remove the symlinked resolv.conf file:

ls -lh /etc/resolv.conf
lrwxrwxrwx 1 root root 24 Dec  3 12:23 /etc/resolv.conf -> /lib/systemd/resolv.conf

3 . Create a new resolv.conf file containing your preferred DNS resolver (for example 9.9.9.9):

echo "nameserver 9.9.9.9" > /etc/resolv.conf

Installing PowerDNS Server

1 . Add the official PowerDNS repository to apt:

echo "deb [arch=amd64] http://repo.powerdns.com/ubuntu bionic-auth-41 main" > /etc/apt/sources.list.d/pdns.list

2 . Import the PowerDNS GPG key:

curl https://repo.powerdns.com/FD380FBB-pub.asc | apt-key add -

3 . Update apt and install the PowerDNS package (pdns-server) and MySQL backend (pdns-backend-mysql):

apt update
apt install pdns-server pdns-backend-mysql -y

4 . Edit the file /etc/powerdns/pdns.d/pdns.local.gmysql.conf with your SQL credentials, created previously so it looks like the following example:

# SQL Configuration (MariaDB)
# Launch gmysql backend
launch+=gmysql
# gmysql parameters
gmysql-host=localhost
gmysql-port=3306
gmysql-dbname=powerdns
gmysql-user=powerdns
gmysql-password=<SECRET_PASSWORD>
gmysql-dnssec=yes

5 . Make a backup of the current configuration file:

mv /etc/powerdns/pdns.conf /etc/powerdns/pdns.conf-orig

6 . Create a new /etc/powerdns/pdns.conf file as following:

# PowerDNS configuration file
# Replace ns1.example.com with your master nameserver's hostname
default-soa-name=ns1.mydomain.com
include-dir=/etc/powerdns/pdns.d
launch=
security-poll-suffix=
setgid=pdns
setuid=pdns

7a . On the master server only include the following lines:

api=yes
# Replace <RANDOM_KEY> with a randomly generated key for API access
api-key=<RANDOM_KEY>
master=yes
slave=no

7b . On the slave server only include the following lines:

master=no
slave=yes

8 . Restart the pdns service:

systemctl restart pdns

9 . Verify that the PowerDNS server responds correctly:

# dig @127.0.0.1

; <<>> DiG 9.11.3-1ubuntu1.3-Ubuntu <<>> @127.0.0.1
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: REFUSED, id: 36221
;; flags: qr rd; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1
;; WARNING: recursion requested but not available

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1680
;; QUESTION SECTION:
;.				IN	NS

;; Query time: 1 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
;; WHEN: Mon Dec 03 12:53:57 UTC 2018
;; MSG SIZE  rcvd: 28

Configuring MySQL Replication

The MySQL replication setup is a very important part allows near instant DNS updates across all linked nameservers. This replication method, has some benefits over the other replication mode of PowerDNS, called supermaster setup, which can be unstable sometimes and does not remove deleted zones across every server automatically.

Configuring the Master Server

The following steps have to be done on the master server only.

1 . Edit the MySQL configuration file (/etc/mysql/mariadb.conf.d/50-server.cnf) as follows:

bind-address = 0.0.0.0

server-id = 1
log_bin = /var/log/mysql/mysql-bin.log
expire_logs_days = 10
max_binlog_size = 100M
binlog_do_db = pdns

2 . Restart MariaDB:

systemctl restart mysql

3 . Connect to MariaDB and create a user for the slave server:

mysql -u root -p
GRANT REPLICATION SLAVE ON *.* TO 'pdns-slave'@'<SLAVE_SERVER_IP>' IDENTIFIED BY '<SECRET_PASSWORD>';
FLUSH PRIVILEGES;

4 . Run show master status; from the MariaDB console:

+------------------+----------+--------------+------------------+
| File             | Position | Binlog_Do_DB | Binlog_Ignore_DB |
+------------------+----------+--------------+------------------+
| mysql-bin.000001 |      634 | powerdns     |                  |
+------------------+----------+--------------+------------------+
1 row in set (0.01 sec)

Note the File (mysql-bin.000001) and Position (634) numbers, as they will be required in a later step.

Configuring the Slave Server

The following steps have to be done on the slave server only.

1 . Edit the MySQL configuration file (/etc/mysql/mariadb.conf.d/50-server.cnf) as follows:

server-id=2
relay-log=slave-relay-bin
relay-log-index=slave-relay-bin.index
replicate-do-db=powerdns

Important: The server-id has variable needs to be unique for each server, if you want to configure more than one slave server increase it accordingly, for example: server-id=2, server-id=3, etc.

2 . Restart MariaDB:

systemctl restart mysql

3 . Log into the MariaDB console:

mysql -u root -p

4 . Run the following command to link the slave to the master server:

change master to
master_host='<MASTER_SERVER_IP>',
master_user='pdns-slave',
master_password='<SECRET_PASSWORD>', master_log_file='mysql-bin.000001', master_log_pos=634;
start slave;

Important: The values mysql-bin.000001 and 634 correspond to the values you noted down in a previous step.

5 . Check the slave status to see if everything worked:

show slave status;

Installing PowerDNS-Admin

PowerDNS-Admin is an advanced web interface that simplifies the management of DNS zones. PowerDNS-Admin will be installed on the master server.

1 . Connect to MariaDB and create a new database and user for PowerDNS-Admin:

mysql -u root -p
CREATE DATABASE pdnsadmin CHARACTER SET utf8 COLLATE utf8_general_ci;
GRANT ALL PRIVILEGES ON pdnsadmin.* TO 'pdnsadminuser'@'%' IDENTIFIED BY '<SECRET_PASSWORD>';
FLUSH PRIVILEGES;
quit;

2 . Install Python 3 and the packages required for building python libraries:

apt install python3-dev libmysqlclient-dev python-mysqldb libsasl2-dev libffi-dev libldap2-dev libssl-dev libxml2-dev libxslt1-dev libxmlsec1-dev pkg-config virtualenv -y

3 . Download and install the yarn GPG key, add the repository to apt and install yarn to build asset files:

curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | apt-key add -
echo "deb https://dl.yarnpkg.com/debian/ stable main" > /etc/apt/sources.list.d/yarn.list
apt update && apt install yarn -y

4 . Download and extract PowerDNS-Admin, then create a virtualenv:

wget https://github.com/ngoduykhanh/PowerDNS-Admin/archive/master.tar.gz
mkdir /opt/web
tar -C /opt/web -xvf master.tar.gz
mv /opt/web/PowerDNS-Admin-master /opt/web/powerdns-admin
cd /opt/web/powerdns-admin
virtualenv -p python3 flask

5 . Activate the Python 3 environment and install the required libraries:

. ./flask/bin/activate
pip install -r requirements.txt
pip install python-dotenv

6 . Create a configuration file from the template shipped with the tool:

cp config_template.py config.py

7 . Edit the config.py file and enter the database credentials. Once edited the file should look like the following example:

SQLA_DB_USER = 'pdnsadminuser'
SQLA_DB_PASSWORD = '<SECRET_PASSWORD>'
SQLA_DB_HOST = '127.0.0.1'
SQLA_DB_NAME = 'pdnsadmin'
SQLALCHEMY_TRACK_MODIFICATIONS = True

8 . Once the file is edited, create the database scheme by running the following commands:

export FLASK_APP=app/__init__.py
flask db upgrade

9 . Run db migrate:

flask db migrate -m "Init DB"

10 . Generate the asset files with yarn and build the application:

yarn install --pure-lockfile
flask assets build

11 . Test if your application is working:

./run.py

An output like the following will appear. Press CTRL+C to exit the application:

* Serving Flask app "app" (lazy loading)
* Environment: production
  WARNING: Do not use the development server in a production environment.
  Use a production WSGI server instead.
* Debug mode: on
[INFO]  * Running on http://127.0.0.1:9191/ (Press CTRL+C to quit)
[INFO]  * Restarting with stat
[WARNING]  * Debugger is active!
[INFO]  * Debugger PIN: 104-611-896

12 . To manage PowerDNS-Admin with systemd, create a new service file called /etc/systemd/system/powerdns-admin.service:

[Unit]
Description=PowerDNS-Admin
After=network.target

[Service]
User=root
Group=root
WorkingDirectory=/opt/web/powerdns-admin
ExecStart=/opt/web/powerdns-admin/flask/bin/gunicorn --workers 2 --bind unix:/opt/web/powerdns-admin/powerdns-admin.sock app:app

[Install]
WantedBy=multi-user.target

13 . Reload systemd, start the powerdns-admin service and enable it to be started automatically during system boot:

systemctl daemon-reload
systemctl start powerdns-admin
systemctl enable powerdns-admin

14 . Verify that the service is running with the following command: systemctl status powerdns-admin

The output should look like as follows:

● powerdns-admin.service - PowerDNS-Admin
   Loaded: loaded (/etc/systemd/system/powerdns-admin.service; enabled; vendor preset: enabled)
   Active: active (running) since Mon 2018-12-03 14:05:12 UTC; 6min ago
 Main PID: 8986 (gunicorn)
    Tasks: 3 (limit: 2295)
   CGroup: /system.slice/powerdns-admin.service
           ├─8986 /opt/web/powerdns-admin/flask/bin/python3 /opt/web/powerdns-admin/flask/bin/gunicorn --workers 2 --bind unix:/opt/web/po
           ├─9002 /opt/web/powerdns-admin/flask/bin/python3 /opt/web/powerdns-admin/flask/bin/gunicorn --workers 2 --bind unix:/opt/web/po
           └─9003 /opt/web/powerdns-admin/flask/bin/python3 /opt/web/powerdns-admin/flask/bin/gunicorn --workers 2 --bind unix:/opt/web/po

Installing Nginx Proxy

A Nginx proxy is being used to access the PowerDNS-Admin interface from a web browser.

1 . Install Nginx:

apt install nginx -y

2 . Create a configuration file called /etc/nginx/conf.d/powerdns-admin.conf:

server {
  listen *:80;
  server_name               <YOUR_DOMAIN_NAME>;

  index                     index.html index.htm index.php;
  root                      /opt/web/powerdns-admin;
  access_log                /var/log/nginx/powerdns-admin.local.access.log combined;
  error_log                 /var/log/nginx/powerdns-admin.local.error.log;

  client_max_body_size              10m;
  client_body_buffer_size           128k;
  proxy_redirect                    off;
  proxy_connect_timeout             90;
  proxy_send_timeout                90;
  proxy_read_timeout                90;
  proxy_buffers                     32 4k;
  proxy_buffer_size                 8k;
  proxy_set_header                  Host $host;
  proxy_set_header                  X-Real-IP $remote_addr;
  proxy_set_header                  X-Forwarded-For $proxy_add_x_forwarded_for;
  proxy_headers_hash_bucket_size    64;

  location ~ ^/static/  {
    include  /etc/nginx/mime.types;
    root /opt/web/powerdns-admin/app;

    location ~*  \.(jpg|jpeg|png|gif)$ {
      expires 365d;
    }

    location ~* ^.+.(css|js)$ {
      expires 7d;
    }
  }

  location / {
    proxy_pass            http://unix:/opt/web/powerdns-admin/powerdns-admin.sock;
    proxy_read_timeout    120;
    proxy_connect_timeout 120;
    proxy_redirect        off;
  }

}

3 . Verify the syntax of the configuration file and reload nginx if everything is fine:

nginx -t
systemctl reload nginx

4 . PowerDNS-Admin is accessible at http://<YOUR_DOMAIN_NAME> now:

Installing a Let's Encrypt Certificate

Let’s Encrypt certbot will be used to automatically obtain and manage the SSL certificate for PowerDNS-Admin. It takes also care about the required modifications in Nginx configuration files:

1 . Add the required repositories and install the tool via apt:

add-apt-repository ppa:certbot/certbot
apt update
apt install software-properties-common  python-certbot-nginx -y

2 . Request a certificate for the server:

certbot --nginx

When asked enter your email address and agree to the terms of Let’s Encrypt.

3 . The connection to PowerDNS-Admin is now secured by SSL.

Creating a First User and Linking the API

1 . Open a web browser and enter the address of your server: https://<YOUR_DOMAIN_NAME>/register. Fill in all required information to create your first admin user.

2 . Once logged into the interface you will be asked to complete the information about the API. Fill in the following information:

PDNS API URL Your PowerDNS API URL (in this case http://127.0.0.1:8081/ as PowerDNS-Admin is running on the master server).
PDNS API KEY Your PowerDNS API key (configured during the setup of the master server).
PDNS VERSION Your PowerDNS version number (the latest version at the time of edition of this tutorial is: 4.1.5).

3 . Once the infomation is provided, the dashboard will be available:

After declaring the new DNS servers with your registrar, you will be able to manage your domains from the PowerDNS Admin panel. For more information, you may refer to the official documentation.

How to deploy AwStats

2018-12-03T00:00:00+00:00

AwStats Overview

AwStats is a very useful tool that can give you an overview of what is happening on your website and assist with site analysis. It is an open-source Web analytics reporting tool that generates advanced web, streaming, FTP or mail server statistics graphically.

It is a complete enterprise grade server and log monitoring software that works as a CGI (Common Gateway Interface) or from command line and which shows you all possible information your log contains, in few graphical web pages, often and quickly.

AwStats uses log file analysis to analyze log files from most web server including Apache, IIS and many other web server.

Requirements:

You have an account and are logged into cloud.scaleway.com

You have configured your SSH Key

You have sudo privileges or access to the root user.

Installing Apache

Before installing AwStats server packages, make sure Apache2 HTTP server is installed.

apt update
apt install apache2

Installing and Configuring AwStats Package

By default, AwStats package is available in the Ubuntu repository.

1 . Install AwStats package

apt-get install awstats

2 . Enable the CGI module in Apache

a2enmod cgi

3 . Restart Apache

systemctl restart apache2

4 . Duplicate the AwStats default configuration file to one with your domain name

cp /etc/AwStats/awstats.conf /etc/awstats/domain-example.conf

5 . Edit the your AwStats domain name configuration file

nano /etc/awstats/domain-example.conf

6 . Update the settings shown below

# Change to Apache log file, by default it's /var/log/apache2/access.log
LogFile="/var/log/apache2/access.log"

# Change to the website domain name
SiteDomain="domain-example.com"
HostAliases="www.domain-example localhost 127.0.0.1"

# When this parameter is set to 1, AwStats adds a button on report page to allow to "update" statistics from a web browser
AllowToUpdateStatsFromBrowser=1

7 . Save and close the file

8 . Build your initial statistics which is generated from the current logs on your server

/usr/lib/cgi-bin/awstats.pl -config=test.com -update

which returns

...
Create/Update database for config "/etc/awstats/awstats.conf" by AwStats version 7.6 (build 20161204)
From data in log file "/var/log/apache2/access.log"...
Phase 1 : First bypass old records, searching new record...
Searching new records from beginning of log file...
Phase 2 : Now process new records (Flush history on disk after 20000 hosts)...
Jumped lines in file: 0
Parsed lines in file: 5
 Found 0 dropped records,
 Found 0 comments,
 Found 0 blank records,
 Found 0 corrupted records,
 Found 0 old records,
 Found 5 new qualified records.

Configuring Apache For AwStats

Copy the content of the cgi-bin folder to the default document root directory of your Apache installation. By default, this is in the /usr/lib/cgi-bin folder.

cp -r /usr/lib/cgi-bin /var/www/html/
chown www-data:www-data /var/www/html/cgi-bin/
chmod -R 755 /var/www/html/cgi-bin/

Testing AwStats

Now you can access your AwStats by visiting the url http://your-server-ip/cgi-bin/awstats.pl?config=test.com

which displays

Setting-up Cron Job to Update Logs

It is recommended to schedule a cron job to regularly update the AwStats database using newly created log entries, so the stats get updated on a regular basis.

1 . Edit the /etc/crontab file

nano /etc/crontab

2 . Add the following line to tell AwStats to update every ten minutes

*/10 * * * * root /usr/lib/cgi-bin/awstats.pl -config=test.com -update

3 . Save and exit.

Installing Jitsi Meet on Debian Stretch

2018-11-30T00:00:00+00:00

Jitsi Meet Overview

Jitsi Meet is a free and fully encrypted open source video conferencing service solution providing high quality and audio without subscription or the need to create an account.

The tool provides features like:

Sharing of desktops, presentations, and more
Inviting users to a conference via a simple, custom URL
Editing documents together using Etherpad
Trading messages and emojis while video conferencing, with integrated chat.

Requirements

You have an account and are logged into console.scaleway.com

You have configured your SSH Key

You have a Scaleway Compute Instance running Debian Stretch (9.0)

For best performances of Jitsi Meet, an instance with at least 4GB RAM is recommended

You have a domain or subdomain pointed to your Compute Instance

Installing Jitsi Meet

1 . Configure a the hostname of the server corresponding to your domain / subdomain name:

hostnamectl set-hostname jitsi
sed -i 's/^127.0.1.1.*$/127.0.1.1 jitsi.mydomain.tld jitsi/g' /etc/hosts

2 . Start by updating the software already installed on the system:

apt update && apt upgrade -y

3 . Install Java 8 on the server:

apt install -y openjdk-8-jre-headless

4 . Setup the JAVA_HOME environment variable:

echo "JAVA_HOME=$(readlink -f /usr/bin/java | sed "s:bin/java::")" | tee -a /etc/profile
source /etc/profile

5 . Install a Nginx web server before installing Jitsi Meet. The Jitsi installation tool will take care about the configuration of Nginx, if it is present on the system. Run the following commands to install and enable Nginx:

apt install -y nginx
systemctl start nginx.service
systemctl enable nginx.service

Important: If Nginx or Apache is not present on the system, Jitsi Meet will automatically install Jetty during the installation.

6 . Download the APT key and setup the repositories of Jitsi, to install the software via apt:

wget -qO - https://download.jitsi.org/jitsi-key.gpg.key | apt-key add -
sh -c "echo 'deb https://download.jitsi.org stable/' > /etc/apt/sources.list.d/jitsi-stable.list"
apt update -y

7 . Launch the installation of Jitsi Meet:

apt install -y jitsi-meet

When asked, enter:

The FQDN of your instance. For example jitsi.mydomain.tld and press Enter.
When asked about the SSL certificate choose the Generate a new self-signed certificate (You will later get a chance to obtain a Let's Encrypt certificate) option and press Enter.

8 . Run the following script to generate a Let’s Encrypt SSL certificate for your instance:

/usr/share/jitsi-meet/scripts/install-letsencrypt-cert.sh

When asked, enter:

Your e-mail address to receive notifications regarding your certificate and press enter to request the certificate.

9 . The Jitsi Meet instance is configured now and ready for a first conference call. Open a Web-browser and type the FQDN of your instance, for example: https://jitsi.mydomain.tld. The following screen will appear:

Enter a name for your conference and press Go to enter the conference room. It is now possible to share the link, to set a password, configure the audio and video quality and more for the conference.

10 . Happy conferencing :)

Setting up a Nginx reverse proxy with Object Storage

2018-11-26T00:00:00+00:00

Object Storage Proxy Overview

The Scaleway Object Storge service lets you store infinite data in buckets. You can either access the data directly via your bucket. This consumes data transfer and will be deducted from your monthly package or be billed in case you exceed your included data transfer.

It is possible to access your bucket via a compute instance as a proxy. Transfer of data between your bucket and the compute platform is free of charge and you can use the bandwidth included with your instance to distribute the files in your bucket.

We use a pre-build and containerized Nginx server running on Docker to access the bucket.

Requirements

You have an account and are logged into console.scaleway.com

You have configured your SSH Key

You have a Scaleway Compute instance

You have a Scaleway Object Storage bucket

Installing Docker

We use a pre-build Docker container with Nginx to run the application. Therefore it is required that Docker Community Edition is installed on the instance.

Configuring The Proxy

1 . The image requires a config file in the container at: /nginx.conf. Use the -v option to mount one from your host.

docker run -p 8000:8000 -v /path/to/nginx.conf:/nginx.conf coopernurse/nginx-s3-proxy

The Nginx configration file should look like the following example:

worker_processes 2;
pid /run/nginx.pid;
daemon off;

events {
	worker_connections 768;
}

http {
	sendfile on;
	tcp_nopush on;
	tcp_nodelay on;
	keepalive_timeout 65;
	types_hash_max_size 2048;
	server_names_hash_bucket_size 64;

	include /usr/local/nginx/conf/mime.types;
	default_type application/octet-stream;

	access_log /usr/local/nginx/logs/access.log;
	error_log  /usr/local/nginx/logs/error.log;

	gzip on;
	gzip_disable "msie6";
	gzip_http_version 1.1;
	gzip_types text/plain text/css application/json application/x-javascript text/xml application/xml application/xml+rss text/javascript;

    proxy_cache_lock on;
    proxy_cache_lock_timeout 60s;
    proxy_cache_path /data/cache levels=1:2 keys_zone=s3cache:10m max_size=30g;

    server {
        listen     8000;

        location / {
            proxy_pass https://scw-bucket.s3.nl-ams.scw.cloud;

            aws_access_key scw-access-key;
            aws_secret_key scw-secret-key;
            s3_bucket scw-bucket;

            proxy_set_header Authorization $s3_auth_token;
            proxy_set_header x-amz-date $aws_date;

            proxy_cache        s3cache;
            proxy_cache_valid  200 302  24h;
        }
    }
}

Replace the following parameters in the file:

scw-bucket: The name of your Scaleway Object Storage bucket
scw-access-key: Your Access key
scw-secret-key: Your Secret key

2 . You can now access your bucket via a web browser by typing : http://your_server_ip:8000

3 . (Optional) If you want to cache data locally, bind a path to /data/cache:

docker run -p 8000:8000 -v /path/to/nginx.conf:/nginx.conf -v /my/cache/path:/data/cache coopernurse/nginx-s3-proxy