Installing and Configuring a CouchDB Cluster on Ubuntu Bionic

CouchDB Overview

Apache CouchDB is an open source database solution that was built with the ease of use and scalability in mind.

CouchDB is database that completely embraces the web. It is possible to store your data with JSON documents and to access your documents with your web browser.

It uses a document-orientated NoSQL database architecture and uses JSON to store data, JavaScript as query language by using MapReduce and HTTP for an API. It has been written in the Erlang programming language.

Unlike traditional relational databases (like for example MariaDB), CouchDB does not store data and relationship in tables. Each database is a collection of independent documents and each maintains its own data and self-contained schema.

Documents are the primary unit of data in CouchDB and consist of any number of fields and attachments. Each document field has an unique name and can contain values of varying types (text, number, boolean, lists, etc). There is no limitation of the size or element count inside a field.

CouchDB uses a form of multiversion concurrency control (MVCC) as document update model, so it does not lock the database file during writes or reads. It remains the task of the application to resolve conflicts in Documents. Resolving a conflict generally involves in merging data into one of the Documents at first, then deleting the obsolete one.

CouchDB’s also supports multi-master replication, which allows it to scale across multiple machines to build high performance systems.

This tutorial explains in 3 steps how to install and secure CouchDB on Ubuntu Bionic Beaver:

Setting up CouchDB
Configuring a Cluster
Securing the Cluster with ufw

Requirements:
You have an account and are logged into console.scaleway.com
You have three Ubuntu Bionic (18.04) servers
You have configured your SSH Key
You have sudo privileges or access to the root user.

Installing CouchDB on the Machines.

This tutorial describes the configuration of a three machine cluster. The following steps have to be run on all three machines.

1 . Make sure that your system is up to date and install the HTTPS-transport module for apt:

apt-get update && apt-get upgrade -y && apt-get install -y apt-transport-https

2 . Add the key and the repository of CouchDB:

curl -L https://couchdb.apache.org/repo/bintray-pubkey.asc | apt-key add -
echo "deb https://apache.bintray.com/couchdb-deb bionic main" | tee -a /etc/apt/sources.list

3 . Update the apt packet manager and install CouchDB:

apt-get update && apt-get install -y couchdb

4 . The installer will ask you if you want to install CouchDB as standalone application or in a clustered configuration. Select Clustered and press Enter. The cluster configuration can be done later from the web interface:

CouchDB Installer Clustered

5 . Enter the Erlang Node Name of your server. The cluster will communicate on the internal Scaleway network and you can find the internal FQDN in the details of the instance:

Scaleway Private DNS

6 . Fill this private DNS name in the form, by keeping the prefix couchdb@:

Erlang DNS name

7 . Set the Erlang Magic Cookie. This is a unique identifier to authenticate for your cluster.

Important: Make sure to choose a secret cookie. As anyone having this cookie can connect to your cluster.

Erlang Magic Cookie

8 . Configure the network interfaces on which CouchDB will be bound. To run a cluster it is important to bind it to 0.0.0.0:

CouchDB Installer Interfaces

9 . Enter the admin password for CouchDB, press enter, re-type the password and press Enter again to continue the installation.

10 . Once the installation has completed, verify that the CouchDB server is running:

curl http://127.0.0.1:5984/

If you see an output like the following the installation has successfully completed:

{"couchdb":"Welcome","version":"2.1.2","features":["scheduler"],"vendor":{"name":"The Apache Software Foundation"}}

Installing a Nginx Proxy to Access the Web-Interface on the Master Server

To protect the Webinterface of CouchDB from unauthorized access, it is required to configure a secure Nginx-Proxy to restrict the access. This steps have to be done on the master server.

1 . Install nginx and apache2-utils:

apt-get install -y nginx apache2-utils

2 . Generate a .htpasswd file to protect your installation from unauthorized visors. Run the following command to create a password for the user admin:

htpasswd -c /etc/nginx/.htpasswd admin

3 . Type the password for the user, then press Enter and re-type the password to confirm.

Remember to use the same password as you have set previously, otherwise the connection may fail.

4 . Edit the file /etc/nginx/sites-enabled/default and put the following content in it:

    server {
      listen 4000 default_server;
      server_name coachdb.local;

      location / {
        proxy_pass         http://localhost:5984;
        proxy_redirect     off;

        proxy_set_header   Host              $host;
        proxy_set_header   X-Forwarded-For   $proxy_add_x_forwarded_for;
        proxy_set_header   X-Forwarded-Proto $scheme;

        auth_basic "Access restricted";
        auth_basic_user_file /etc/nginx/.htpasswd;
      }

    }

5 . Reload the configuration of Nginx:

service nginx reload

6 . Access Project Fauxton, the configuration interface of CouchDB by typing http://YOUR_SERVER_IP:4000/_utils/#/setup. Login with the user admin and the password that you have set during the installation:

CouchDB Fauxton Interface

Creating a CouchDB Cluster

You can easily create a CouchDB cluster from the Fauxton Webinterface.

1 . Click on the Create Cluster button to be redirected to the following form:

CouchDB Fauxton Cluster Configuration

2 . In the top part of the form enter the credentials for the admin user, the interface on which you want the node to listen, the port and the total number of nodes in the cluster.

3 . In the second part enter the details of each node:

Remote Host will be the DNS name of the remote node
Bind Address must be 0.0.0.0
Port can be left at the default value of 5984

4 . Click on add node and repeat this step for each node of the cluster.

5 . Once all nodes are added, click on Configure Cluster to build the cluster.

Verifying the state of a Cluster

Once the cluster has been created, it is possible to verify the installation from the Fauxon interface.

Click on Verify in the menu on the left, then on Verify Installation.

If the cluster is working properly, all tests will marked as checked:

CouchDB Fauxton Verify Installation

Creating a new Database from the Fauxton Interface

1 . To create a new Database, click on the button in the top bar of the Fauxton Interface

2 . Enter the name for the new database

3 . Click Create:

CouchDB Fauxton Verify Installation

Managing a Database from the Fauxton Interface

It is possible to manage a database directly from the Fauxton Interface. It provides options to view, edit and add documents directly from this interface:

CouchDB Fauxton Manage Database

Editing a Document from the Fauxton Interface

To edit a Document from the Fauxton Interface, click on the corresponding line to edit the content of the document directly from the web interface:

CouchDB Fauxton Edit Document

Using the API of CouchDB

As seen, CouchDB provides a HTTP-API and it is possible to use it with curl. To authenticate with the API, it is required to provide your credentials in the form user:password@host.

Getting a list of all Databases on the server

To retrieve a list of all databases on the server type:

curl -X GET http://admin:password@127.0.0.1:5984/_all_dbs

It will return a list like the following:

[ "_replicator" , "_users" ]

Creating a Database

1 . To create a new database, use a command with the following syntax:

$ curl -X PUT http://admin:password@127.0.0.1:5984/database_name

2 . Replace database_name with the name of the new database.

For example to create the database my_database run the following command:

$ curl -X PUT http://admin:password@127.0.0.1:5984/my_database

It will return the following output, if everything went well:

{"ok":true}

Getting Database Information

To get information related to a database, run :

-X GET http://admin:pasword@127.0.0.1:5984/my_database

It will return the a JSON output similar to the following:

{
  "db_name":"my_database",
  "update_seq":"0-g1AAAALreJy10D8OgjAchuFGHbyFXqANhRboJIP30F__BUkVo0HjpDfRm-hN9CZY0tU4wfJ-2zN8DiE0LccaLVXdqFLLQnNFbZJZLI1hmJnY4jxPDLZcq5gJDTYRZH_YnIhydaPJUYEzZ7gQVW-d10aA5Kxt2wpQv66c-8rFDzqFSOQCGNZpRDHjYLAAnmElNM0kl1Sm8J8uOno1CL3u6OsQ9G7ii25-vH7v_e_APwL_HIh_Bf49EP8JfPd99QW04OuF",
  "sizes":
          {
            "file":34360,
            "external":0,
            "active":0
          },
  "purge_seq":0,
  "other":
          {
            "data_size":0
          },
  "doc_del_count":0,
  "doc_count":0,
  "disk_size":34360,
  "disk_format_version":6,
  "data_size":0,
  "compact_running":false,
  "cluster":
            {
              "q":8,
              "n":3,
              "w":2,
              "r":2
            },
  "instance_start_time":"0"
}

Getting a List of All Nodes in a Cluster

To get a list of all nodes in a cluster:

curl -X GET http://admin:password@127.0.0.1:5984/_membership

It will return a JSON list like the following:

{
  "all_nodes":
              ["couchdb@6a0989a4-d601-45ae-9a57-c9d17b5b1b6a.priv.cloud.scaleway.com",
              "couchdb@d5c1f37f-bee4-4e2f-883e-f5dc249daf39.priv.cloud.scaleway.com",
              "couchdb@fc978f3f-e70b-4281-b8e7-fedbfddea536.priv.cloud.scaleway.com"],
  "cluster_nodes":
              ["couchdb@6a0989a4-d601-45ae-9a57-c9d17b5b1b6a.priv.cloud.scaleway.com",
              "couchdb@d5c1f37f-bee4-4e2f-883e-f5dc249daf39.priv.cloud.scaleway.com",
              "couchdb@fc978f3f-e70b-4281-b8e7-fedbfddea536.priv.cloud.scaleway.com"]
}

Learning more about the API

To learn more about managing your CouchDB via the API, check out the official documentation.

Securing the Cluster

A limitation of CouchDB is that a clustered infrastructure must be bound to all interfaces of your node. This means anyone knowing your secret cookie can join your cluster, as there is no access restriction by default.

To secure it, it is recommended to install a firewall on each node to restrict the communication with external clients.

1 . Install ufw on the node:

apt-get install ufw

2 . Set the rules :

ufw allow OpenSSH #to allow SSH connections
ufw allow from 10.0.0.0/8 to any port 5984 #to allow the connection from the 10.0.0.0/8 subnet. You can also specify the individual IPs of your instances.
ufw allow 4000 #allow the connection to port 4000 for the web interface.

3 . Activate the firewall rules:

ufw enable

Connections from the Internet to your CouchDB cluster are limited now. As an additional security measure, you can disable the public IP address of the nodes as they communicate via the internal IP with each other.

You now have a basic CouchDB cluster up and running. For more information how to configure and use CouchDB for your applications, you may refer to the oficial documentation.