Configuring a High-Availability Storage with GlusterFS on Ubuntu

• glusterfs
• network
• filesystem
• high-availability-storage
• Ubuntu

GlusterFS is an open-source, scalable network filesystem suitable for high data-intensive workloads such as media streaming, cloud storage, and CDN (Content Delivery Network). Each storage server will be a mirror of the other storage server, and files will be replicated automatically across both storage servers.

This tutorial shows you how to:

Configuring the Host File

The first step we need to follow before installing glusterfs on all servers is configuring the hosts’ file and add GlusterFS repository to each server.

1. Log in to each server and edit the /etc/hosts file.

   nano /etc/hosts
   Copy code

2. Paste hosts configuration below:

   ip_address gluster01
   ip_address gluster02
   ip_address client01
   Copy code

3. Save and exit.

4. Ping each server using the hostname as below:

   ping -c 3 gluster01
   ping -c 3 gluster02
   ping -c 3 client01
   Copy code

Each hostname will resolve to each server’s IP address.

--- gluster01 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2033ms
rtt min/avg/max/mdev = 0.560/0.604/0.627/0.031 ms

--- gluster02 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2050ms
rtt min/avg/max/mdev = 0.497/0.593/0.688/0.080 ms

--- client01 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2036ms
rtt min/avg/max/mdev = 0.672/0.728/0.802/0.054 ms

Copy code

Adding the GlusterFS Repository

Install the software-properties-common package to all systems.

apt install software-properties-common -y
Copy code

Add the glusterfs key and repository to all systems.

sudo add-apt-repository ppa:gluster/glusterfs-9
Copy code

Installing a GlusterFS Server

1. Install the glusterfs server on both gluster01 and gluster02 servers.

   apt install glusterfs-server -y
   Copy code

2. Start the glusterd service and enable it to launch every time at system boot

   systemctl start glusterd
   systemctl enable glusterd
   Copy code

   Glusterfs server is now up and running on the gluster01 and gluster02 servers.

3. Check the services and the installed software version

   systemctl status glusterd
   glusterfsd --version
   Copy code

The command should return an active (running) status and glusterfs 9.5 version.

Configuring GlusterFS Servers

The next step is configuring the servers by creating a trusted storage pool and creating a distributed glusterfs volume.

• From the gluster01 server, we need to add the gluster02 server to the glusterfs storage pool.

  gluster peer probe gluster02
  Copy code

  The result peer probe: success is displayed. We added the gluster02 server to the storage trusted pool.

• Check the storage pool status and list.

  gluster peer status
  gluster pool list
  Copy code

  root@gluster01:~# gluster peer status
  Number of Peers: 1
  
  Hostname: gluster02
  Uuid: 17e7a76f-f616-42e5-b741-63a07fd091d6
  State: Peer in Cluster (Connected)
  root@gluster01:~# gluster pool list
  UUID                    Hostname    State
  17e7a76f-f616-42e5-b741-63a07fd091d6    gluster02   Connected
  ecc9cafa-b25d-477e-b6bc-403c051e752d    localhost   Connected
  Copy code

The gluster02 server is connected to the peer cluster, and it’s on the pool list.

After creating the trusted storage pool, we will create a new distributed glusterfs volume. We will create the new glusterfs volume based on the system directory.

Setting up the Distributed GlusterFS Volume

1. Create a new directory /glusterfs/distributed on each gluster01 and gluster02 server.

   mkdir -p /glusterfs/distributed
   Copy code

2. From the glusterO1 server, create a distributed glusterfs volume named ‘vol01’ with two replicas: ‘gfs01’ and ‘gfs02’

   gluster volume create vol01 transport tcp gluster01:/glusterfs/distributed gluster02:/glusterfs/distributed force
   Copy code

   volume create: vol01: success: please start the volume to access data
   Copy code

3. Start the ‘vol01’ and check the volume info

   gluster volume start vol01
   gluster volume info vol01
   Copy code

   root@gluster01:/# gluster volume start vol01
   volume start: vol01: success
   root@gluster01:/# gluster volume info vol01
   
   Volume Name: vol01
   Type: Replicate
   Volume ID: 814b103e-522c-48d2-8d1c-3301e10f3416
   Status: Started
   Snapshot Count: 0
   Number of Bricks: 1 x 2 = 2
   Transport-type: tcp
   Bricks:
   Brick1: gluster01:/glusterfs/distributed
   Brick2: gluster02:/glusterfs/distributed
   Options Reconfigured:
   transport.address-family: inet
   nfs.disable: on
   performance.client-io-threads: off
   Copy code

At this stage, we created the Replicate type vol01 volume and two bricks on the gluster01 and gluster02 servers. All data will be distributed automatically to each replica server.

Configuring GlusterFS Client

In this step, we will mount the glusterfs volume vol01 to the Ubuntu client, and we need to install the glusterfs-client to the client server.

1. Install the glusterfs-client on client01.

   apt install glusterfs-client -y
   Copy code

2. Create a new directory: /mnt/glusterfs.

   mkdir -p /mnt/glusterfs
   Copy code

3. Mount the distributed glusterfs volume (vol01) to the /mnt/glusterfs directory.

   mount -t glusterfs gluster01:/vol01 /mnt/glusterfs
   Copy code

4. Check the amount of volume available in the system.

   df -h /mnt/glusterfs
   Copy code

   Note:

   To mount glusterfs permanently to the Ubuntu client system, we can add the volume to the /etc/fstab.

5. Edit the /etc/fstab configuration file: vim /etc/fstab.

6. Paste the following configuring:gluster01:/vol01 /mnt/glusterfs glusterfs defaults,_netdev 0 0.

7. Save and exit.

8. Reboot the server. When online, the glusterfs volume ‘vol01’ is mounted automatically through the fstab.

Testing Replication & Mirroring

1. Mount the glusterfs volume vol01 to each glusterfs servers.

   • On gluster01: mount -t glusterfs gluster01:/vol01 /mnt
   • On gluster02: mount -t glusterfs gluster02:/vol01 /mnt

2. Back on client01, go to the ‘/mnt/glusterfs’ directory.

   cd /mnt/glusterfs
   Copy code

3. Create three files using the touch command.

   touch file01 file02 file03
   Copy code

4. Check on each gluster01 and gluster02 that the files that we’ve created from the client machine are displayed.

   cd /mnt/
   ls -lah
   Copy code

   The gluster01 machine returns:

   root@gluster01:/mnt# ls -lah
   total 8.0K
   drwxr-xr-x  3 root root 4.0K Oct  1 15:40 .
   drwxr-xr-x 24 root root 4.0K Sep 28 14:11 ..
   -rw-r--r--  1 root root    0 Oct  1 15:40 file01
   -rw-r--r--  1 root root    0 Oct  1 15:40 file02
   -rw-r--r--  1 root root    0 Oct  1 15:40 file03
   Copy code

   The gluster02 machine returns:

   root@gluster02:/mnt# ls -lah
   total 8.0K
   drwxr-xr-x  3 root root 4.0K Oct  1 15:40 .
   drwxr-xr-x 24 root root 4.0K Sep 28 14:11 ..
   -rw-r--r--  1 root root    0 Oct  1 15:40 file01
   -rw-r--r--  1 root root    0 Oct  1 15:40 file02
   -rw-r--r--  1 root root    0 Oct  1 15:40 file03
   Copy code

As you can see, all the files we created from the client machine are distributed to all the glusterfs volume node servers.