System Monitoring with Netdata

Netdata - Overview

Netdata is a system monitoring system, designed to be distributed, lightweight, flexible and open source. The source code is available for download from GitHub.

The application is able to detect hundreds of metrics automatically, simplyfing their configuration and integration. The applications core is written in C and hundreds of collectors are available to collect thousands of different metrics while being very lightweight and not requiring additional computing power. Netdata can either run autonomously, without any third-party components, or it can be integrated to existing monitoring toolchains like Prometheus, Graphite, OpenTSDB, Kafka, Grafana, and more.

Requirements

Installing Netdata

1 . Log into the your virtual cloud instance using SSH.

2 . Update the repsitory lists of the apt packet manager and use the upgrade command to upgrade the software already installed on the instance:

apt update && apt upgrade -y

3 . Run the automatic installer of Netdata:

bash <(curl -Ss https://my-netdata.io/kickstart.sh)

The automatic installer will run some tests to the detect the operating system version and install additional dependencies using the apt packet manager. During the automatic installation, you will be asked several time to confirm the installation of additional packages using apt. When prompted, press Enter or Y to validate.

/etc/os-release information:
NAME            : Ubuntu
VERSION         : 18.04.4 LTS (Bionic Beaver)
ID              : ubuntu
ID_LIKE         : debian
VERSION_ID      : 18.04

We detected these:
Distribution    : ubuntu
Version         : 18.04
Codename        : 18.04.4 LTS (Bionic Beaver)
Package Manager : install_apt_get
Packages Tree   : debian
Detection Method: /etc/os-release
Default Python v: 2

[...]

Once the pre-requirements are fulfilled, the main installation prompt displays. It provides information where the Netdata daemon, configuration, plugin, log files and more will be installed. Press Enter to continue the automatic installation:

^
|.-.   .-.   .-.   .-.   .  netdata
|   '-'   '-'   '-'   '-'   real-time performance monitoring, done right!
+----+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+--->


You are about to build and install netdata to your system.

It will be installed at these locations:

 - the daemon     at /usr/sbin/netdata
 - config files   in /etc/netdata
 - web files      in /usr/share/netdata
 - plugins        in /usr/libexec/netdata
 - cache files    in /var/cache/netdata
 - db files       in /var/lib/netdata
 - log files      in /var/log/netdata
 - pid file       at /var/run/netdata.pid
 - logrotate file at /etc/logrotate.d/netdata

This installer allows you to change the installation path.
Press Control-C and run the same command with --help for help.


NOTE:
Anonymous usage stats will be collected and sent to Google Analytics.
To opt-out, pass --disable-telemetry option to the installer or export
the enviornment variable DO_NOT_TRACK to a non-zero or non-empty value
(e.g: export DO_NOT_TRACK=1).

Press ENTER to build and install netdata to your system >

Once the installation completes, a confirmation message displays:

--- We are done! ---

 ^
 |.-.   .-.   .-.   .-.   .-.   .  netdata                          .-.   .-
 |   '-'   '-'   '-'   '-'   '-'   is installed and running now!  -'   '-'
 +----+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+--->

 enjoy real-time performance and health monitoring...

4 . Optionally: Netdata sends by default anonymous usage statistics to Google Analytics to identify how the product is used and also for quality assusrance statistics.
It is possible to opt-out from these statistics using an empty file called .opt-out-from-anonymous-statistics which stored in the Netdata configuration directory. It stops the statistics script from running. Create the file using the touch command:

touch /etc/netdata/.opt-out-from-anonymous-statistics

Logging Into Netdata

1 . Open a web browser on your local computer, for example Chrome or Firefox.

2 . Point it to http://<netdata_instance_ip>:19999/ (replace <netdata_instance_ip> with the public IP address of your instance.)

3 . The Netdata dashboard displays:

By default Netdata monitors a wide list of services on the machine. Browse around to see the details of all of them.

Adding Multiple Nodes Into Netdata

Netdata is a distributed application, and it is possible to add multiple nodes to a single account.

1 . From the Netdata dashboard, click on Netdata in the menu bar, then on Nodes

2 . The Netdata Cloud splash screen displays. Enter your email address to generate a magic link. Alternatively, use a Google or GitHub account to authenticate.

3 . Click on the magic link, sent to the email address entered in the previous step.

4 . The Netdata cloud dashboard displays. Repeat this step on all machines running Netdata to add them to the dashboard:

Configuring Additional Metrics

Netdata comes by default with a broad range of internal plugins to gather system statistics. To collect non-system metrics, Netdata supports many data collection plugins for various softwares, like web servers, load balancers, mail servers and more. While internal plugins are written in the C programming language and run as threads inside the netdata daemon, external plugins may be written in any programming language and are spawn as independent long-running processes by the netdata daemon and communicate with the netdata daemon via pipes (stdout communication).

To facilitate the plugin development and to limit the number of threads and processes, Netdata supports plugin orchestrators. Currently there are orchestrators available for four different programmation languages: C, python, node.js and bash.

In the following example, a plugin to monitor a NGINX web server using Netdata is being configured.

1 . The plugin relies on data provided by the Nginx module stub_status. Make it available by adding the following configuration in a Nginx server block:

location /stub_status {
  stub_status;
  # for security reasons: only allow access from the localhost.
  allow 127.0.0.1;
  # deny anyone else
  deny all;
 }

2 . The Netdata nginx plugin requires the external plugin python.d.plugin, an orchestrator for modules written in python. Open the Netdata configuration file in a text editor, for example nano:

nano /etc/netdata/netdata.conf

Scroll down to the [plugins] section of the file, and uncomment the line python.d = yes. The fill should look as follows:

[...]
[plugins]
        # PATH environment variable = /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/sbin:/usr/sbin:/usr/local/bin:$
        # PYTHONPATH environment variable =
        # proc = yes
        # diskspace = yes
        # cgroups = yes
        # tc = yes
        # idlejitter = yes
        # enable running new plugins = yes
        # check for new plugins every = 60
        # slabinfo = no
        # fping = yes
        # perf = yes
        # charts.d = yes
        # node.d = yes
        # ioping = yes
        # apps = yes
        python.d = yes
        # go.d = yes
[...]

Save it and exit the text editor, when using nano, press CTRL and O to write the file, then CTRL and X to exit nano.

3 . Create and open a new configuration file for the Nginx plugin:

nano /etc/netdata/python.d/nginx.chart.py

Copy the following configuration in the file, save it and exit the text editor:

# update_every sets the default data collection frequency.
# If unset, the python.d.plugin default is used.
# update_every: 1
update_every : 10

# priority controls the order of charts at the netdata dashboard.
# Lower numbers move the charts towards the top of the page.
# If unset, the default for python.d.plugin is used.
# priority: 60000
priority     : 90100

# the JOB name is a unique identifier to the service. localipv4 is the name that will appear in the Netdata logs, the friendly name "Local Nginx IPv4" displays in the control panel
localipv4:
  name : 'Local_Nginx_IPv4'
  url  : 'http://127.0.0.1/stub_status'

4 . Restart the Netdata service:

service netdata restart

5 . Point your web browser to the Netata interface and click on Nginx web configuration in the menu on the right. The Nginx statistics displays:

For more information regarding all available plugins, refer to the official Netdata documentation.

Note: For security reasons, it is recommended to setup a Nginx reverse proxy and to block access to the application from the Internet using a firewall.

Discover the Cloud That Makes Sense