Jump toUpdate content

Monitoring Systems with Netdata

Reviewed on 10 May 2021 • Published on 07 February 2020
  • monitoring
  • Netdata
  • server

Netdata is a system monitoring system, designed to be distributed, lightweight, flexible and open source. The source code is available for download from GitHub.

The application can detect hundreds of metrics automatically, simplifying their configuration and integration. The application’s core is written in C and hundreds of collectors are available to collect thousands of different metrics while being very lightweight and not requiring additional computing power. Netdata can either run autonomously, without any third-party components, or it can be integrated to existing monitoring toolchains like Prometheus, Graphite, OpenTSDB, Kafka, Grafana, and more.

Requirements:

Installing Netdata

  1. Connect to your instance via SSH.

  2. Update the repsitory lists of the apt package manager and use the upgrade command to upgrade the software already installed on the Instance:

    apt update && apt upgrade -y
  3. Run the automatic installer of Netdata:

    bash <(curl -Ss https://my-netdata.io/kickstart.sh)

    The automatic installer will run some tests to the detect the operating system version and install additional dependencies using the apt package manager. During the automatic installation, you will be asked to confirm the installation of additional packages using apt several times. When prompted, press Enter or Y to validate.

    /etc/os-release information:
    NAME : Ubuntu
    VERSION : 18.04.4 LTS (Bionic Beaver)
    ID : ubuntu
    ID_LIKE : debian
    VERSION_ID : 18.04

    We detected these:
    Distribution : ubuntu
    Version : 18.04
    Codename : 18.04.4 LTS (Bionic Beaver)
    Package Manager : install_apt_get
    Packages Tree : debian
    Detection Method: /etc/os-release
    Default Python v: 2

    [...]
    Once the pre-requirements are fulfilled, the main installation prompt displays. It provides information where the Netdata daemon, configuration, plugin, log files and more will be installed. Press `Enter` to continue the automatic installation:
    ^
    |.-. .-. .-. .-. . netdata
    | '-' '-' '-' '-' real-time performance monitoring, done right!
    +----+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+--->
    You are about to build and install netdata to your system.
    It will be installed at these locations:

    - the daemon at /usr/sbin/netdata
    - config files in /etc/netdata
    - web files in /usr/share/netdata
    - plugins in /usr/libexec/netdata
    - cache files in /var/cache/netdata
    - db files in /var/lib/netdata
    - log files in /var/log/netdata
    - pid file at /var/run/netdata.pid
    - logrotate file at /etc/logrotate.d/netdata

    This installer allows you to change the installation path.
    Press Control-C and run the same command with --help for help.

    NOTE:
    Anonymous usage stats will be collected and sent to Google Analytics.
    To opt-out, pass --disable-telemetry option to the installer or export
    the enviornment variable DO_NOT_TRACK to a non-zero or non-empty value
    (e.g: export DO_NOT_TRACK=1).

    Press ENTER to build and install netdata to your system >

    Once the installation completes, a confirmation message displays:

    --- We are done! ---

    ^
    |.-. .-. .-. .-. .-. . netdata .-. .-
    | '-' '-' '-' '-' '-' is installed and running now! -' '-'
    +----+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+--->

    enjoy real-time performance and health monitoring...

Optionally, Netdata sends by default anonymous usage statistics to Google Analytics to identify how the product is used and also for quality assusrance statistics.

It is possible to opt-out from these statistics using an empty file called .opt-out-from-anonymous-statistics which stored in the Netdata configuration directory. It stops the statistics script from running.

  1. Create the file using the touch command:

    touch /etc/netdata/.opt-out-from-anonymous-statistics

Logging Into Netdata

  1. Open a web browser on your local computer, for example Chrome or Firefox.

  2. Point it to http://<netdata_instance_ip>:19999/ (replace <netdata_instance_ip> with the public IP address of your instance.)

  3. The Netdata dashboard displays:

By default Netdata monitors a wide list of services on the machine. Browse around to see the details of all of them.

Adding Multiple Nodes Into Netdata

Netdata is a distributed application, and it is possible to add multiple nodes to a single account.

  1. From the Netdata dashboard, click Netdata in the menu bar, then on Nodes.

  2. The Netdata Cloud splash screen displays. Enter your email address to generate a magic link. Alternatively, use a Google or GitHub account to authenticate.

  3. Click the magic link, sent to the email address entered in the previous step.

  4. The Netdata cloud dashboard displays. Repeat this step on all machines running Netdata to add them to the dashboard:

Configuring Additional Metrics

Netdata comes by default with a broad range of internal plugins to gather system statistics. To collect non-system metrics, Netdata supports many data collection plugins for various softwares, like web servers, load balancers, mail servers and more. While internal plugins are written in the C programming language and run as threads inside the netdata daemon, external plugins may be written in any programming language and are spawn as independent long-running processes by the netdata daemon and communicate with the netdata daemon via pipes (stdout communication).

To facilitate the plugin development and to limit the number of threads and processes, Netdata supports plugin orchestrators. Currently there are orchestrators available for four different programmation languages: C, python, node.js and bash.

In the following example, a plugin to monitor a NGINX web server using Netdata is being configured.

The plugin relies on data provided by the Nginx module stub_status.

  1. Add the following configuration in a Nginx server block:

    location /stub_status {
    stub_status;
    # for security reasons: only allow access from the localhost.
    allow 127.0.0.1;
    # deny anyone else
    deny all;
    }
  2. The Netdata nginx plugin requires the external plugin python.d.plugin, an orchestrator for modules written in python. Open the Netdata configuration file in a text editor, for example nano:

    nano /etc/netdata/netdata.conf
  3. Scroll down to the [plugins] section of the file, and uncomment the line python.d = yes. The fill should look as follows:

    [...]
    [plugins]
    # PATH environment variable = /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/sbin:/usr/sbin:/usr/local/bin:$
    # PYTHONPATH environment variable =
    # proc = yes
    # diskspace = yes
    # cgroups = yes
    # tc = yes
    # idlejitter = yes
    # enable running new plugins = yes
    # check for new plugins every = 60
    # slabinfo = no
    # fping = yes
    # perf = yes
    # charts.d = yes
    # node.d = yes
    # ioping = yes
    # apps = yes
    python.d = yes
    # go.d = yes
    [...]
  4. Save it and exit the text editor, when using nano, press CTRL and O to write the file, then CTRL and X to exit nano.

  5. Create and open a new configuration file for the Nginx plugin:

    nano /etc/netdata/python.d/nginx.chart.py

    Copy the following configuration in the file, save it and exit the text editor:

    # update_every sets the default data collection frequency.
    # If unset, the python.d.plugin default is used.
    # update_every: 1
    update_every : 10

    # priority controls the order of charts at the netdata dashboard.
    # Lower numbers move the charts towards the top of the page.
    # If unset, the default for python.d.plugin is used.
    # priority: 60000
    priority : 90100

    # the JOB name is a unique identifier to the service. localipv4 is the name that will appear in the Netdata logs, the friendly name "Local Nginx IPv4" displays in the control panel
    localipv4:
    name : 'Local_Nginx_IPv4'
    url : 'http://127.0.0.1/stub_status'
  6. Restart the Netdata service:

    service netdata restart
  7. Point your web browser to the Netata interface and click Nginx web configuration in the menu on the right. The Nginx statistics displays:

For more information regarding all available plugins, refer to the official Netdata documentation.

Note:

For security reasons, it is recommended to setup a Nginx reverse proxy and to block access to the application from the Internet using a firewall.