Jump toUpdate content

Monitoring Systems with Netdata

Reviewed on 10 May 2021 • Published on 07 February 2020

Netdata is a system monitoring system, designed to be distributed, lightweight, flexible and open source. The source code is available for download from GitHub.

The application can detect hundreds of metrics automatically, simplifying their configuration and integration. The application’s core is written in C and hundreds of collectors are available to collect thousands of different metrics while being very lightweight and not requiring additional computing power. Netdata can either run autonomously, without any third-party components, or it can be integrated to existing monitoring toolchains like Prometheus, Graphite, OpenTSDB, Kafka, Grafana, and more.

Requirements:

Installing Netdata

  1. Connect to your instance via SSH.

  2. Update the repsitory lists of the apt packet manager and use the upgrade command to upgrade the software already installed on the instance:

    apt update && apt upgrade -y
  3. Run the automatic installer of Netdata:

    bash <(curl -Ss https://my-netdata.io/kickstart.sh)

    The automatic installer will run some tests to the detect the operating system version and install additional dependencies using the apt packet manager. During the automatic installation, you will be asked several time to confirm the installation of additional packages using apt. When prompted, press Enter or Y to validate.

    /etc/os-release information:NAME            : UbuntuVERSION         : 18.04.4 LTS (Bionic Beaver)ID              : ubuntuID_LIKE         : debianVERSION_ID      : 18.04
    We detected these:Distribution    : ubuntuVersion         : 18.04Codename        : 18.04.4 LTS (Bionic Beaver)Package Manager : install_apt_getPackages Tree   : debianDetection Method: /etc/os-releaseDefault Python v: 2
    [...]Once the pre-requirements are fulfilled, the main installation prompt displays. It provides information where the Netdata daemon, configuration, plugin, log files and more will be installed. Press `Enter` to continue the automatic installation:^|.-.   .-.   .-.   .-.   .  netdata|   '-'   '-'   '-'   '-'   real-time performance monitoring, done right!+----+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+--->You are about to build and install netdata to your system.It will be installed at these locations:
      - the daemon     at /usr/sbin/netdata  - config files   in /etc/netdata  - web files      in /usr/share/netdata  - plugins        in /usr/libexec/netdata  - cache files    in /var/cache/netdata  - db files       in /var/lib/netdata  - log files      in /var/log/netdata  - pid file       at /var/run/netdata.pid  - logrotate file at /etc/logrotate.d/netdata
    This installer allows you to change the installation path.Press Control-C and run the same command with --help for help.
    NOTE:Anonymous usage stats will be collected and sent to Google Analytics.To opt-out, pass --disable-telemetry option to the installer or exportthe enviornment variable DO_NOT_TRACK to a non-zero or non-empty value(e.g: export DO_NOT_TRACK=1).
    Press ENTER to build and install netdata to your system >
    Once the installation completes, a confirmation message displays:
    --- We are done! ---
    ^|.-.   .-.   .-.   .-.   .-.   .  netdata                          .-.   .-|   '-'   '-'   '-'   '-'   '-'   is installed and running now!  -'   '-'+----+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+--->
    enjoy real-time performance and health monitoring...

Optionally, Netdata sends by default anonymous usage statistics to Google Analytics to identify how the product is used and also for quality assusrance statistics.

It is possible to opt-out from these statistics using an empty file called .opt-out-from-anonymous-statistics which stored in the Netdata configuration directory. It stops the statistics script from running.

  1. Create the file using the touch command:

    touch /etc/netdata/.opt-out-from-anonymous-statistics

Logging Into Netdata

  1. Open a web browser on your local computer, for example Chrome or Firefox.

  2. Point it to http://<netdata_instance_ip>:19999/ (replace <netdata_instance_ip> with the public IP address of your instance.)

  3. The Netdata dashboard displays:

By default Netdata monitors a wide list of services on the machine. Browse around to see the details of all of them.

Adding Multiple Nodes Into Netdata

Netdata is a distributed application, and it is possible to add multiple nodes to a single account.

  1. From the Netdata dashboard, click Netdata in the menu bar, then on Nodes.

  2. The Netdata Cloud splash screen displays. Enter your email address to generate a magic link. Alternatively, use a Google or GitHub account to authenticate.

  3. Click the magic link, sent to the email address entered in the previous step.

  4. The Netdata cloud dashboard displays. Repeat this step on all machines running Netdata to add them to the dashboard:

Configuring Additional Metrics

Netdata comes by default with a broad range of internal plugins to gather system statistics. To collect non-system metrics, Netdata supports many data collection plugins for various softwares, like web servers, load balancers, mail servers and more. While internal plugins are written in the C programming language and run as threads inside the netdata daemon, external plugins may be written in any programming language and are spawn as independent long-running processes by the netdata daemon and communicate with the netdata daemon via pipes (stdout communication).

To facilitate the plugin development and to limit the number of threads and processes, Netdata supports plugin orchestrators. Currently there are orchestrators available for four different programmation languages: C, python, node.js and bash.

In the following example, a plugin to monitor a NGINX web server using Netdata is being configured.

The plugin relies on data provided by the Nginx module stub_status.

  1. Add the following configuration in a Nginx server block:

    location /stub_status {  stub_status;  # for security reasons: only allow access from the localhost.  allow 127.0.0.1;  # deny anyone else  deny all;}
  2. The Netdata nginx plugin requires the external plugin python.d.plugin, an orchestrator for modules written in python. Open the Netdata configuration file in a text editor, for example nano:

    nano /etc/netdata/netdata.conf
  3. Scroll down to the [plugins] section of the file, and uncomment the line python.d = yes. The fill should look as follows:

    [...][plugins]        # PATH environment variable = /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/sbin:/usr/sbin:/usr/local/bin:$        # PYTHONPATH environment variable =        # proc = yes        # diskspace = yes        # cgroups = yes        # tc = yes        # idlejitter = yes        # enable running new plugins = yes        # check for new plugins every = 60        # slabinfo = no        # fping = yes        # perf = yes        # charts.d = yes        # node.d = yes        # ioping = yes        # apps = yes        python.d = yes        # go.d = yes[...]
  4. Save it and exit the text editor, when using nano, press CTRL and O to write the file, then CTRL and X to exit nano.

  5. Create and open a new configuration file for the Nginx plugin:

    nano /etc/netdata/python.d/nginx.chart.py

    Copy the following configuration in the file, save it and exit the text editor:

    # update_every sets the default data collection frequency.# If unset, the python.d.plugin default is used.# update_every: 1update_every : 10
    # priority controls the order of charts at the netdata dashboard.# Lower numbers move the charts towards the top of the page.# If unset, the default for python.d.plugin is used.# priority: 60000priority     : 90100
    # the JOB name is a unique identifier to the service. localipv4 is the name that will appear in the Netdata logs, the friendly name "Local Nginx IPv4" displays in the control panellocalipv4:  name : 'Local_Nginx_IPv4'  url  : 'http://127.0.0.1/stub_status'
  6. Restart the Netdata service:

    service netdata restart
  7. Point your web browser to the Netata interface and click Nginx web configuration in the menu on the right. The Nginx statistics displays:

For more information regarding all available plugins, refer to the official Netdata documentation.

Note:

For security reasons, it is recommended to setup a Nginx reverse proxy and to block access to the application from the Internet using a firewall.