Setting up the Elastic Stack (formerly ELK Stack)

• Instances
• elastic-metal
• ELK-stack
• ELK-logging
• elastic-stack
• elasticsearch
• logstash
• kibana

The Elastic Stack, formerly known as the ELK Stack, is a powerful suite of open-source tools designed for real-time data search, analysis, and visualization. It offers comprehensive capabilities for collecting, processing, and visualizing large volumes of data. Its components are:

• Elasticsearch A distributed, RESTful search and analytics engine based on the Lucene library.
• Logstash A flexible data collection, processing, and enrichment pipeline.
• Kibana A visualization and exploration tool for analyzing and visualizing data stored in Elasticsearch.
• Beats Lightweight data shippers for ingesting data into Elasticsearch or Logstash.

Before you start

To complete the actions presented below, you must have:

• A Scaleway account logged into the console
• Owner status or IAM permissions allowing you to perform actions in the intended Organization
• An SSH key
• An Instance or an Elastic Metal server with at least 4 GB of RAM

Install Elasticsearch

1. Download and install the Elasticsearch signing key:

   curl -fsSL https://artifacts.elastic.co/GPG-KEY-elasticsearch | gpg --dearmor -o /usr/share/keyrings/elasticsearch-archive-keyring.gpg
   

2. Add the Elasticsearch repository.

   echo "deb [signed-by=/usr/share/keyrings/elasticsearch-archive-keyring.gpg] https://artifacts.elastic.co/packages/8.x/apt stable main" | tee /etc/apt/sources.list.d/elastic-8.x.list
   

3. Update the apt package repositories.

   apt update
   

4. Install Elasticsearch using apt.

   apt install elasticsearch
   

5. Start and enable the Elasticsearch service.

   systemctl start elasticsearch
   systemctl enable elasticsearch
   

Install and configure Logstash

1. Using the same repository added for Elasticsearch, you can simply install Logstash:

   apt install logstash
   

2. Once installed, you can create and modify configuration files for Logstash to set up your data pipelines. These are typically found in /etc/logstash/conf.d/.

3. Start and enable the Logstash service:

   systemctl start logstash
   systemctl enable logstash
   

Install and configure Kibana

1. Install Kibana using the repository:

   apt install kibana
   

2. Start and enable the Kibana service:

   systemctl start kibana
   systemctl enable kibana
   

3. By default, Kibana is accessible on http://localhost:5601. If you need to access it from a remote machine, edit the Kibana configuration file /etc/kibana/kibana.yml and set the server host:

   server.host: "0.0.0.0"
   

Secure the Elastic stack

It is important to secure your ELK Stack, especially if it is exposed to the public internet. You can complete your setup using the following additional resources:

• Use a firewal like ufw or iptables to restrict access to your Instance.
• Secure Elasticsearch using its built-in security features or with plugins.
• Consider setting up an HTTPS reverse proxy using a third party web server like Nginx or Apache to access Kibana securely.

Test the installation

Make sure everything is working:

• Elasticsearch Run the following command to test your Elasticsearch installation: curl -X GET "localhost:9200/"
• Kibana: Navigate to http://your_server_ip:5601 in your web browser.

Now, you should have a basic Elastic stack up and running! Adjust configurations as needed for your specific use case and further secure and optimize your setup for production use. Refer to the official Elastic documentation for the most accurate and up-to-date instructions and advanced configuration information.