Mastering Network Monitoring: How to Build a Modern NMS with Suricata, Elasticsearch, and Real-time Alerts

What Is NMS?

Network Management System (NMS) is a set of tools used to monitor, manage, and secure computer networks. By using NMS, network administrators can detect threats, identify performance issues, and take corrective actions proactively.

Benefits of NMS?

Improved Network Security:

• Faster Threat Detection: NMS can proactively monitor network traffic and detect anomalies or suspicious activities that indicate a cyber attack.

• In-depth Log Analysis: NMS can identify complex attack patterns and exploit unknown vulnerabilities by collecting and analyzing logs from various network devices.

• Faster Response: Instant threat notifications allow the security team to respond quickly to minimize the impact of an attack.

Improved Network Performance:

• Real-time Performance Monitoring: NMS provides comprehensive visibility into network performance, allowing quick identification of bottlenecks and other performance issues.

• Resource Optimization: With accurate performance data, administrators can optimize the use of bandwidth, CPU, and other resources.

• Capacity Planning: NMS assists in network capacity planning to meet the growing needs of the business.

Reduced Downtime:

• Proactive Detection: NMS can detect potential issues before they become serious disruptions, thus reducing downtime.

• Quick Problem Diagnosis: With comprehensive data, administrators can quickly identify the root cause of issues and make repairs.

Examples of NMS Use in Practice?

• DDoS Attack Detection: NMS can detect unusual traffic spikes indicating a DDoS attack.

• Application Performance Monitoring: NMS can monitor application response times and identify performance issues affecting users.

• Traffic Trend Analysis: NMS can analyze network traffic trends to identify usage patterns and anticipate future bandwidth needs.

Installation and Configuration of NMS with Suricata, ELK Stack, and Filebeat

Prerequisites

Before starting the installation, ensure that we have a stable operating system that meets the following requirements:

• Operating System: Prepare a server for NMS, a client, and Kali Linux for attacking the NMS server.

• Root Access: We need root access or a user with sudo privileges to run installation commands.

• Network: Ensure our system is connected to the network and has internet access to download necessary packages.

Installation and Initial Configuration of Suricata

1. Setup for installing a stable version of Suricata.

    apt-update
    apt-get install software-properties-common
    add-apt-repository ppa:oisf/suricata-stable
    apt update
    "If so, you can install Suricata"
    apt-get install suricata
   

2. Initial Configuration of Suricata

configuration of Suricata in /etc/suricata/suricata.yml

vim /etc/suricata/suricata.yml

suricata-update
systemctl restart suricata
systemctl status suricata

3. Testing Suricata

First, we check if there are logs from Suricata, then we attempt to DDoS the NMS server to view the logs.

tail -f /var/log/suricata/fast.log

head -n20 /var/log/suricata/eve.json

Let's test how the logs appear when attacked by a DDoS by accessing our Kali Linux used for the DoS on the NMS server.

hping3 -S -d 100 –flood -p 80 172.23.1.51

then we check our NMS server with tail -f /var/log/suricata/fast.log, and also see if our server's CPU usage increases; if it does, then the DoS is successful and the message displayed by Suricata is also successful.

we can see that our CPU experiences a high spike due to the DoS, which means the Suricata logs are correct.

Installation and Initial Configuration of Elasticsearch

1. Install Elasticsearch

    wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo gpg --dearmor -o /usr/share/keyrings/elasticsearch-keyring.gpg
    apt-get install apt-transport-https
    echo "deb [signed-by=/usr/share/keyrings/elasticsearch-keyring.gpg] https://artifacts.elastic.co/packages/8.x/apt stable main" | tee /etc/apt/sources.list.d/elastic-8.x.list
    apt-get update && sudo apt-get install elasticsearch
   

   

   During the installation process, there will be information like this, copy the elastic password and save it in Word/Notepad for later use when connecting to Kibana.

2. Elasticsearch Configuration

After installing, before starting, we first configure it in elasticsearch.yml

vim /etc/elasticsearch/elasticsearch.yml

• • cluster.name\= whatever

    • node.name\=use hostname

    • network.host\= can use server IP or 0.0.0.0

    • for http.port=use the default port

ufw allow 9200/tcp
systemctl daemon-reload
systemctl enable elasticsearch.service --now
systemctl status elasticsearch

Now, for Elasticsearch, we're done. Next, let's install Kibana.

3. Testing Elasticsearch

If the installation and configuration process is complete, let's check if Elasticsearch is running. Open a browser and go to https://ip_server:9200

Installation and Configuration of Kibana

1. Installation of Kibana

for Kibana installation, it's different from before. Since we've already added the repo and key, we can just install it directly.

apt install kibana

2. kibana configuration

A fresh installation of Kibana cannot be accessed in the browser yet. We need to configure it with Elasticsearch first to access the Kibana dashboard.

vim /etc/kibana/kibana.yml

• server.port= use the default Kibana port

• server.host= can use 0.0.0.0/server IP

ufw allow 5061/tcp
systemctl restart kibana
systemctl status kibana

Now, there is information in the Go to http://172.23.1.51. Open the browser and click the link in the Kibana status.

3. Connect Kibana with Elasticsearch

Before we can access the Kibana dashboard, we need to connect our Kibana with Elasticsearch. There are several steps, such as creating enrollment and verifying the code.

/usr/share/elasticsearch/bin/elasticsearch-create-enrollment-token --scope kibana

pastekan token yang kita buat di elasticsearch ke enrollment token kibana, jika sudah klik Configure Elastic

open the server and type
/usr/share/kibana/bin/kibana-verification-code

Copy and paste the verification code into the Kibana verification field, then click Verify.

• username= elasic

• password= password that appears when we first install Elasticsearch.

if you forget the password, you can create a new password with command: /usr/share/elasticsearch/bin/elasticsearch-reset-password -u elastic

This is the Kibana dashboard view, but there are no logs yet because we haven't configured our Filebeat with Suricata.

Installation and Configuration of Filebeat

1. Install Filebeat

We need Filebeat to pull metrics from Suricata to Elasticsearch because Suricata cannot send metrics directly. So, Filebeat acts as an agent. Below are the steps to install Filebeat and configure it to pull Suricata metrics.

curl -L -O https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-8.14.3-amd64.deb
dpkg -i filebeat-8.14.3-amd64.deb

2. Filebeat Configuration

Now, to allow Filebeat to pull metrics from Suricata to Elasticsearch, we need to configure it first in Filebeat. The configuration is in filebeat.yml.

vim /etc/filebeat/filebeat.yml

There are 2 things we need to configure in filebeat.yml, which are for Elasticsearch and Kibana:

• host elastic = point to the Elastic website

• protocol = match our protocol, either http or https

• credential = enter the credentials from Elastic that we use to log in

• ssl = point ssl to http_ca.crt located in /etc/elasticsearch/cert

And for Kibana, we only need to configure the host.

3. Connect Filebeat to Suricata

filebeat modules enable suricata
vim /etc/filebeat/modules.d/suricata.yml

var.path= point to the JSON and log files in Suricata

filebeat setup
systemctl daemon-reload
systemctl enable --now filebeat
systemctl status filebeat

Now, we can see that the status is running. Let's try opening it in the Kibana dashboard.

4. Testing Network Monitoring System

Now let's see if the Suricata logs pulled by Filebeat to Elastic have appeared in Kibana. Open the browser at http://ip_server:5061

Klik Analytics>Dashboard>[Filebeat Suricata] Alert Overview

We can see that the logs pulled by Filebeat have been successfully sent to Elasticsearch. We can also clearly see the logs from Suricata here, and they are easy to understand.

Configuration of Netflow and Connect Mikrotik with NMS Server

This time, let's try monitoring our Mikrotik traffic. Let's see if Kibana can display the current traffic and if we can see any traffic activity. We will use Netflow Filebeat to do this, and of course, we need to connect our Mikrotik with the NMS server.

1. Netflow Configuration

Before we connect Mikrotik with the NMS server, we need to set up our Netflow in Filebeat located at /etc/filebeat/modules.d/netflow.yml.

cd /etc/filebeat/modules.d
filebeat modules enable netflow

2. Connect Mikrotik ke Server NMS

Open Mikrotik and set up the traffic flow to our NMS server.

IP>Traffic Flow

click enabled and direct it to all interfaces, then click Apply > Targets > Add New

• Enabled= check

• Dst. Address= direct to the NMS server

• port= set the port to 2055, which we configured earlier in netflow

• Version= use version IPFIX

Once done, click Apply > Ok

Once connected, let's check if our netflow can pull traffic from the Mikrotik. Open the Kibana dashboard and click Analytics>Dashboard>[Filebeat Netflow] Traffic Analysis.

Well, our Filebeat Netflow has successfully pulled traffic from Mikrotik to the Kibana Dashboard.

3. Testing Netflow Traffic

Once connected, let's check if our Netflow can see heavy traffic and identify which specific IP is causing it. We'll try to DoS the Mikrotik using Kali Linux to test it.

#Kali Linux

hping3 -S -d 10000 --flood -p 80 172.23.1.244

let's check our Kali Linux IP, then run our DoS script on the Mikrotik and see if the Kibana Dashboard can identify the IP causing the heavy traffic.

#Server NMS

Open our Kibana dashboard, click Analytics > Dashboard > [Filebeat Netflow] Top-N Flows, and then scroll down.

And yes, our Filebeat Netflow can show us which IP is causing the heavy traffic, and we can also see how many packets are being sent by our Kali Linux.

Alerting Suricata to Telegram Bot

After we create a monitoring dashboard and successfully monitor our server and network, what's next? Well, next we set up alerting so that when an anomaly appears, it will be immediately detected and send a warning to our account.

1. Create Bot_Telegram

open Bot Father in Telegram to create a new bot

Bot Father>New Bot>Name Bot>Username Bot

open our bot by searching bot_name>/start

Salin HTTP Api dan simpan karena akan digunakan untuk konfigurasi di server kita, jika sudah kita lihat ID telegram kita, bisa search userinfobot>/start

Now, in this part, we copy our ID and save it because it will be used for configuration on the server.

2. Install and Configure suricata-telegram-notifier

git clone https://github.com/infokek/suricata-telegram-notifier
mv suricata-telegram-notifier/configs/service.ini.example
vim suricata-telegram-notifier/configs/service.ini

• Bot_token= use the HTTP API we copied earlier when creating the bot in bot_father

• Chat_ID= use our Telegram ID that we copied earlier in the bot userinfobot

• Interfaces= use the server interface

We can see our interface with the command ip a

rm suricata.yml
cp /etc/suricata/suricata.yml 
cd ~/suricata-telegram-notifier/
./install

3. Testing Alert

We will test our alert to see if it sends a notification when there is an anomaly. We will use the BruteForce attack method on our NMS SSH server because I'm tired of only using DoS. Okay, let's open our Kali Linux to start the attack.

hydra -l root -P rockyou.txt 172.23.1.51 ssh

Well, our brute force attack has been started. Next, let's check our alert bot to see if it sends a notification.

and yes, the alert we created was successful

Conclusion

Network Management System (NMS) is an essential tool for monitoring, managing, and securing computer networks. NMS enhances security, performance, and reduces network downtime through proactive threat detection, log analysis, and real-time performance monitoring.

This article explains the installation and configuration of NMS using Suricata for threat detection, ELK Stack (Elasticsearch, Logstash, Kibana) for data collection and analysis, and Filebeat as a log shipping agent. Practical steps include software installation, system configuration, testing, and integration to effectively monitor and manage networks.

With a configured NMS, administrators can monitor traffic, detect attacks, and take quick action through automatic notifications, including integration with Telegram bots for alerting. This system enables early problem detection and faster response to security incidents.

Important to remember: Cybersecurity is an ever-evolving process. Therefore, it's crucial to regularly update tools, configurations, and knowledge about the latest threats.