Indonesia Website Awards
Indonesia Website Awards

server monitoring is now a mandatory that must be done for every individual / group / company that owns a server, whether on-premise or on-cloud. Especially if the servers that are owned are many. We need a tool or application that can help monitor it all. Therefore, Grafana is here to meet these needs. Quoted from Wikipedia – “Grafana is a multi-platform open source analytics and interactive visualization web application. It provides charts, graphs, and alerts for the web when connected to supported data sources”. So with Grafana we can prepare all the needs for server monitoring. Therefore, in this article I will share information about Server Monitoring with Grafana, Prometheus and node_exporter.

Server Monitoring with Grafana (Concept)

Before starting all kinds of things related to installation. I will explain the concept first regarding server monitoring with Grafana, Prometheus and node_exporter. Look at the following picture :

monitoring server with grafana

Server Monitoring with Grafana, Prometheus and node_exporter (Explanation of the Concept)

From the picture above, I want to give an example of implementing server monitoring with grafana, prometheus and node_exporter. Suppose we have several servers, like the example above, there are servers a, b, c, d, e and f. We want to monitor all of these servers simultaneously. We need to put in place centralized access to monitor it. in the picture above, the user monitors via server a. Where on server a, Grafana is installed (for interactive visualization), Prometheus (for processing metrics data), and node_exporter (for collecting metrics data). Then what about servers b, c, d, e and f? For other than server a, we only need to install node_exporter, provided that node_exporter can be accessed by prometheus on server a. So that the user simply monitors it by accessing grafana via a browser which will point to server a, where prometheus server a will visualize all the data that node_exporter has obtained from all servers (b, c, d, e and f). Then we just go straight to the installation steps.

Install Prometheus

The first thing we install is prometheus. Prometheus is an open-source application that we will use to monitor the system. Prometheus has tons of measurements we need to monitor servers. Then for the visualization later we will combine it with grafana. For how to install it, please read my previous post on How to Install Prometheus on Linux for Server Monitoring.

Install node_exporter

The second one we install is node_exporter. node_exporter is a daemon that functions to collect metrics / data on the server that we have, which prometheus will process the data. node_exporter will be installed to every node server that we have. as an illustration, node_exporter on each server will send their respective server metrics data and then send it to the Prometheus server for processing. For how to install it, please read my previous post on How to Install node_exporter on Linux for Data Server Monitoring.

Connecting Prometheus with node_exporter

Next, if you have installed promethues and node_exporter we will connect them. Prometheus can access metrics data from other servers that have node_exporter installed by adding it to the prometheus.yml configuration file. Simply add a job for this node_exporter

Plain text
Copy to clipboard
Open code in new window
EnlighterJS 3 Syntax Highlighter
- job_name: 'node_exporter'
scrape_interval: 5s
static_configs:
- targets: ['IP_SERVER:9100']
- job_name: 'node_exporter' scrape_interval: 5s static_configs: - targets: ['IP_SERVER:9100']
  - job_name: 'node_exporter'
    scrape_interval: 5s
    static_configs:
      - targets: ['IP_SERVER:9100']

into the prometheus.yml configuration file after the jobs section for prometheus.

Then, based on the example of the explanation concept that I have explained, more or less the prometheus.yml configuration file will be like this:

Plain text
Copy to clipboard
Open code in new window
EnlighterJS 3 Syntax Highlighter
global:
scrape_interval: 15s
scrape_configs:
- job_name: 'prometheus'
scrape_interval: 5s
static_configs:
- targets: ['IP_SERVER_A:9090']
- job_name: 'node_exporter'
scrape_interval: 5s
static_configs:
- targets: ['IP_SERVER_A:9100', 'IP_SERVER_B:9100', 'IP_SERVER_C:9100', 'IP_SERVER_D:9100', 'IP_SERVER_E:9100', 'IP_SERVER_F:9100']
global: scrape_interval: 15s scrape_configs: - job_name: 'prometheus' scrape_interval: 5s static_configs: - targets: ['IP_SERVER_A:9090'] - job_name: 'node_exporter' scrape_interval: 5s static_configs: - targets: ['IP_SERVER_A:9100', 'IP_SERVER_B:9100', 'IP_SERVER_C:9100', 'IP_SERVER_D:9100', 'IP_SERVER_E:9100', 'IP_SERVER_F:9100']
global:
  scrape_interval: 15s
scrape_configs:
  - job_name: 'prometheus'
    scrape_interval: 5s
    static_configs:
      - targets: ['IP_SERVER_A:9090']
  - job_name: 'node_exporter'
    scrape_interval: 5s
    static_configs:
      - targets: ['IP_SERVER_A:9100', 'IP_SERVER_B:9100', 'IP_SERVER_C:9100', 'IP_SERVER_D:9100', 'IP_SERVER_E:9100', 'IP_SERVER_F:9100']

If each server has node_exporter installed, we just need to add it to

- targets:
- targets: by giving a comma like the example above. After finishing with the configuration file, restart the prometheus service.

systemctl restart prometheus
systemctl restart prometheus.

When it’s finished, we try to access Prometheus

http://IP_ADDR_or_DOMAIN:9090
http://IP_ADDR_or_DOMAIN:9090

Then we execute one of the queries / expressions, namely “up” to just test whether prometheus can access the metrics of all servers installed with node_exporter. If it is connected, it will be like this:

connecting prometheus with node_exporter

 

Install Grafana

Install Prometheus already, install node_exporter already, connect prometheus with node_exporter as well. great job! Now we move on to the grafana installation.

Install grafana on CentOS 7

For CentOS 7, here’s how:

  • Create grafana repository

sudo nano /etc/yum.repos.d/grafana.repo
sudo nano /etc/yum.repos.d/grafana.repo

  • fill in this
Plain text
Copy to clipboard
Open code in new window
EnlighterJS 3 Syntax Highlighter
[grafana]
name=grafana
baseurl=https://packages.grafana.com/oss/rpm
repo_gpgcheck=1
enabled=1
gpgcheck=1
gpgkey=https://packages.grafana.com/gpg.key
sslverify=1
sslcacert=/etc/pki/tls/certs/ca-bundle.crt
[grafana] name=grafana baseurl=https://packages.grafana.com/oss/rpm repo_gpgcheck=1 enabled=1 gpgcheck=1 gpgkey=https://packages.grafana.com/gpg.key sslverify=1 sslcacert=/etc/pki/tls/certs/ca-bundle.crt
[grafana]
name=grafana
baseurl=https://packages.grafana.com/oss/rpm
repo_gpgcheck=1
enabled=1
gpgcheck=1
gpgkey=https://packages.grafana.com/gpg.key
sslverify=1
sslcacert=/etc/pki/tls/certs/ca-bundle.crt
  • update repository, then install with yum
Plain text
Copy to clipboard
Open code in new window
EnlighterJS 3 Syntax Highlighter
yum update
yum install grafana
yum update yum install grafana
yum update
yum install grafana
  • activate, run and check status grafana
Plain text
Copy to clipboard
Open code in new window
EnlighterJS 3 Syntax Highlighter
systemctl enable grafana-server
systemctl start grafana-server
systemctl status grafana-server
systemctl enable grafana-server systemctl start grafana-server systemctl status grafana-server
systemctl enable grafana-server
systemctl start grafana-server
systemctl status grafana-server

Install grafana on ubuntu

For Ubuntu, here’s how:

  • Add and install the APT repository for grafana
Plain text
Copy to clipboard
Open code in new window
EnlighterJS 3 Syntax Highlighter
sudo apt-get install -y gnupg2 curl software-properties-common
curl https://packages.grafana.com/gpg.key | sudo apt-key add -
sudo add-apt-repository "deb https://packages.grafana.com/oss/deb stable main"
sudo apt-get install -y gnupg2 curl software-properties-common curl https://packages.grafana.com/gpg.key | sudo apt-key add - sudo add-apt-repository "deb https://packages.grafana.com/oss/deb stable main"
sudo apt-get install -y gnupg2 curl software-properties-common
curl https://packages.grafana.com/gpg.key | sudo apt-key add -
sudo add-apt-repository "deb https://packages.grafana.com/oss/deb stable main"
  • Update repo and install grafana
Plain text
Copy to clipboard
Open code in new window
EnlighterJS 3 Syntax Highlighter
sudo apt-get update
sudo apt-get install grafana
sudo apt-get update sudo apt-get install grafana
sudo apt-get update
sudo apt-get install grafana
  • activate, run and check status grafana
Plain text
Copy to clipboard
Open code in new window
EnlighterJS 3 Syntax Highlighter
systemctl enable grafana-server
systemctl start grafana-server
systemctl status grafana-server
systemctl enable grafana-server systemctl start grafana-server systemctl status grafana-server
systemctl enable grafana-server
systemctl start grafana-server
systemctl status grafana-server

Allow ports on firewall (optional)

If you manage your port access with a firewall, you need to open port 3000 for grafana access. Here are a few ways to open ports for grafana:

  • allow ports on ufw firewall

sudo ufw allow 3000/tcp
sudo ufw allow 3000/tcp

  • allow ports on iptables
Plain text
Copy to clipboard
Open code in new window
EnlighterJS 3 Syntax Highlighter
iptables -A INPUT -p tcp --dport 3000 -j ACCEPT
iptables-save | sudo tee /etc/sysconfig/iptables
systemctl restart iptables
iptables -A INPUT -p tcp --dport 3000 -j ACCEPT iptables-save | sudo tee /etc/sysconfig/iptables systemctl restart iptables
iptables -A INPUT -p tcp --dport 3000 -j ACCEPT
iptables-save | sudo tee /etc/sysconfig/iptables
systemctl restart iptables

Disable registration and anonymous access (optional)

Usually, I disable this feature by going to the grafana configuration file in 

/etc/grafana/grafana.ini
/etc/grafana/grafana.ini. Find
allow_sign_up
allow_sign_up under
[user]
[user] , remove the
;
; (semicolon) lthen type
false
false into that value. like this :

Plain text
Copy to clipboard
Open code in new window
EnlighterJS 3 Syntax Highlighter
...
[users]
# disable user signup / registration
allow_sign_up = false
...
... [users] # disable user signup / registration allow_sign_up = false ...
...
[users]
# disable user signup / registration
allow_sign_up = false
...

Then find

enabled
enabled under
[auth.anonymous]
[auth.anonymous] , remove the
;
; (semicolon) then type
false
false into that value. like this :

Plain text
Copy to clipboard
Open code in new window
EnlighterJS 3 Syntax Highlighter
...
[auth.anonymous]
# enable anonymous access
enabled = false
...
... [auth.anonymous] # enable anonymous access enabled = false ...
...
[auth.anonymous]
# enable anonymous access
enabled = false
...

When it’s finished,, restart grafana and check grafana status

Plain text
Copy to clipboard
Open code in new window
EnlighterJS 3 Syntax Highlighter
systemctl restart grafana-server
systemctl status grafana-server
systemctl restart grafana-server systemctl status grafana-server
systemctl restart grafana-server
systemctl status grafana-server

Access grafana, then update user and password

After the installation is complete, then we need to update the grafana admin password. Access grafana at

http://IP_ADDR_or_DOMAIN:3000
http://IP_ADDR_or_DOMAIN:3000 , then login using
user: admin
user: admin and
password: admin
password: admin.

monitoring server with grafana - login page

Grafana Login

then you will be asked to change the admin password,

monitoring server with grafana - change password

Grafana Change Admin Password

If you have successfully changed the password, you will be directed to the grafana homepage

grafana homepage

Grafana Homepage

So far we have successfully logged in and accessed Grafana. However, we can’t monitor anything because the dashboard is still empty. Next, let’s connect Prometheus with Grafana so that we can create a server monitoring dashboard.

 

Add prometheus datasource to Grafana

Before creating the dashboard, we need to add Prometheus as the data source that we will use to create the dashboard.

  • First of all, we go to the Configuration menu, then click Add data source,
grafana - configuration

Grafana > Configuration

 

  • Next, we Select Prometheus as the datasource
grafana - configuration - add datasource

Grafana > Configuration > Add data source

 

  • Fill in the configuration section as needed, most importantly fill in the prometheus URL with
    http://IP_ADDR_or_DOMAIN:9090
    http://IP_ADDR_or_DOMAIN:9090 according to where our prometheus is.
grafana data sources setup

Grafana Data Sources Setup

 

  • Then click Save & Test to save and test whether the configuration is successful.
grafana data sources setup 2

Grafana Data Sources Setup

 

  • If the configuration is successful, a notification will appear Data source is working
grafana test datasource

Grafana Test Configuration

At this point we have successfully added the prometheus source data to the grafana. Next we will try to make a simple monitoring dashboard.

 

Create simple monitoring dashboard

I will create a simple dashboard example. Approximately the dashboard display will be like this:

monitoring server with grafana

Simple Monitoring Dashboard

From the GIF animation above, we can see, I created a dashboard to monitor my 3 servers, namely main-server, node-a and node-b. To make it easier to say, we initialize main-server as $server-1, node-a as $server-2 and node-b as $server-3.

 

    To shorten the time, you can download the JSON file below which I have created to upload on Grafana to become a dashboard like above. But, before you use it, change the values of $server-1, $server-2 and $server-3 to be the IP address of each of your own servers.

    → Download simple-dashboard

    If you have adjusted it, you can accept the following steps :

    • Go to Grafana, access menu Dashboard > Manage > Import
    • Upload JSON file or Import via panel json , to upload the json file you only need to upload the json file that you downloaded and adjusted the value, then to import via the json panel, you only need to paste the json code that you have adjusted into the form that you have adjusted. provided.
    • then click Import

    For iImport via panel json the process is more or less like the following GIF animation :

    grafana upload simple dashboard

    Upload Simple Dashboard

    Okay, you have succeeded in making a simple server monitoring dashboard. If there is a chance I will try to explain how the dashboard is made.

     

    Alright, it looks like I’ve been explaining Server Monitoring for a while with Grafana, Prometheus and node_exporter. If there is anything you want to ask, please ask in the comments column. Hopefully my writing can be useful, Thank you very much 😀 😎

     

     


    4 Comments

    மருத்துவ ஆலோசனை 2021 டிசம்பர் · 11 December 2021 at 6:38 pm

    bookmarked!!, I love your website!

      Lukems · 12 December 2021 at 9:46 pm

      thanks for helping you with this article 🙂

    AntonioEpisa · 29 December 2021 at 10:27 pm

    Will come up with new guide related to Prometheus with Grafana, subscribe to newsletter and stay tuned.

      Lukems · 14 January 2022 at 1:57 pm

      Thank you, later I will add content about Prometheus with Grafana

    Leave a Reply

    Avatar placeholder

    Your email address will not be published. Required fields are marked *