Monitoring your Cardano Stake pool servers is a very important and basic thing to do, in order to keep an eye on memory usage (which can be tricky), CPU, missed slots, block height etc.

So, like many others SPO, i’ve set up a dedicated monitoring server with Grafana and Prometheus. The main issue if you want to access your Grafana Dashboard from anywhere, out-of-the-box, is that you have to expose the application port (http:3000 by default) on the public address of your server. Not so secure right ? To avoid that, a very popular solution is to simply create an SSH tunnel with a port forwarding option to your Grafana Server, and block any traffic on your server, on port 3000 (with UFW , or a perimeter firewall).

But honestly we can do better 😉

A more elegant and still secure solution is to configure a reverse proxy, with a SSL certificate. It is a simple way to increase your Grafana access security, without the need to use a SSH tunnel. I’m going to show you how to install an NGINX Reverse Proxy on your Grafana Server, and then add a valid SSL certificate with Let’s Encrypt (I’m using Ubuntu 22.0.4 LTS on my monitoring server) :

Pre-requisite : You must have a proper domain name; and FQDN set to your grafana’s public IP address. For exemple : grafana.yourdomain.com –> <your public IP address>. You can buy a domain name basically on any hosting site, like OVH or namecheap. You’ll have access to a DNS config tool where you’ll be able to set up your DNS entry. There are plenty of guides out there to do this 😉

 

Nginx Reverse Proxy Installation

Install nginx

sudo apt install nginx

Check nginx status

sudo systemctl status nginx

Now you should be able to visit the public IP address of your server http://<your ip address> ! That should lead you to the default Nginx page

We are going to create an Nginx config file for your grafana server

cd /etc/nginx/sites-enabled
sudo nano <your FQDN>.conf

(change <your FQDN> with your actual FQDN like grafana.yourdomain.com

Next we enable Nginx for your FQDN and activate HTTP reverse proxy to your grafana 

server { 
listen 80;
server_name <your FQDN>;

location / {
proxy_set_header Host $http_host;
proxy_pass http://localhost:3000/;
} 
}

Save your file and restart Nginx

sudo systemctl restart nginx

Now access your monitoring server http://<your FQDN> : you should see the Grafana login page !

Nginx cleanup : remove the default enabled site

rm /etc/nginx/sites-enabled/default

SSL Let’s Encrypt installation with NGINX

Now that we have a working Reverse Proxy on our monitoring server, we are going to add SSL layer to encrypt properly access to your Cardano Stakepool Grafana dashboard. To do this, we are going to use a free SSL certificate provider, Let’s Encrypt, with Certbot.

We use snap to install certbot on our grafana server. First check make sure you have the lasted version of Snap; and then install Certbot

sudo snap install core; sudo snap refresh core
sudo snap install --classic certbot

Make a sym link so you can use certbot command anywhere

sudo ln -s /snap/bin/certbot /usr/bin/certbot

Start the installation and follow the process of installing SSL certificate for your FQDN

sudo certbot --nginx

At the end of installation, you should be able to access your Grafana Server with HTTPS :

https://<your FQDN>

Congratulations 🙂

 

Post install Nginx hardening

First of all, you have to let ports 80 and 443 opened on your Monitoring Server. Nginx will automatically forward any HTTP requests to HTTPS, but it’s important to have both open, in order for Certbot to renew your certificate every 3 month. You also have to block any external connections to port 3000, but let localhost:3000 allowed (this is mandatory for Nginx to forward trafic form Reverse Proxy). You can do it with UFW.

In this example deny any incoming connections, except SSH, HTTP and HTTPS. I allow any outgoing connections. Modify to suit your needs :

sudo ufw default deny incoming
sudo ufw default allow outgoing
sudo ufw allow ssh
sudo ufw allow 80/tcp
sudo ufw allow 443/tcp
sudo ufw enable

Next we are going to block any unwanted HTTP method, except POST GET and HEAD

sudo nano /etc/nginx/sites-enabled/<your FQDN.conf>

Paste these lines inside your location / block :

limit_except GET HEAD POST { deny all; }

Save and close.

No open Nginx configuration file

sudo nano /etc/nginx/nginx.conf

Remove old cipher suites TLSv1.0 and TLSv1.1 : the SSL Settings should look like this :

##
# SSL Settings
##

ssl_protocols TLSv1.2 TLSv1.3; # Dropping SSLv3, ref: POODLE
ssl_prefer_server_ciphers on;

Prevent DoS and Buffer Oversized attacks on your Grafana server : Add this lines inside http block :

## Start: Size Limits & Buffer Overflows ##
client_body_buffer_size 3K;
client_header_buffer_size 1k;
client_max_body_size 80k;
large_client_header_buffers 2 2k;
## END: Size Limits & Buffer Overflows ##

### Directive describes the zone, in which the session states are stored i.e. store in slimits. ###
### 1m can handle 32000 sessions with 32 bytes/session, set to 5m x 32000 session ###
limit_conn_zone $binary_remote_addr zone=addr:5m;

### Control maximum number of simultaneous connections for one session i.e. ###
### restricts the amount of connections from a single ip address ###
limit_conn addr 10;

These limits work for me, on my Cardano Stakepool Dashboard. But you may need to adjust the values if something is broken after you restart Nginx.

Save and close.

Restart Nginx server

sudo systemctl restart nginx

Congratulations, you now have a nice and secure access to your Grafana Monitoring Server. You can check on your Cardano Stakepool Nodes, easily from anywhere 🙂

Next step : How to configure Google OATH to access your Grafana Dashboard