High availability (HA) for Pi-hole (running two Pi-hole's)

this is a great feature request and I look forward to have it implemented directly on the UI

Hi All-
I'd suggest renaming this feature to something like, "Sync Settings Between 2 or More Pi-hole instances."

It occurs to me that trying to load-balance or round-robin through a proxy IP creates a new single point of failure (e.g., the load-balancer crashes, and host are only configured with the IP of the load-balancer).

Most (if not all) routers are able to configure DHCP clients with 2 or more DNS servers. It seems more attainable for the noob to buy 2 Rpi's. Run the installer on the first Rpi and configure their preferred settings, then run the installer on a second device and choose an option to, "Configure this Pihole as a backup to existing Pihole."

The second PiHole would need remote access to the first Pihole, and a script to periodically check for changes to certain configuration files or settings; and if-changed, import those settings.

Considerations would need to be made if RP1 is configured as a DHCP server (i.e, to avoid having 2 active DHCP servers). The RPi2 would need to "survive on its own" if RPi1 goes down (i.e., RPi2 should perform its own gravity updates and not rely on RPi1 syncing). The user interface should indicate the last time RPi2 has completed a sync, and a warning or error should be visible if RPi1 is not accessible by RPi2.

I am going to try to adapt the script identified above for my scenario. I think it captures most of the basic settings files, but I need it to also sync my 02-pihole.conf in /etc/dnsmasq.d for conditional forwarding (More than one "Conditional Forwarding" entry in the GUI - #5 by sbellon).

Cheers!

3 Likes

I have my nodes set-up in that manner.

node 1 is the master and it is also the dhcp server.

Node 2 is a secondary DNS that probes the Pi-hole instance and checks for FTL errors and or if the host is up (DNS1 that is).

If the host has any errors or it is offline, it becomes the DHCP server for the network (maintaining the same leases and IP assignments within the network).

As soon as node 1 is fixed, it reverts itself to secondary diaabling the dhcp side.

No blacklists whitelists or other settings are migrated. It would be easy to implement though via rsync.

2 Likes

Very cool. How are you probing the health of Node 1 and checking the FTL errors? Are you using a linux package for cluster syncing? Or is this a custom bash+cron solution you developed?

I ask because my linux scripting-fu is weak and I'd rather not reinvent the wheel.

It is a bash script that i wrote ...

I shared it on the forum already somewhere :blush: (can’t rememwbr where - trying to find it).

L.E. Can’t find it.

I’ll tweak mine and remove the sensitive stuff and publish it tomorrow.

1 Like

found it here: Good solution to automatically revert to "normal" if Pi Hole dies? - #4 by jfb

Yep. That one.

I was embarrassed about how nasty it is (but still works) and i rebuilt the whole damn thing with functions. Less code repetition and a lot easier to debug.
It’s also sexy to look at :blush:

Using this script why does it update gravity afterwards on the remote machine. it basically renders the script useless as both piholes wont match blocked domain numbers

@RamSet I was hoping to see your new updated script that uses function and more sexiness.

Also inquiring into the status of this as a native feature in the application now that it is 2020.

1 Like

I’m mobile right now. Will post the link to it in a few hours.

Here we go:

This one uses the configuration files (copied) from the original (monitored) Pi-hole instance, located in /etc/dnsmasq.d/
Files (according to the script) need to be stored in /home/pi/stuff/ (path can be changed in the pass function).

Take note of the file names and adjust accordingly the steps where they are deleted.

Do not delete 01-pihole.conf from the backup Pi-hole as that will render the instance unusable.

My systems are heavily embedded with pushover notifications and hence this code, does that too (via an external command).

The pushover lines can be removed without any impact on the overall functionality of the "monitor".

This might work for some people....
UPDATED: Confirmed this works (Implemented on my systems)

LSYNCD

I haven't tested on pihole yet, but I use it at work to keep a HA pair of servers in sync.
You have the option to select/exclude dirs/files you want sync'd

Both Servers

sudo mkdir /etc/lsyncd
sudo vi /etc/lsyncd/lsyncd.conf
sudo mkdir /var/log/lsyncd
sudo touch /var/log/lsyncd/lsyncd.status
sudo touch /var/log/lsyncd/lsyncd.log

=

On the Primary PiHole (IP=192.168.1.1)

sudo vi /etc/lsyncd/lsyncd.conf.lua

settings {
  logfile = "/var/log/lsyncd/lsyncd.log",
  statusFile = "/var/log/lsyncd/lsyncd.status",
  statusInterval = 20
}
 
sync {
  default.rsyncssh,
  delete = false,
  source = "/etc/pihole/",
  host = "192.168.1.2",
  targetdir = "/etc/pihole/",
  excludeFrom = "/etc/lsyncd/lsyncd.exclude",
  delay = 20,
  rsync = {
    archive = true,
    owner = true,
    perms = true,
    group = true,
    compress = false,
    whole_file = true,
  }
}

=
On the Secondary PiHole (IP = 192.168.1.2)

sudo vi /etc/lsyncd/lsyncd.conf.lua

settings {
  logfile = "/var/log/lsyncd/lsyncd.log",
  statusFile = "/var/log/lsyncd/lsyncd.status",
  statusInterval = 20
}
 
sync {
  default.rsyncssh,
  delete = false,
  source = "/etc/pihole/",
  host = "1912.168.1.1",
  targetdir = "/etc/pihole/",
  excludeFrom = "/etc/lsyncd/lsyncd.exclude",
  delay = 20,
  rsync = {
    archive = true,
    owner = true,
    perms = true,
    group = true,
    compress = false,
    whole_file = true,
  }
}

=
On both servers...

cat /etc/lsync/lsyncd.exclude
setupVars.conf
local.list
localbranches
localversions
dhcp.leases
install.log
local.list
*.db
*.db-journal
.gravity*
.list*
whitelist/.git*

=
Usage

sudo systemctl status lsyncd -l
sudo systemctl force-reload lsyncd
sudo systemctl start lsyncd
sudo systemctl restart lsyncd

If you are using any crontab entries to update adlists/whitelists/...etc, this might conflict?

2 Likes

What about using docker? You could put together a docker file that replaces the list files with what you have in your volume. Any time you update the file in the volume, just roll the containers and they'll pick up the changes.

Still trying to get a LB to work. (Internet consist of 1 bar of 4Glte where I live... not even DSL...)

Have you made any progress here? I was wondering if two pihole containers could just share the same volume...

You can easily sync the /etc/pihole/gravity.db database between your devices, possibly even by mounting it from a network share. This database is a single source of truth containing blocked domains, adlists, configured clients, groups and black-/whitelists.

Statistics synchronization is still something which is unlikely to come as it would require a radical change to how FTL processes data as FTL stores everything in memory + precomputes statistics to be able to reply to your requests within milliseconds. If you have used Pi-hole in the pre-FTL times (three years ago), you know the difference in speed through the introduction of FTL was several orders of magnitude and only made using Pi-hole on low-end hardware really enjoyable.

1 Like

Hi,
After 5.0 release, where settings include groups and per-user blocking , the need of a unique configuration for networks using 2 pi-holes is more needed.
Please consider adding it.

Thanks in advance
Best Regards,

1 Like

As I said, syncing this one file is sufficient. You should run pihole restartdns reload-lists after the synchronization and all your groups and per-client settings are in sync.

The issue is: How? Some users will want to sync it though a third party (some will use an internal NAS, some maybe even an external party like Dropbox). Others will maybe want Pi-hole to maintain/operate a file sync system themselves.

I'm aware that there are many votes on this. However, not only given the limited time of the developers, but also that it still seems rather unclear how the technical realization should really look like, I'm afraid this may still take a bit of time. If anyone can come up with a good solution and opens a PR against our publicly available code base, it will surely speed up the discussion around this.

Have you checked out @fireheadman's idea above? I'd rather not like to add this into our automated installer, however, we can surely work on a guide just like Redirecting... which is used by many users to set up custom deployments of their Pi-holes.

3 Likes

I'm still using the old reddit script mentioned here earlier.
Do I need to adjust the list of files after upgrading to 5.0 ?
Currently I have this:
FILES=(black.list blacklist.txt regex.list whitelist.txt lan.list) #list of files you want to sync

Hi DL6ER,
So your advice is to sync only:
/etc/pihole/gravity.db
using any method (for example lsyncd) and run:
pihole restartdns reload-lists
after the sync.
Is this enough to maintain secondary pi-hole as the main one?

For the automatic sync I suggest some sort of API between pi-holes so when a change is done to one of them it's sent to the others. I understand it's not trivial.

Best Regards,

1 Like