I have just completed deploying redundant pi-hole DNS/DHCP servers in our environment and they are working quite well.
So that both servers will be able to respond to DNS requests for all clients with DHCP leases, I have scripts on each that regenerates /etc/pihole/custom.list from the other servers /etc/pihole/dhcp.leases. But this is being accomplished using cron which is a bit clunky and incurs some latency.
I was wondering if there was a "hook" in pihole I could take advantage of when the dhcp leases change rather than poling with cron every 5 minutes.
Made some improvements to this service to allow stopping/restarting cleanly. Be sure to modify the "Target" section with your hostnames. And keep in mind, you need to install inotifywait and set up public key authentication for the root accounts between the two systems.
/lib/systemd/system/SyncDHCP.service
[Unit]
Description=Updates DHCP DNS between two pihole servers
After=network.target
[Service]
ExecStart=/usr/bin/SyncDHCP.sh Start
ExecStop=/usr/bin/SyncDHCP.sh Stop
ExecRestart=/usr/bin/SyncDHCP.sh Restart
[Install]
WantedBy=multi-user.target
/usr/bin/SyncDHCP.sh
#! /usr/bin/bash
Flag="/tmp/SyncDHCP.running"
Src="/etc/pihole/dhcp.leases"
function Msg {
printf "%s - %s\n" $(date +%u%m%d%H%M%S) "$*"
}
function Stop {
Msg "Stopping SyncDHCP service..."
rm -f $Flag
proc=$(ps -aux | awk '/inotifywait.*dhcp\.leases/{print $2}')
[ -n "$proc" ] && kill -HUP $proc
sleep 3 # Wait for filesystem cleanup
}
function Start {
Msg "Starting SyncDHCP service..."
Dst=$(mktemp)
Tmp=$(mktemp)
# Modify these target names to conform to your deployment
if [ $(hostname) == "dns1" ]
then
Target=dns2
else
Target=dns1
fi
# Create the "run flag file"
touch $Flag
while [ -e $Flag ]
do
inotifywait -e modify $Src | \
while read -r f a
do
size=$(cksum $Src | awk '{print $2}')
if [ "$size" -ne 0 ]
then
awk '$4!="*"{printf "%s %s\n",$3,$4}' $Src | sort -k3 > $Tmp
if ! cmp -s $Dst $Tmp
then
diff $Dst $Tmp | awk '$1~/<|>/{
if ($1~/</)printf "released %s %s\n",$2,$3;
if ($1~/>/)printf "assigned %s %s\n",$2,$3;
}'
scp $Tmp root@$Target:/etc/pihole/custom.list > /dev/null && \
ssh root@$Target "chmod 744 /etc/pihole/custom.list;pihole restartdns reload" && \
cp $Tmp $Dst && \
Msg "Updated custom.list on $Target"
fi
fi
done
done
rm -f $Dst
rm -f $Tmp
}
function Restart {
Stop
Start
}
case "$1" in
Start)
Start
;;
Stop)
Stop
;;
Restart)
Restart
;;
Status)
systemctl status SyncDHCP.service
;;
*)
echo "Usage: $0 {Start|Stop|Restart|Status}"
;;
esac
2 Likes
This could be tackled by DNS means, by simply configuring your two Pi-hole's DHCP and DNS options.
Configure your DHCP servers for a different Pi-hole domain name and a distinctive Range of IP addresses to hand out via Settings | DHCP, e.g. lan1 and lan2.
Then head to the bottom of Settings | DNS and enable Pi-hole's Conditional Forwarding and point requests for lan1 to your second Pi-hole and vice versa.
2 Likes
Hmm, multiple local domain names is not desirable in our case. Are you certain that is required?
2 Likes
Pi-hole will forward the DNS request based on that domain name.
If you'd use the same local domain on both Pi-holes, you'd have configured a partial DNS loop for unknown hostnames.
You may be able to short-circuit that loop by defining a client-specific blocking rule on just one(!) Pi-hole for local hostname requests from the respective other.
Try adding the following rule as RegEx filters:
^[^\.]+(\.local\.domain)?$
where local.domain is your local domain name.
In addition, you'd have to create a Group (e.g. 'prevent local loops'), then add your respective other Pi-hole as its sole Client and attach it to your 'prevent local loops' group.
2 Likes
Yes, I thought I had tried this and wound up with DNS loops.
Our environment is a little involved: 6 local subnets, DHCP forwarding to the piholes is handle by our Cisco router, each subnet requires separate (but single) domain name AND search domains list for each lease. To top it off, we have a parent company to which I must forward requests for the corporate domains.
Our current method is working great and it is fast!
Even if get conditional forwarding working, aren't I going to have an extra forward for (potentially) half the systems on our local subnets?
Do you use the "Local DNS" functionality in the Pi-hole admin console? Or is everything in your environment a static DHCP lease that you manage this way? I tried out your script and it seems the script logic nukes the existing custom.list file, which is where local DNS records are stored (which are different compared to static DHCP leases).
I.e.,
awk '$4!="*"{printf "%s %s\n",$3,$4}' $Src | sort -k3 > $Tmp
Generates the $Tmp file from $Src, $Src being dhcp.leases. The diff is nice to show logging, but the $Tmp file goes on as is to overwrite the contents of custom.list, which is fine as long as you don't use the local DNS functionality (particularly relevant for static IPs that don't use DHCP at all).
I guess you could create static leases for hosts configured for a static IP; they'll never retrieve the lease but it doesn't matter as long as the lease and static IP are the same. That or tweak the script logic to not trample existing data, but the former seems easier. Just curious what you did in your environment to not trample existing, manually-configured DNS records.
We do not use the "Local DNS"; in our environment, custom.list is used only for the purpose of storing "the other" servers active leases. We DO have some static DHCP leases but we handle them with a .conf file under /etc/dnsmasq.d. Any purely static IPs we just put in /etc/hosts.
Thanks for the reply.
We DO have some static DHCP leases but we handle them with a .conf file under /etc/dnsmasq.d.
I assume you have to manually sync this .conf file on both Pi-holes; is that right? I take it something like gravity-sync isn't expecting your custom .conf file and won't keep it synced between both hosts as it would if it were looking at /etc/dnsmasq.d/04-pihole-static-dhcp.conf, which is updated via the webpage (and you're not using that).
Any purely static IPs we just put in /etc/hosts.
Same question here: /etc/hosts must be manually replicated on both Pi-holes, correct?
Bottom line is that your automation focuses solely on syncing DHCP leases. Static DHCP leases and static DNS records must be manually configured in /etc/dnsmasq.d/xx-custom.conf and /etc/hosts respectively. Let me know if I misunderstood something and thanks again.
Really wish at least Pi-hole would allow multi-subnet DHCP in the webpage at a minimum. Ideally, native, first-party sync capability for failover would be lovely. But I understand it's a consumer-type product. It's just that it's even gained enterprise adoption in some cases, which is awesome and crazy at the same time.
Out of curiosity, why not use the webpage for this in combination with gravity-sync? gravity-sync would handle replication of custom.list and 04-pihole-static-dhcp.conf.
Unrelated note:
It seems you have to make sure each Pi-hole does not have an overlapping pool, since you never transfer the dhcp.leases contents into the dhcp.leases file on the other Pi-hole; you're only moving it into custom.list. So in this scenario, and assuming two Pi-holes have the same pool they're serving from:
- New client comes online
- Pi-hole 2 responds and hands out a lease (192.168.1.2), also updating custom.list on Pi-hole 1
- A second, new client comes online
- Pi-hole 1 responds and, unaware that 192.168.1.2 has been handed out by Pi-hole 2, hands out a lease (192.168.1.2), also updating custom.list on Pi-hole 2
Now each custom.list on contains a row meant to address two separate hosts that are sharing a conflicting IP. I'm curious why you didn't try and synchronize dhcp.leases between the two servers.
1 Like
Not familiar with gravity-sync. SSH/SCP is native, works fine, and is useful for myriad other purposes. Once you have something that works, you move on to the next problem.
Correct: non-overlapping pools. Nothing new there. SOP since the 1990s (yes, I have been doing this that long, longer, actually).
2 Likes
I dealt with the overwriting of the custom.list file by saving a master copy called custom.list.permanent and then concatenating it to $Tmp just before the secure copy command to the other host.
Perm="/etc/pihole/custom.list.permanent"
cat $Perm >> $Tmp
Of course this means that I now must add and edit local DNS records directly in custom.list.permanent instead of through the pi-hole Web UI.
That's one way. And I'm glad someone is finding this useful!
But, since pihole utilizes custom.list and /etc/hosts equally effectively, one could "awk and cat" to /etc/hosts on the alternate server. This would free the custom.list to be used exclusively (as intended) by the web interface. I didn't do this in my implementation because I am "old school" and add all my statics directly to /etc/hosts - I had no use for custom.list. So it became the target for the dynamic address changes.
So far, my piholes are working great (and I have bigger fish to fry elsewhere). But if I need to come back to this, I will investigate appending the dynamics directly to a static block in /etc/hosts on the alternate server. Because it would be nice if that part of the web interface worked as intended.
1 Like
Is your complete solution that allows you to synchronize two PiHole instances available somewhere?
The two files listed above (SyncDHCP.sh,SyncDHCP.service) are all it takes.
Configuration:
- Configure both Piholes with non-overlapping address pools.
- Copy the files to the given directory and set permissions (.sh must be chmod'd 744)
- Establish public key authentication between root on both systems
-
- apt install openssh-server
-
-
- Install inotify-tools (apt install inotify-tools)
- Enable the service (systemctl enable SyncDHCP.service)
1 Like
I haven't yet upgraded to the Pi-hole v6 docker image due to concern that this solution will need to be tweaked. Does the script still work with v6?
Has not been tested with v6. I was sorta hoping they built this capability into the next version...
1 Like
I don't see where is pihole configuration copy from dns1 to dns2, i thought you make a backup configuration from dns1 and upload it to dns2, and and copy dhcp leases in this two files from this topic.
Yes. That is what SyncDHCP service does:
In the "Start" function, inotifywait is used to trap changes to the local /etc/pihole/dhcp.leases. Then, after some qualification testing (not all changes require a copy) and conversion (awk), the service uses scp to copy those changes to the alternate servers /etc/pihole/custom.list and ssh to have the alternate reload immediately.
The result is redundant DHCP/DNS servers. Regardless of which server provides the lease, the change is immediately copied to the alternate.