Gravity Sync, an easy way to keep multiple Pi-hole In-sync

What is better than a Pi-hole blocking ads via DNS on your network? That's right, Two Pi-hole! But if you have more than one Pi-hole (PH) in your network you'll want a simple way to keep the list configurations identical between the two. I've recently been toiling away at a script to do exactly this, and what started as a few lines lines of bash was shared with some friends, and has grown into what I now call Gravity Sync -- which I'd now like to share with you all for your feedback.

https://github.com/vmstan/gravity-sync

At it's core, Gravity Sync is maybe a handful of core bash commands, that uses rsync to reach out to a remote host, copy the running gravity.db file that contains the Pi-hole blocklist, and then replaces the copy on the local system. What Gravity Sync provides is an easy way to keep this happening in the background. Ideally you set it and forget it. In the long term, it would be awesome if the Pi-hole team made this entire script unncessary.

Gravity Sync will not overwrite device specific settings such as local network configuration, admin/API passwords/keys, local hostfiles, upstream DNS resolvers, etc. It will also not keep DHCP settings or device leases synchronized.

Gravity Sync requires Pi-hole 5.0 or higher.

You will need to designate one Pi-Hole as primary and one as secondary. This is where you'll make all your configuration changes through the Web UI, doing things such as; manual whitelisting, adding blocklists, device/group management, and other list settings. Gravity Sync will pull the configuration of the primary PH to the secondary. It will also bring over the downloaded blocklist files after a pihole -g update on the primary, so you do not need to reach out to all your blocklist hosts for updates after syncing.

The designation of primary and secondary is purely at your discretion and depends on your desired use case.

Additionally, some things to consider:

  • Gravity Sync is regularly tested during development with Ubuntu and Raspberry Pi OS (previously, Raspbian). As Gravity Sync is just an (admittedly) long bash script, it will likely work on other Linux distributions that have the bash shell installed. But please file an Issue if you're unable to run it on another platform.
  • Gravity Sync has not been tested with Docker container deployments of Pi-hole, and is not expected to work there without major modifications. You will need Pi-hole setup with a "traditional" install directly in the base operating system.

I would encourage you to read through the documentation on the GitHub site before deployment. The actual install and configuration is fairly simple. I've worked to build in all of the functions necessary to deploy and operate Gravity Sync with as little work by the end user as possible.

I look forward to any feedback you might have, and would encourage you to file any Issues on GitHub.

Thanks!

11 Likes

This is some awesome stuff @vmstan! Obviously, this neat little script got out of hand and is now full fledged software with its own home on GitHub. Cool!

I'm curious, I'm probably just not that smart, but can you explain to me why you had this need to run 2 different Pi-holes in your network with identical configurations? I'd love to hear more about your setup and how this lead to you writing all this code.

1 Like

I have two RPi4 in my house that both run Pi-hole in an HA config using keepalived, so that if for some reason one of the Pi die, need to do updates, or I want to do something else disruptive to one of them, I can take it offline without disrupting my wife's need to resolve Facebook.com :wink: In this keepalived method there is only one DNS address handed out to clients, which bounces between the two Pi.

The Easy Way to do this, is just to have your router/DHCP server handing out two DNS Pi-Hole IP addresses. I don't really like this option as much, as clients pick which one they want to resolve to and I want a central place to review lookups for troubleshooting.

Regardless of the method you use to implement redundant Pi-holes, the script functions the same.

You could also use this to keep Pi-hole's at different physical locations in-sync, but that would require poking holes in firewalls and NATing SSH access, which is a tutorial for another time.

Holy cow! You created a professional 2 tiered DNS fallback for redundancy purposes at home? I love it.

Thanks again for this software to keep both in sync, which is a must have since you do not want to fall back on a second device that is not up to speed with the main one.

Now something like this must have been born from necessity. Did you have your main device running Pi-hole go down on you before? Can you tell us some more about what happened?

I'm an IT infrastructure engineer. 2 is 1 and 1 is none. :smiley:

4 Likes

1.7 release that was just published will now also manage the custom.list file that contains the "Local DNS Records" function within the Pi-hole interface. If you do not want this feature enabled it can be bypassed by adding a SKIP_CUSTOM='1' to your GS .conf file. Sync will be trigged during a pull operation if there are changes to either file.

Cool solution! I also wanted to run redundant Pi-holes. I wound up deploying to Docker Swarm with GlusterFS to keep the instances in sync. Haven't had any issues with the 5.0 db. My latest blog post is about swarm MACVLAN routing and links to my Ansible playbook, http://jpft.win/docker-swarm-macvlan/

Just to understand it correctly how i would be able to setup this for PiHole v5,
so I actually use the first part of the keepalive from the HA setup here
and use your script to keep them in to sync ?
And don't use the part to sync what's explained in that guide that worked with pihole 4.x

Yep. All the keepalived specific instructions apply regardless of if you sync the databases or not, but GS will keep them happy.

1 Like

Hi vmstan,

thank you so much for gravity sync! I am currently running pi-hole on two separate RPis but will go for a HA/keepalived system, now that I read gravity sync's documentation and the benefits of keepalived.

However, against your advice I use primary RPi to act as DHCP-server as well, since I love the possibility to configure the (quasi static)DHCP-addresses and have a working name resolution for those devices as well. Could you advise how to make the name resolution for those DHCP-clients also available on secondary RPi? I could include all entries in custom.list manually again, but since they are all pretty defined in "/etc/dnsmasq.d/04-pihole-static-dhcp.conf" this would mean double work. How do you implement the name resolution for your dhcp-clients?

Many thanks for your thoughts!

I use a Ubiquiti USG for DHCP/router, because I have multiple VLAN/subnets (one for servers, one for wireless, one for wired devices, one for guest users) but this is how I tell them where to look for name resolution on those networks. The only thing I have listed in Local DNS Settings are systems in the server network with static IP addresses assigned.

Create a file called: /etc/dnsmasq.d/10-dns-lookups.conf

server=/8.168.192.in-addr.arpa/192.168.7.1
server=/9.168.192.in-addr.arpa/192.168.7.1
server=/3.168.192.in-addr.arpa/192.168.7.1

server=/guest.smg/192.168.7.1
server=/254.168.192.in-addr.arpa/192.168.7.1

Assuming you only have one network, and if your secondary Pi-hole doesn't know about the internal domain name, I think you'd only need the last two lines in the file on the secondary, substituting your domain name and the IP address of the primary pihole/DHCP server on the first, and again with your subnet and pihole/DHCP on the second. In my configuration they point to the router interface on the same subnet as the servers/Pihole.

This is only an educated guess :slight_smile:

Restart Pihole's DNS after to pickup the changes.

1 Like

vmstan,

many thanks for your reply, I'll give it a try!

Thanks for this excellent creation / solution! Two quick questions...

First, why not run this from the gravity.sh script so it just runs upon any actual changes vs. running every 30 minutes. Would this be an option? I can see, since you pull from primary to secondary why it may not work with your initial design.

This leads to me second question. Is there a reason this could not run on the primary and push to the secondary? The main reason I ask is that my primary is running on an internal, local, Pi server. But my secondary is running in GCP. So, while it would be possible for me to set up a NAT for the device, and secure the access, it would be easier for me to push from my local LAN externally to the publicly sitting Pi-Hole running in GCP.

Thoughts? Thanks in advance!

The reason why I don't trigger it with Pi-hole directly is I didn't want to build something that required modifying the base code, especially since it would require reconfiguration anytime your Pi-hole was updated. Less risk for me in being targeted for "breaking Pi-hole" than already have with shuffling files around. However I don't see why it wouldn't work if that's what you wanted to do.

You should be able to push by default. The original script was really built around the pull concept at the core, with push as really a recovery method should the primary go down and you needed to make changes there. In 2.0 the "smart" sync should go either way, so if you were running it on the primary it would just be detecting changes locally and pushing them to the GCP instance.

Try it and let me know :slight_smile:

Great! Thanks for the response and makes sense. I'll give it a try via push. Thanks again!

Hi vmstan,

A quick question: does gravity sync also synchronizes the long term/query database as well? It would be nice to see all querys from both piholes when I search the history.

Thanks!

Please open issues on the Github for the app.

It does not.

Hi @vmstan,

great script and seems to be working for me, two questions (please remember i'm a noob):

  1. When i run the command ./gravity-sync.sh compare on my secondary pi i get the response No custom.list Detected on secondary pi (see picture). Is this normal? Everything seems to work fine.
    NoCustomList
  2. In your tutorial on github under the section configuration it says create a configuration file with ./gravity-sync.sh config. This must be run from the home directory /home/pi/gravity-sync as all of the gravity-sync.sh commands either they will not run, correct?

Thanks, Mark

Please open issues on the Github for the app.