Share the same gravity.db across multiple containers

Feature Requests
1

As it is not possible to assign clients to a specific DNS resolver, I am running two instances of Pi-Hole in Docker. Since I want to use the exact same rules and blocklists with all devices I have both instances share the same gravity.db file.

It works perfectly fine except when updating the blocklist because a container can't remove a volume:

rm: cannot remove '/etc/pihole/gravity.db': Device or resource busy
mv: cannot move '/etc/pihole/gravity_temp.db' to '/etc/pihole/gravity.db': Device or resource busy

If you mount the whole /etc/pihole/ folder both instances also share their DNS resolver config.

It would solve this issue if either Pi-Hole would directly modify the gravity.db file, which, however seems to be a lot slower or if the file would be stored in an extra folder so the folder instead of the file itself can be mounted as a volume.

This doesn't seem like a perfect solution either because it would probably break many custom scripts working with the gravity.db file.

However, even if it comes with some issues, it would be great if the user had the option to enable either of these behaviours (unless someone can think of a better solution ?).

2

Without any further testing, I'd suggest just putting the gravity.db file into another folder. FTL will follow the new patch when configured in /etc/pihole/pihole-FTL.conf (see GRAVITYDB), however, I seem to recall having seen a bug report that pihole -g does not honor the changed location.

Fixing the latter should give you want you want/need, right?

1 Like
5

Yes, this would be perfect. I didn't realize that you could change the path in the FTL config.
However, as you anticipated, it doesn't work:

root@feb86c61f9ae:/# pihole -g
Error: no such table: info
  [i] Neutrino emissions detected...
  [✗] Pulling blocklist source list into range
  [i] No source list found, or it is empty

  [i] Building tree...
  [✗] Unable to build gravity tree in /etc/pihole/gravity_temp.db
  Error: index idx_gravity already exists

  [✗] Unable to update gravity timestamp in database /etc/pihole/gravity.db
  Error: near line 2: no such table: info
Error: no such table: vw_gravity
Error: no such table: vw_gravity
  [i] Number of gravity domains:  ( unique domains)
Error: near ")": syntax error
Error: no such table: vw_blacklist
  [i] Number of exact blacklisted domains:
Error: no such table: vw_regex_blacklist
  [i] Number of regex blacklist filters:
Error: no such table: vw_whitelist
  [i] Number of exact whitelisted domains:
Error: no such table: vw_regex_whitelist
  [i] Number of regex whitelist filters:
  [✓] Flushing DNS cache
  [✓] Cleaning up stray matter

  [✓] DNS service is listening
     [✓] UDP (IPv4)
     [✓] TCP (IPv4)
     [✓] UDP (IPv6)
     [✓] TCP (IPv6)

  [✓] Pi-hole blocking is enabled
7
1 Like
8

Thanks!

Bug fix available here:

@i6nfinity Could you test if

pihole checkout core fix/GRAVITDB_in_pihole-g

makes this working as intended for you?

1 Like
9

While updating there is one error with sed and the database commands are printed out:

  [✓] Fetching branches from https://github.com/pi-hole/pi-hole.git
  [i] 32 branches available for Pi-hole Core

  [✓] Switching to branch: 'fix/GRAVITDB_in_pihole-g' from 'refs/heads/master'
Already up to date.
  [i] Running installer to upgrade your installation

  [✓] Root user check

        .;;,.
        .ccccc:,.
         :cccclll:.      ..,,
          :ccccclll.   ;ooodc
           'ccll:;ll .oooodc
             .;cll.;;looo:.
                 .. ','.
                .',,,,,,'.
              .',,,,,,,,,,.
            .',,,,,,,,,,,,....
          ....''',,,,,,,'.......
        .........  ....  .........
        ..........      ..........
        ..........      ..........
        .........  ....  .........
          ........,,,,,,,'......
            ....',,,,,,,,,,,,.
               .',,,,,,,,,'.
                .',,,,,,'.
                  ..'''.

  [✓] Update local cache of available packages
  [i] Existing PHP installation detected : PHP version 7.3.19-1~deb10u1
  [i] Performing unattended setup, no whiptail dialogs will be displayed

  [✗] Checking apt-get for upgraded packages
      Kernel update detected. If the install fails, please reboot and try again
  [i] Installer Dependency checks...
  [✓] Checking for dhcpcd5
  [✓] Checking for git
  [✓] Checking for iproute2
  [✓] Checking for whiptail
  [✓] Checking for dnsutils

  [✓] Supported OS detected
  [i] SELinux not detected
  [✓] Check for existing repository in /etc/.pihole
  [✓] Update repo in /etc/.pihole

  [✓] Check for existing repository in /var/www/html/admin
  [i] Update repo in /var/www/html/admin...HEAD is now at 989e1ba0 Merge pull request #1737 from pi-hole/release/v5.4
  [✓] Update repo in /var/www/html/admin

  [i] Main Dependency checks...
  [✓] Checking for cron
  [✓] Checking for curl
  [✓] Checking for iputils-ping
  [✓] Checking for lsof
  [✓] Checking for netcat
  [✓] Checking for psmisc
  [✓] Checking for sudo
  [✓] Checking for unzip
  [✓] Checking for wget
  [✓] Checking for idn2
  [✓] Checking for sqlite3
  [✓] Checking for libcap2-bin
  [✓] Checking for dns-root-data
  [✓] Checking for libcap2
  [✓] Checking for lighttpd
  [✓] Checking for php7.3-common
  [✓] Checking for php7.3-cgi
  [✓] Checking for php7.3-sqlite3
  [✓] Checking for php7.3-xml
  [✓] Checking for php7.3-json
  [✓] Checking for php7.3-intl

  [✓] Enabling lighttpd service to start on reboot...
  [✓] Checking for user 'pihole'

  [i] FTL Checks...

  [✓] Detected ARMv7 processor (with hard-float support)
  [i] Checking for existing FTL binary...
  [i] Latest FTL Binary already installed (v5.7). Confirming Checksum...
  [i] Checksum correct. No need to download!
  [✓] Installing scripts from /etc/.pihole

  [i] Installing configs from /etc/.pihole...
  [i] Existing dnsmasq.conf found... it is not a Pi-hole file, leaving alone!
  [✓] Copying 01-pihole.conf to /etc/dnsmasq.d/01-pihole.conf

  [i] Installing blocking page...
  [✓] Creating directory for blocking page, and copying files
  [i] Backing up index.lighttpd.html
      No default index.lighttpd.html file found... not backing up

  [✓] Installing sudoer file

  [✓] Installing latest Cron script

  [✓] Installing latest logrotate script
  [i] Backing up /etc/dnsmasq.conf to /etc/dnsmasq.conf.old
  [i] man not installed
sed: cannot rename /etc/pihole/sedAq8UAq: Device or resource busy
  [i] Testing if systemd-resolved is enabled
  [i] Systemd-resolved does not need to be restarted
  [✓] Restarting lighttpd service...
  [✓] Enabling lighttpd service to start on reboot...
  [i] Restarting services...
  [✓] Enabling pihole-FTL service to start on reboot...
  [✓] Restarting pihole-FTL service...
  [i] Upgrading gravity database from version 13 to 14
  [✓] Deleting existing list cache
  [i] Neutrino emissions detected...
  [i] Storing gravity database in /etc/pihole/gravity/gravity.db
  [✓] Pulling blocklist source list into range

  [✓] Preparing new gravity database
  [i] Using libz compression

  [i] Target: https://raw.githubusercontent.com/PolishFiltersTeam/KADhosts/master/KADhosts_without_controversies.txt
  [✓] Status: Retrieval successful
  [i] Analyzed 0 domains

  <...>

  [✓] Storing downloaded domains in new gravity database
  [✓] Building tree
  [i] Swapping databases....timeout 30000

ATTACH DATABASE '/etc/pihole/gravity/gravity.db' AS OLD;

BEGIN TRANSACTION;

DROP TRIGGER tr_domainlist_add;
DROP TRIGGER tr_client_add;
DROP TRIGGER tr_adlist_add;

INSERT OR REPLACE INTO "group" SELECT * FROM OLD."group";
INSERT OR REPLACE INTO domain_audit SELECT * FROM OLD.domain_audit;

INSERT OR REPLACE INTO domainlist SELECT * FROM OLD.domainlist;
INSERT OR REPLACE INTO domainlist_by_group SELECT * FROM OLD.domainlist_by_group;

INSERT OR REPLACE INTO adlist SELECT * FROM OLD.adlist;
INSERT OR REPLACE INTO adlist_by_group SELECT * FROM OLD.adlist_by_group;

INSERT OR REPLACE INTO info SELECT * FROM OLD.info;

INSERT OR REPLACE INTO client SELECT * FROM OLD.client;
INSERT OR REPLACE INTO client_by_group SELECT * FROM OLD.client_by_group;


CREATE TRIGGER tr_domainlist_add AFTER INSERT ON domainlist
    BEGIN
      INSERT INTO domainlist_by_group (domainlist_id, group_id) VALUES (NEW.id, 0);
    END;

CREATE TRIGGER tr_client_add AFTER INSERT ON client
    BEGIN
      INSERT INTO client_by_group (client_id, group_id) VALUES (NEW.id, 0);
    END;

CREATE TRIGGER tr_adlist_add AFTER INSERT ON adlist
    BEGIN
      INSERT INTO adlist_by_group (adlist_id, group_id) VALUES (NEW.id, 0);
    END;


COMMIT;
  [✓] Swapping databases
  [i] Number of gravity domains: 5227684 (3374331 unique domains)
  [i] Number of exact blacklisted domains: 2
  [i] Number of regex blacklist filters: 22
  [i] Number of exact whitelisted domains: 52
  [i] Number of regex whitelist filters: 3
  [✓] Flushing DNS cache
  [✓] Cleaning up stray matter

  [✓] DNS service is listening
     [✓] UDP (IPv4)
     [✓] TCP (IPv4)
     [✓] UDP (IPv6)
     [✓] TCP (IPv6)

  [✓] Pi-hole blocking is enabled

  [i] The install log is located at: /etc/pihole/install.log
Update Complete!

  Current Pi-hole version is fix/GRAVITDB_in_pihole-g v5.2.4-26-gf99cdc97.
  Current AdminLTE version is v5.4.
  Current FTL version is v5.7.

As you can see the gravity update works. The DNS resolution also works fine, but the web interface is just a blank white page and when trying to restart the container I get these error logs:

[s6-init] making user provided files available at /var/run/s6/etc...exited 0.
[s6-init] ensuring user provided files have correct perms...exited 0.
[fix-attrs.d] applying ownership & permissions fixes...
[fix-attrs.d] 01-resolver-resolv: applying...
[fix-attrs.d] 01-resolver-resolv: exited 0.
[fix-attrs.d] done.
[cont-init.d] executing container initialization scripts...
[cont-init.d] 20-start.sh: executing...
/opt/pihole/webpage.sh: line 13: dnsmasqconfig: readonly variable
[cont-init.d] 20-start.sh: exited 1.
[cont-finish.d] executing container finish scripts...
[cont-finish.d] done.
[s6-finish] waiting for services.
[s6-finish] sending all processes the TERM signal.
10

Oh yeah, a left-over debug command. I removed it already.

This is something in the installer, may be related to my comment below:

I'm not experienced with the docker container and don't actually even know if the checkout works reliably in a container. Sorry, I should have mentioned this but I forget this myself...

11

It doesn't. We'd have to make a new image for that.

1 Like
12

This topic was automatically closed 180 days after the last reply. New replies are no longer allowed.