DNS resolution unexpectedly stops after upgrade to 5.0

DL6ER · June 28, 2020, 8:39pm

@dkozinn Could you add

DEBUG_DATABASE=true

to /etc/pihole/pihole-FTL.conf (create if it does not exist, requires sudo powers) and run pihole restartdns ?

This should provide us with some more details next time this happens.

dkozinn · June 28, 2020, 8:52pm

I've done that.

Will the needed information be in the pihole -d output or is there something else I need to provide?

yubiuser · June 28, 2020, 8:58pm

/var/log/pihole-FTL.log is needed again.

dkozinn · July 2, 2020, 3:51pm

Ok, it happened again. I'd run a couple of pihole -g to see somehow that might trigger the issue and it did. The debug token is https://tricorder.pi-hole.net/3jtkw1zriw. The last 5000 lines of pihole-FTL.log are here: https://pastebin.com/Sagym27m

It did eventually recover without intervention after a few minutes.

yubiuser · July 2, 2020, 5:15pm

From a quick look it seems like the interesting lines are between 3821 and 3991.

[2020-07-02 11:41:59.853 1202] Reloading DNS cache

[2020-07-02 11:43:26.199 1202] Compiled 0 whitelist and 0 blacklist regex filters in 5.9 msec

dkozinn · July 2, 2020, 5:25pm

In that section, part of that it's doing is setting the local DNS records. Prior to 5.0, I'd had those in /etc/hosts but switched to using the Local DNS Records tab in the GUI when I switched to 5.0. I don't know if that's relevant but I wanted to mention it.

I've confirmed that all other records in /etc/hosts have been commented out, the only one remaining is the one for the pi-hole device itself.

DanSchaper · July 2, 2020, 5:35pm

*** [ DIAGNOSING ]: Name resolution (IPv4) using a random blocked domain and a known ad-serving domain
[✗] Failed to resolve ads.suite6ixty6ix.com via localhost (127.0.0.1)
[✗] Failed to resolve ads.suite6ixty6ix.com via Pi-hole (192.168.0.48)
[✓] doubleclick.com is 172.217.12.142 via a remote, public DNS server (8.8.8.8)

dkozinn · July 2, 2020, 6:19pm

@DanSchaper That's what happens when I have this issue. Things don't resolve, not even during the diagnostics. I've hat at least one time where I couldn't upload the output because it couldn't resolve tricorder.pi-hole.net when the run finished.

DL6ER · July 3, 2020, 8:04am

[2020-07-02 11:34:05.568 1202] Querying count of distinct domains in gravity database table vw_gravity
[... ignore messages in between, they are from the independent database thread ...]
[2020-07-02 11:35:31.749 1202] Querying count of distinct domains in gravity database table vw_regex_blacklist
[2020-07-02 11:35:31.750 1202] INFO: No regex blacklist entries found

Almost 1.5 minutes are spent on reading one plain value from the database, it seems. We do not count the number ourselves here but use a prepared value we just have to read:

github.com/pi-hole/FTL

Use precomputed gravity count from database

pi-hole:release/v5.0 ← pi-hole:tweak/gravity_count

opened 08:32PM - 04 Feb 20 UTC

DL6ER

+16 -4

**By submitting this pull request, I confirm the following:** - [X] I have re…ad and understood the [contributors guide](https://github.com/pi-hole/pi-hole/blob/master/CONTRIBUTING.md). - [X] I have checked that [another pull request](https://github.com/pi-hole/FTL/pulls) for this purpose does not exist. - [X] I have considered, and confirmed that this submission will be valuable to others. - [X] I accept that this submission may not be used, and the pull request closed at the will of the maintainer. - [X] I give this submission freely, and claim no ownership to its content. **How familiar are you with the codebase?:** ## 10 --- This speeds up FTL start and reload actions. See discussion [here](https://discourse.pi-hole.net/t/disabling-pi-hole-breaks-dns-for-1-min/274929).

So what is being done here is

SELECT value FROM info WHERE property = 'gravity_count'; // <------ This takes almost 90 seconds.
SELECT COUNT(DISTINCT domain) FROM vw_regex_blacklist; // Return immediately, result is 0

I added more debugging output in a branch specifically made for you, please run

pihole checkout ftl fix/dkozinn-debugging

I'll be gone over the weekend but will check your results on Monday. As this specific branch is built on the code scheduled for v5.1, it may even (by accident) already contain a fix for you so no guarantee that the issue you're seeing can still be triggered.

dkozinn · July 3, 2020, 3:30pm

I've installed the version as you've requested, and I'll keep an eye on it. Normally the issue reproduces itself most times with the weekly Gravity update, I'll try to reproduce it as I've done previously.

I appreciate all the effort you have all been putting into this.

BTW, K2DBK here. 73.

yubiuser · July 6, 2020, 1:05pm

Maybe that's the problem: for some reason pihole doesn't use the prepared value but instead calculates the number of distinct domains from vw_gravity again....

github.com

pi-hole/FTL/blob/3d7c095cb10a1b692c80dd28f6051dea91d45baa/src/database/gravity-db.c#L569


      
          
          	// Checking for smaller than GRAVITY_LIST is omitted due to list being unsigned
          	if(list >= UNKNOWN_TABLE)
          	{
          		logg("gravityDB_getTable(%u): Requested list is not known!", list);
          		return false;
          	}
          
          	char *querystr = NULL;
          	// Build correct query string to be used depending on list to be read
          	if(list != GRAVITY_TABLE && asprintf(&querystr, "SELECT COUNT(DISTINCT domain) FROM %s", tablename[list]) < 18)
          	{
          		logg("readGravity(%u) - asprintf() error", list);
          		return false;
          	}
          	// We get the number of unique gravity domains as counted and stored by gravity. Counting the number
          	// of distinct domains in vw_gravity may take up to several minutes for very large blocking lists on
          	// very low-end devices such as the Raspierry Pi Zero
          	else if(list == GRAVITY_TABLE && asprintf(&querystr, "SELECT value FROM info WHERE property = 'gravity_count';") < 18)
          	{
          		logg("readGravity(%u) - asprintf() error", list);

Might be useful to add an explicite debug log for a) using the prepared value b) re-calculating. Right now it uses Querying count of distinct domains in gravity database table vw_gravity for both situations if I understand the code correctly. (Might also help here: https://github.com/pi-hole/pi-hole/issues/3507#issuecomment-654149786)

DL6ER · July 6, 2020, 2:08pm

I agree logging can/should be improved here, however, the situation is already clear right now. Just not clear in the sense of "obvious".

You already quoted the correct location, let's explore why this is showing that FTL is not counting the number of gravity domains here. I may be wrong about this and it would be awesome if you could show me where. Until then, I'll go ahead and tell you why I think I'm right.

There are two branches here, you quoted the first but this one which is only chosen if list != GRAVITY_TABLE.

Since we have GRAVITY_TABLE == 0 and tablename[GRAVITY_TABLE] == "vw_gravity" it is guaranteed that FTL is using the second branch when it is logging

Querying count of distinct domains in gravity database table vw_gravity

github.com

pi-hole/FTL/blob/3d7c095cb10a1b692c80dd28f6051dea91d45baa/src/database/gravity-db.c#L577


      
          	char *querystr = NULL;
          	// Build correct query string to be used depending on list to be read
          	if(list != GRAVITY_TABLE && asprintf(&querystr, "SELECT COUNT(DISTINCT domain) FROM %s", tablename[list]) < 18)
          	{
          		logg("readGravity(%u) - asprintf() error", list);
          		return false;
          	}
          	// We get the number of unique gravity domains as counted and stored by gravity. Counting the number
          	// of distinct domains in vw_gravity may take up to several minutes for very large blocking lists on
          	// very low-end devices such as the Raspierry Pi Zero
          	else if(list == GRAVITY_TABLE && asprintf(&querystr, "SELECT value FROM info WHERE property = 'gravity_count';") < 18)
          	{
          		logg("readGravity(%u) - asprintf() error", list);
          		return false;
          	}
          
          	if(config.debug & DEBUG_DATABASE)
          		logg("Querying count of distinct domains in gravity database table %s", tablename[list]);
          
          	// Prepare query
          	int rc = sqlite3_prepare_v2(gravity_db, querystr, -1, &table_stmt, NULL);

Meaning

SELECT value FROM info WHERE property = 'gravity_count';

is used here.

DL6ER · July 6, 2020, 2:41pm

@dkozinn Please try

pihole checkout ftl tweak/gravity_counting_logging

I'm not sure it will change something, however, I simplified some logic which should have been correct before (see post above) but now we do everything fully explicit - it may or not help with things.

--... ...--

dkozinn · July 6, 2020, 4:01pm

I've done that, but pihole -c says "FTL offline" next to Blocked, and the stats are all zero.

  Hostname: pi1.xxxxx.xxx     (Raspberry Pi 3, Model B+)
    Uptime: 3 days, 00:24:11                                                                                     
 Task Load: 0.57 0.49 0.46     (Active: 2 of 50 tasks)
 CPU usage: 8%                 (4x 1.4 GHz @ 140f)
 RAM usage: 12%                (Used: 115 MB of 926 MB)
 HDD usage: 41%                (Used: 11 GB of 28 GB)
  LAN addr: 192.168.0.48       (Gateway: 192.168.0.1)
   Pi-hole: Active             (Blocking: 0 sites)
 Ads Today: 0%                 (Total: 0 of 0)
Local Qrys: 0%                 (6 DNS servers)
   Blocked: **FTL offline**                                                                                          
Top Advert:                                                                                                      
Top Domain:                                                                                                      
Top Client:

However, I see that in the web interface the number appear to be correct, and data is being written to the logs, so perhaps this is a cosmetic issue with pihole -c.

Just to check, I've restarted pihole as well as rebooting the device and the output looks the same.

The good news is that blocking is definitely working.

I've run pihole -g a few times and had no issues, but as I mentioned this wasn't reliably reproducible so I suppose for now I'll just let it run and see what happens.

Anything else I can do?

yubiuser · July 6, 2020, 7:11pm

I tried hard but didn't succeed. I'm not used to read C and didn't come up with the trick to read it backwards from the output.

The new branch is much better readable Let's see if it sheds light on the mysterious delay.

yubiuser · July 8, 2020, 1:48pm

I could reproduce the issue.

  Pi-hole version is development v5.0-118-ge76f011 (Latest: v5.0)
  AdminLTE version is v5.0 (Latest: v5.0)
  FTL version is new/tre-regex vDev-bc7d84e (Latest: v5.0)

I only use one the default group. Switching it off in group management and then on again stops DNS resolution. Pihole hangs at Querying count of distinct domains in gravity database table vw_gravity. Manually restating by pihole restartdns works without issues.

[2020-07-08 15:26:48.270 10745] gravityDB_open(): Trying to open /etc/pihole/gravity.db in read-only mode
[2020-07-08 15:26:48.271 10745] gravityDB_open(): Setting location for temporary object to MEMORY
[2020-07-08 15:26:48.271 10745] gravityDB_open(): Preparing audit query
[2020-07-08 15:26:48.274 10745] gravityDB_open(): Setting busy timeout to 1000
[2020-07-08 15:26:48.274 10745] gravityDB_open(): Setting busy timeout to zero
[2020-07-08 15:26:48.274 10745] gravityDB_open(): Successfully opened gravity.db
[2020-07-08 15:26:48.274 10745] Querying count of distinct domains in gravity database table vw_gravity
[2020-07-08 15:29:55.711 11047] Using log file /var/log/pihole-FTL.log
[2020-07-08 15:29:55.712 11047] ########## FTL started! ##########

Switching to tweak/gravity_counting_logging and dis/enable default group didn't stop DNS resolution.

~~However I noticed that the count for regex black/whitelist is wrong....~~

Maybe vw_gravity was a red hering and it's more the counting of the regex entries.....?
What happens first: counting or logging?

Full log

2020-07-08 15:37:27.701 11740M] ########## FTL started! ##########
[2020-07-08 15:37:27.701 11740M] FTL branch: tweak/gravity_counting_logging
[2020-07-08 15:37:27.701 11740M] FTL version: vDev-5f9a590
[2020-07-08 15:37:27.701 11740M] FTL commit: 5f9a590
[2020-07-08 15:37:27.701 11740M] FTL date: 2020-07-06 16:36:26 +0200
[2020-07-08 15:37:27.701 11740M] FTL user: pihole
[2020-07-08 15:37:27.701 11740M] Compiled for aarch64 (compiled on CI) using aarch64-linux-gnu-gcc (Debian 6.3.0-18) 6.3.0 20170516
[2020-07-08 15:37:27.701 11740M] Starting config file parsing (/etc/pihole/pihole-FTL.conf)
[2020-07-08 15:37:27.701 11740M]    SOCKET_LISTENING: only local
[2020-07-08 15:37:27.701 11740M]    AAAA_QUERY_ANALYSIS: Show AAAA queries
[2020-07-08 15:37:27.702 11740M]    MAXDBDAYS: max age for stored queries is 90 days
[2020-07-08 15:37:27.702 11740M]    RESOLVE_IPV6: Resolve IPv6 addresses
[2020-07-08 15:37:27.702 11740M]    RESOLVE_IPV4: Resolve IPv4 addresses
[2020-07-08 15:37:27.702 11740M]    DBINTERVAL: saving to DB file every minute
[2020-07-08 15:37:27.702 11740M]    DBFILE: Using /etc/pihole/pihole-FTL.db
[2020-07-08 15:37:27.702 11740M]    MAXLOGAGE: Importing up to 24.0 hours of log data
[2020-07-08 15:37:27.702 11740M]    PRIVACYLEVEL: Set to 0
[2020-07-08 15:37:27.702 11740M]    IGNORE_LOCALHOST: Show queries from localhost
[2020-07-08 15:37:27.702 11740M]    BLOCKINGMODE: Null IPs for blocked domains
[2020-07-08 15:37:27.702 11740M]    ANALYZE_ONLY_A_AND_AAAA: Disabled. Analyzing all queries
[2020-07-08 15:37:27.702 11740M]    DBIMPORT: Importing history from database
[2020-07-08 15:37:27.702 11740M]    PIDFILE: Using /run/pihole-FTL.pid
[2020-07-08 15:37:27.702 11740M]    SOCKETFILE: Using /run/pihole/FTL.sock
[2020-07-08 15:37:27.702 11740M]    SETUPVARSFILE: Using /etc/pihole/setupVars.conf
[2020-07-08 15:37:27.703 11740M]    MACVENDORDB: Using /etc/pihole/macvendor.db
[2020-07-08 15:37:27.703 11740M]    GRAVITYDB: Using /etc/pihole/gravity.db
[2020-07-08 15:37:27.703 11740M]    PARSE_ARP_CACHE: Active
[2020-07-08 15:37:27.703 11740M]    CNAME_DEEP_INSPECT: Active
[2020-07-08 15:37:27.703 11740M]    DELAY_STARTUP: No delay requested.
[2020-07-08 15:37:27.703 11740M]    NICE: Set process niceness to -10 (default)
[2020-07-08 15:37:27.703 11740M]    BLOCK_ESNI: Enabled, blocking _esni.{blocked domain}
[2020-07-08 15:37:27.703 11740M]    NAMES_FROM_NETDB: Enabled, trying to get names from network database
[2020-07-08 15:37:27.703 11740M] *****************************
[2020-07-08 15:37:27.703 11740M] * Debugging enabled         *
[2020-07-08 15:37:27.703 11740M] * DEBUG_DATABASE        YES *
[2020-07-08 15:37:27.703 11740M] * DEBUG_NETWORKING      NO  *
[2020-07-08 15:37:27.704 11740M] * DEBUG_LOCKS           NO  *
[2020-07-08 15:37:27.704 11740M] * DEBUG_QUERIES         NO  *
[2020-07-08 15:37:27.704 11740M] * DEBUG_FLAGS           NO  *
[2020-07-08 15:37:27.704 11740M] * DEBUG_SHMEM           NO  *
[2020-07-08 15:37:27.704 11740M] * DEBUG_GC              NO  *
[2020-07-08 15:37:27.704 11740M] * DEBUG_ARP             NO  *
[2020-07-08 15:37:27.704 11740M] * DEBUG_REGEX           NO  *
[2020-07-08 15:37:27.704 11740M] * DEBUG_API             NO  *
[2020-07-08 15:37:27.704 11740M] * DEBUG_OVERTIME        NO  *
[2020-07-08 15:37:27.704 11740M] * DEBUG_EXTBLOCKED      NO  *
[2020-07-08 15:37:27.704 11740M] * DEBUG_CAPS            NO  *
[2020-07-08 15:37:27.704 11740M] * DEBUG_DNSMASQ_LINES   NO  *
[2020-07-08 15:37:27.704 11740M] * DEBUG_VECTORS         NO  *
[2020-07-08 15:37:27.704 11740M] * DEBUG_RESOLVER        NO  *
[2020-07-08 15:37:27.704 11740M] *****************************
[2020-07-08 15:37:27.704 11740M] Finished config file parsing
[2020-07-08 15:37:27.706 11740M] dbquery: "SELECT VALUE FROM ftl WHERE id = 0;"
[2020-07-08 15:37:27.707 11740M]          ---> Result 6 (int)
[2020-07-08 15:37:27.707 11740M] Database version is 6
[2020-07-08 15:37:27.707 11740M] Database successfully initialized
[2020-07-08 15:37:27.708 11740M] DB_read_queries(): "SELECT * FROM queries WHERE timestamp >= 1594129047"
[2020-07-08 15:37:27.709 11740M] New upstream server: 127.0.0.1 (0/128)
[2020-07-08 15:37:27.715 11740M] Resizing "/FTL-strings" from 4096 to 8192
[2020-07-08 15:37:27.733 11740M] Resizing "/FTL-strings" from 8192 to 12288
[2020-07-08 15:37:27.757 11740M] Resizing "/FTL-queries" from 262144 to 524288
[2020-07-08 15:37:27.759 11740M] Resizing "/FTL-strings" from 12288 to 16384
[2020-07-08 15:37:27.760 11740M] New upstream server: 10.0.1.1 (1/128)
[2020-07-08 15:37:27.918 11740M] Resizing "/FTL-queries" from 524288 to 786432
[2020-07-08 15:37:28.085 11740M] Resizing "/FTL-queries" from 786432 to 1048576
[2020-07-08 15:37:28.253 11740M] Resizing "/FTL-queries" from 1048576 to 1310720
[2020-07-08 15:37:28.421 11740M] Resizing "/FTL-queries" from 1310720 to 1572864
[2020-07-08 15:37:28.567 11740M] Resizing "/FTL-queries" from 1572864 to 1835008
[2020-07-08 15:37:28.596 11740M] Resizing "/FTL-strings" from 16384 to 20480
[2020-07-08 15:37:28.630 11740M] Resizing "/FTL-queries" from 1835008 to 2097152
[2020-07-08 15:37:28.670 11740M] Resizing "/FTL-queries" from 2097152 to 2359296
[2020-07-08 15:37:28.673 11740M] Resizing "/FTL-strings" from 20480 to 24576
[2020-07-08 15:37:28.747 11740M] Resizing "/FTL-queries" from 2359296 to 2621440
[2020-07-08 15:37:28.749 11740M] Resizing "/FTL-strings" from 24576 to 28672
[2020-07-08 15:37:28.805 11740M] Imported 40411 queries from the long-term database
[2020-07-08 15:37:28.807 11740M]  -> Total DNS queries: 40411
[2020-07-08 15:37:28.807 11740M]  -> Cached DNS queries: 4290
[2020-07-08 15:37:28.807 11740M]  -> Forwarded DNS queries: 28785
[2020-07-08 15:37:28.808 11740M]  -> Blocked DNS queries: 7336
[2020-07-08 15:37:28.808 11740M]  -> Unknown DNS queries: 0
[2020-07-08 15:37:28.808 11740M]  -> Unique domains: 1275
[2020-07-08 15:37:28.808 11740M]  -> Unique clients: 13
[2020-07-08 15:37:28.808 11740M]  -> Known forward destinations: 2
[2020-07-08 15:37:28.808 11740M] Successfully accessed setupVars.conf
[2020-07-08 15:37:28.808 11740M] DEBUG: argv[0] = ""
[2020-07-08 15:37:28.808 11740M] DEBUG: argv[1] = "-k"
[2020-07-08 15:37:28.817 11742M] PID of FTL process: 11742
[2020-07-08 15:37:28.818 11742/T11743] Listening on port 4711 for incoming IPv4 telnet connections
[2020-07-08 15:37:28.819 11742/T11744] Listening on port 4711 for incoming IPv6 telnet connections
[2020-07-08 15:37:28.819 11742M] Reloading DNS cache
[2020-07-08 15:37:28.819 11742M] Blocking status is enabled
[2020-07-08 15:37:28.820 11742M] *****************************
[2020-07-08 15:37:28.820 11742M] * Debugging enabled         *
[2020-07-08 15:37:28.820 11742M] * DEBUG_DATABASE        YES *
[2020-07-08 15:37:28.820 11742M] * DEBUG_NETWORKING      NO  *
[2020-07-08 15:37:28.820 11742M] * DEBUG_LOCKS           NO  *
[2020-07-08 15:37:28.820 11742M] * DEBUG_QUERIES         NO  *
[2020-07-08 15:37:28.820 11742M] * DEBUG_FLAGS           NO  *
[2020-07-08 15:37:28.820 11742M] * DEBUG_SHMEM           NO  *
[2020-07-08 15:37:28.824 11742M] * DEBUG_GC              NO  *
[2020-07-08 15:37:28.825 11742M] * DEBUG_ARP             NO  *
[2020-07-08 15:37:28.825 11742M] * DEBUG_REGEX           NO  *
[2020-07-08 15:37:28.825 11742M] * DEBUG_API             NO  *
[2020-07-08 15:37:28.826 11742M] * DEBUG_OVERTIME        NO  *
[2020-07-08 15:37:28.826 11742M] * DEBUG_EXTBLOCKED      NO  *
[2020-07-08 15:37:28.826 11742M] * DEBUG_CAPS            NO  *
[2020-07-08 15:37:28.826 11742M] * DEBUG_DNSMASQ_LINES   NO  *
[2020-07-08 15:37:28.826 11742M] * DEBUG_VECTORS         NO  *
[2020-07-08 15:37:28.826 11742M] * DEBUG_RESOLVER        NO  *
[2020-07-08 15:37:28.826 11742M] *****************************
[2020-07-08 15:37:28.826 11742/T11745] Listening on Unix socket
[2020-07-08 15:37:28.827 11742M] dbquery: "DELETE FROM message;"
[2020-07-08 15:37:28.837 11742M]          ---> OK
[2020-07-08 15:37:28.838 11742M] gravityDB_open(): Trying to open /etc/pihole/gravity.db in read-only mode
[2020-07-08 15:37:28.838 11742M] gravityDB_open(): Setting location for temporary object to MEMORY
[2020-07-08 15:37:28.840 11742M] gravityDB_open(): Preparing audit query
[2020-07-08 15:37:28.843 11742M] gravityDB_open(): Setting busy timeout to 1000
[2020-07-08 15:37:28.843 11742M] gravityDB_open(): Setting busy timeout to zero
[2020-07-08 15:37:28.843 11742M] gravityDB_open(): Successfully opened gravity.db
[2020-07-08 15:37:28.843 11742M] Querying count of distinct domains in gravity database table vw_gravity: SELECT value FROM info WHERE property = 'gravity_count';
[2020-07-08 15:37:28.844 11742M] Querying count of distinct domains in gravity database table vw_regex_blacklist: SELECT COUNT(DISTINCT domain) FROM vw_regex_blacklist
[2020-07-08 15:37:28.851 11742M] Querying count of distinct domains in gravity database table vw_regex_whitelist: SELECT COUNT(DISTINCT domain) FROM vw_regex_whitelist
[2020-07-08 15:37:28.854 11742M] Querying gravity database for client 10.0.1.1 (getting best match)
[2020-07-08 15:37:28.855 11742M] SQL: Comparing 10.0.1.1 vs. 10.0.1.0/16 (subnet 255.255.0.0) - !! MATCH !!
[2020-07-08 15:37:28.855 11742M] SQL: Comparing 10.0.1.1 vs. 10.0.1.0/16 (subnet 255.255.0.0) - !! MATCH !!
[2020-07-08 15:37:28.855 11742M] SQL: Comparing 10.0.1.1 vs. 10.0.10.1/24 (subnet 255.255.255.0) - NO MATCH
[2020-07-08 15:37:28.855 11742M] SQL: Comparing 10.0.1.1 vs. 10.0.1.0/16 (subnet 255.255.0.0) - !! MATCH !!
[2020-07-08 15:37:28.856 11742M] Querying gravity database for client 10.0.1.1 (getting groups)
[2020-07-08 15:37:28.857 11742M] Querying gravity database for client 10.0.99.192 (getting best match)
[2020-07-08 15:37:28.858 11742M] SQL: Comparing 10.0.99.192 vs. 10.0.1.0/16 (subnet 255.255.0.0) - !! MATCH !!
[2020-07-08 15:37:28.858 11742M] SQL: Comparing 10.0.99.192 vs. 10.0.1.0/16 (subnet 255.255.0.0) - !! MATCH !!
[2020-07-08 15:37:28.859 11742M] SQL: Comparing 10.0.99.192 vs. 10.0.10.1/24 (subnet 255.255.255.0) - NO MATCH
[2020-07-08 15:37:28.859 11742M] SQL: Comparing 10.0.99.192 vs. 10.0.1.0/16 (subnet 255.255.0.0) - !! MATCH !!
[2020-07-08 15:37:28.859 11742M] Querying gravity database for client 10.0.99.192 (getting groups)
[2020-07-08 15:37:28.861 11742M] Querying gravity database for client 10.0.40.3 (getting best match)
[2020-07-08 15:37:28.861 11742M] SQL: Comparing 10.0.40.3 vs. 10.0.1.0/16 (subnet 255.255.0.0) - !! MATCH !!
[2020-07-08 15:37:28.862 11742M] SQL: Comparing 10.0.40.3 vs. 10.0.1.0/16 (subnet 255.255.0.0) - !! MATCH !!
[2020-07-08 15:37:28.862 11742M] SQL: Comparing 10.0.40.3 vs. 10.0.10.1/24 (subnet 255.255.255.0) - NO MATCH
[2020-07-08 15:37:28.862 11742M] SQL: Comparing 10.0.40.3 vs. 10.0.1.0/16 (subnet 255.255.0.0) - !! MATCH !!
[2020-07-08 15:37:28.862 11742M] Querying gravity database for client 10.0.40.3 (getting groups)
[2020-07-08 15:37:28.864 11742M] Querying gravity database for client 10.0.10.64 (getting best match)
[2020-07-08 15:37:28.865 11742M] SQL: Comparing 10.0.10.64 vs. 10.0.1.0/16 (subnet 255.255.0.0) - !! MATCH !!
[2020-07-08 15:37:28.865 11742M] SQL: Comparing 10.0.10.64 vs. 10.0.1.0/16 (subnet 255.255.0.0) - !! MATCH !!
[2020-07-08 15:37:28.865 11742M] SQL: Comparing 10.0.10.64 vs. 10.0.10.1/24 (subnet 255.255.255.0) - !! MATCH !!
[2020-07-08 15:37:28.865 11742M] SQL: Comparing 10.0.10.64 vs. 10.0.10.1/24 (subnet 255.255.255.0) - !! MATCH !!
[2020-07-08 15:37:28.865 11742M] SQL: Comparing 10.0.10.64 vs. 10.0.10.1/24 (subnet 255.255.255.0) - !! MATCH !!
[2020-07-08 15:37:28.865 11742M] Querying gravity database for client 10.0.10.64 (getting groups)
[2020-07-08 15:37:28.867 11742M] Querying gravity database for client 10.0.10.136 (getting best match)
[2020-07-08 15:37:28.867 11742M] SQL: Comparing 10.0.10.136 vs. 10.0.1.0/16 (subnet 255.255.0.0) - !! MATCH !!
[2020-07-08 15:37:28.868 11742M] SQL: Comparing 10.0.10.136 vs. 10.0.1.0/16 (subnet 255.255.0.0) - !! MATCH !!
[2020-07-08 15:37:28.868 11742M] SQL: Comparing 10.0.10.136 vs. 10.0.10.1/24 (subnet 255.255.255.0) - !! MATCH !!
[2020-07-08 15:37:28.868 11742M] SQL: Comparing 10.0.10.136 vs. 10.0.10.1/24 (subnet 255.255.255.0) - !! MATCH !!
[2020-07-08 15:37:28.868 11742M] SQL: Comparing 10.0.10.136 vs. 10.0.10.1/24 (subnet 255.255.255.0) - !! MATCH !!
[2020-07-08 15:37:28.868 11742M] Querying gravity database for client 10.0.10.136 (getting groups)
[2020-07-08 15:37:28.869 11742M] Querying gravity database for client 10.0.99.33 (getting best match)
[2020-07-08 15:37:28.870 11742M] SQL: Comparing 10.0.99.33 vs. 10.0.1.0/16 (subnet 255.255.0.0) - !! MATCH !!
[2020-07-08 15:37:28.870 11742M] SQL: Comparing 10.0.99.33 vs. 10.0.1.0/16 (subnet 255.255.0.0) - !! MATCH !!
[2020-07-08 15:37:28.870 11742M] SQL: Comparing 10.0.99.33 vs. 10.0.10.1/24 (subnet 255.255.255.0) - NO MATCH
[2020-07-08 15:37:28.870 11742M] SQL: Comparing 10.0.99.33 vs. 10.0.1.0/16 (subnet 255.255.0.0) - !! MATCH !!
[2020-07-08 15:37:28.870 11742M] Querying gravity database for client 10.0.99.33 (getting groups)
[2020-07-08 15:37:28.871 11742M] Querying gravity database for client 10.0.10.84 (getting best match)
[2020-07-08 15:37:28.872 11742M] SQL: Comparing 10.0.10.84 vs. 10.0.1.0/16 (subnet 255.255.0.0) - !! MATCH !!
[2020-07-08 15:37:28.872 11742M] SQL: Comparing 10.0.10.84 vs. 10.0.1.0/16 (subnet 255.255.0.0) - !! MATCH !!
[2020-07-08 15:37:28.872 11742M] SQL: Comparing 10.0.10.84 vs. 10.0.10.1/24 (subnet 255.255.255.0) - !! MATCH !!
[2020-07-08 15:37:28.872 11742M] SQL: Comparing 10.0.10.84 vs. 10.0.10.1/24 (subnet 255.255.255.0) - !! MATCH !!
[2020-07-08 15:37:28.872 11742M] SQL: Comparing 10.0.10.84 vs. 10.0.10.1/24 (subnet 255.255.255.0) - !! MATCH !!
[2020-07-08 15:37:28.872 11742M] Querying gravity database for client 10.0.10.84 (getting groups)
[2020-07-08 15:37:28.873 11742M] Querying gravity database for client 10.0.10.190 (getting best match)
[2020-07-08 15:37:28.874 11742M] SQL: Comparing 10.0.10.190 vs. 10.0.1.0/16 (subnet 255.255.0.0) - !! MATCH !!
[2020-07-08 15:37:28.874 11742M] SQL: Comparing 10.0.10.190 vs. 10.0.1.0/16 (subnet 255.255.0.0) - !! MATCH !!
[2020-07-08 15:37:28.874 11742M] SQL: Comparing 10.0.10.190 vs. 10.0.10.1/24 (subnet 255.255.255.0) - !! MATCH !!
[2020-07-08 15:37:28.874 11742M] SQL: Comparing 10.0.10.190 vs. 10.0.10.1/24 (subnet 255.255.255.0) - !! MATCH !!
[2020-07-08 15:37:28.874 11742M] SQL: Comparing 10.0.10.190 vs. 10.0.10.1/24 (subnet 255.255.255.0) - !! MATCH !!
[2020-07-08 15:37:28.874 11742M] Querying gravity database for client 10.0.10.190 (getting groups)
[2020-07-08 15:37:28.875 11742M] Querying gravity database for client 127.0.0.1 (getting best match)
[2020-07-08 15:37:28.875 11742M] SQL: Comparing 127.0.0.1 vs. 10.0.1.0/16 (subnet 255.255.0.0) - NO MATCH
[2020-07-08 15:37:28.875 11742M] SQL: Comparing 127.0.0.1 vs. 10.0.10.1/24 (subnet 255.255.255.0) - NO MATCH
[2020-07-08 15:37:28.876 11742M] Querying gravity database for client 10.0.1.2 (getting best match)
[2020-07-08 15:37:28.877 11742M] SQL: Comparing 10.0.1.2 vs. 10.0.1.0/16 (subnet 255.255.0.0) - !! MATCH !!
[2020-07-08 15:37:28.877 11742M] SQL: Comparing 10.0.1.2 vs. 10.0.1.0/16 (subnet 255.255.0.0) - !! MATCH !!
[2020-07-08 15:37:28.877 11742M] SQL: Comparing 10.0.1.2 vs. 10.0.10.1/24 (subnet 255.255.255.0) - NO MATCH
[2020-07-08 15:37:28.877 11742M] SQL: Comparing 10.0.1.2 vs. 10.0.1.0/16 (subnet 255.255.0.0) - !! MATCH !!
[2020-07-08 15:37:28.877 11742M] Querying gravity database for client 10.0.1.2 (getting groups)
[2020-07-08 15:37:28.878 11742M] Querying gravity database for client 10.0.1.4 (getting best match)
[2020-07-08 15:37:28.879 11742M] SQL: Comparing 10.0.1.4 vs. 10.0.1.0/16 (subnet 255.255.0.0) - !! MATCH !!
[2020-07-08 15:37:28.879 11742M] SQL: Comparing 10.0.1.4 vs. 10.0.1.0/16 (subnet 255.255.0.0) - !! MATCH !!
[2020-07-08 15:37:28.879 11742M] SQL: Comparing 10.0.1.4 vs. 10.0.10.1/24 (subnet 255.255.255.0) - NO MATCH
[2020-07-08 15:37:28.879 11742M] SQL: Comparing 10.0.1.4 vs. 10.0.1.0/16 (subnet 255.255.0.0) - !! MATCH !!
[2020-07-08 15:37:28.879 11742M] Querying gravity database for client 10.0.1.4 (getting groups)
[2020-07-08 15:37:28.880 11742M] Querying gravity database for client 10.0.1.3 (getting best match)
[2020-07-08 15:37:28.880 11742M] SQL: Comparing 10.0.1.3 vs. 10.0.1.0/16 (subnet 255.255.0.0) - !! MATCH !!
[2020-07-08 15:37:28.881 11742M] SQL: Comparing 10.0.1.3 vs. 10.0.1.0/16 (subnet 255.255.0.0) - !! MATCH !!
[2020-07-08 15:37:28.881 11742M] SQL: Comparing 10.0.1.3 vs. 10.0.10.1/24 (subnet 255.255.255.0) - NO MATCH
[2020-07-08 15:37:28.881 11742M] SQL: Comparing 10.0.1.3 vs. 10.0.1.0/16 (subnet 255.255.0.0) - !! MATCH !!
[2020-07-08 15:37:28.881 11742M] Querying gravity database for client 10.0.1.3 (getting groups)
[2020-07-08 15:37:28.882 11742M] Querying gravity database for client 10.0.10.6 (getting best match)
[2020-07-08 15:37:28.882 11742M] SQL: Comparing 10.0.10.6 vs. 10.0.1.0/16 (subnet 255.255.0.0) - !! MATCH !!
[2020-07-08 15:37:28.882 11742M] SQL: Comparing 10.0.10.6 vs. 10.0.1.0/16 (subnet 255.255.0.0) - !! MATCH !!
[2020-07-08 15:37:28.882 11742M] SQL: Comparing 10.0.10.6 vs. 10.0.10.1/24 (subnet 255.255.255.0) - !! MATCH !!
[2020-07-08 15:37:28.882 11742M] SQL: Comparing 10.0.10.6 vs. 10.0.10.1/24 (subnet 255.255.255.0) - !! MATCH !!
[2020-07-08 15:37:28.882 11742M] SQL: Comparing 10.0.10.6 vs. 10.0.10.1/24 (subnet 255.255.255.0) - !! MATCH !!
[2020-07-08 15:37:28.883 11742M] Querying gravity database for client 10.0.10.6 (getting groups)
[2020-07-08 15:37:28.883 11742M] Compiled 1 whitelist and 25 blacklist regex filters for 13 clients in 39.4 msec
[2020-07-08 15:37:37.418 11761/F11742] TCP worker forked
[2020-07-08 15:37:37.418 11761/F11742] gravityDB_open(): Trying to open /etc/pihole/gravity.db in read-only mode
[2020-07-08 15:37:37.419 11761/F11742] gravityDB_open(): Setting location for temporary object to MEMORY
[2020-07-08 15:37:37.419 11761/F11742] gravityDB_open(): Preparing audit query
[2020-07-08 15:37:37.423 11761/F11742] gravityDB_open(): Setting busy timeout to 1000
[2020-07-08 15:37:37.423 11761/F11742] gravityDB_open(): Setting busy timeout to zero
[2020-07-08 15:37:37.423 11761/F11742] gravityDB_open(): Successfully opened gravity.db
[2020-07-08 15:37:37.423 11761/F11742] TCP worker terminating (client disconnected)
[2020-07-08 15:37:47.900 11742M] Initializing gravity statements for 10.0.10.136
[2020-07-08 15:37:47.901 11742M] Querying gravity database for client 10.0.10.136 (getting best match)
[2020-07-08 15:37:47.902 11742M] SQL: Comparing 10.0.10.136 vs. 10.0.1.0/16 (subnet 255.255.0.0) - !! MATCH !!
[2020-07-08 15:37:47.902 11742M] SQL: Comparing 10.0.10.136 vs. 10.0.1.0/16 (subnet 255.255.0.0) - !! MATCH !!
[2020-07-08 15:37:47.902 11742M] SQL: Comparing 10.0.10.136 vs. 10.0.10.1/24 (subnet 255.255.255.0) - !! MATCH !!
[2020-07-08 15:37:47.902 11742M] SQL: Comparing 10.0.10.136 vs. 10.0.10.1/24 (subnet 255.255.255.0) - !! MATCH !!
[2020-07-08 15:37:47.902 11742M] SQL: Comparing 10.0.10.136 vs. 10.0.10.1/24 (subnet 255.255.255.0) - !! MATCH !!
[2020-07-08 15:37:47.903 11742M] Querying gravity database for client 10.0.10.136 (getting groups)
[2020-07-08 15:37:47.903 11742M] gravityDB_open(): Preparing vw_whitelist statement for client 10.0.10.136
[2020-07-08 15:37:47.903 11742M] get_client_querystr: SELECT EXISTS(SELECT domain from vw_whitelist WHERE domain = ? AND group_id IN (0));
[2020-07-08 15:37:47.904 11742M] gravityDB_open(): Preparing vw_gravity statement for client 10.0.10.136
[2020-07-08 15:37:47.904 11742M] get_client_querystr: SELECT EXISTS(SELECT domain from vw_gravity WHERE domain = ? AND group_id IN (0));
[2020-07-08 15:37:47.905 11742M] gravityDB_open(): Preparing vw_blacklist statement for client 10.0.10.136
[2020-07-08 15:37:47.905 11742M] get_client_querystr: SELECT EXISTS(SELECT domain from vw_blacklist WHERE domain = ? AND group_id IN (0));
[2020-07-08 15:37:47.906 11742M] domain_in_list("discovery-v6.syncthing.net", 0xaaaaef9b53e8, whitelist): 0
[2020-07-08 15:37:47.907 11742M] domain_in_list("discovery-v6.syncthing.net", 0xaaaaef9b3c78, blacklist): 0
[2020-07-08 15:37:47.907 11742M] domain_in_list("discovery-v6.syncthing.net", 0xaaaaef9b37c8, gravity): 0
[2020-07-08 15:37:51.295 11742M] Received: Real-time signal 0 (34 -> 0)
[2020-07-08 15:37:51.296 11742M] dbquery: "DELETE FROM message;"
[2020-07-08 15:37:51.306 11742M]          ---> OK
[2020-07-08 15:37:51.307 11742M] gravityDB_open(): Trying to open /etc/pihole/gravity.db in read-only mode
[2020-07-08 15:37:51.307 11742M] gravityDB_open(): Setting location for temporary object to MEMORY
[2020-07-08 15:37:51.307 11742M] gravityDB_open(): Preparing audit query
[2020-07-08 15:37:51.310 11742M] gravityDB_open(): Setting busy timeout to 1000
[2020-07-08 15:37:51.310 11742M] gravityDB_open(): Setting busy timeout to zero
[2020-07-08 15:37:51.311 11742M] gravityDB_open(): Successfully opened gravity.db
[2020-07-08 15:37:51.311 11742M] Querying count of distinct domains in gravity database table vw_gravity: SELECT value FROM info WHERE property = 'gravity_count';
[2020-07-08 15:37:51.312 11742M] Querying count of distinct domains in gravity database table vw_regex_blacklist: SELECT COUNT(DISTINCT domain) FROM vw_regex_blacklist
[2020-07-08 15:37:51.314 11742M] INFO: No regex blacklist entries found
[2020-07-08 15:37:51.314 11742M] Querying count of distinct domains in gravity database table vw_regex_whitelist: SELECT COUNT(DISTINCT domain) FROM vw_regex_whitelist
[2020-07-08 15:37:51.314 11742M] INFO: No regex whitelist entries found
[2020-07-08 15:37:51.315 11742M] Querying gravity database for client 10.0.1.1 (getting best match)
[2020-07-08 15:37:51.315 11742M] SQL: Comparing 10.0.1.1 vs. 10.0.1.0/16 (subnet 255.255.0.0) - !! MATCH !!
[2020-07-08 15:37:51.315 11742M] SQL: Comparing 10.0.1.1 vs. 10.0.1.0/16 (subnet 255.255.0.0) - !! MATCH !!
[2020-07-08 15:37:51.315 11742M] SQL: Comparing 10.0.1.1 vs. 10.0.10.1/24 (subnet 255.255.255.0) - NO MATCH
[2020-07-08 15:37:51.315 11742M] SQL: Comparing 10.0.1.1 vs. 10.0.1.0/16 (subnet 255.255.0.0) - !! MATCH !!
[2020-07-08 15:37:51.316 11742M] Querying gravity database for client 10.0.1.1 (getting groups)
[2020-07-08 15:37:51.317 11742M] Querying gravity database for client 10.0.99.192 (getting best match)
[2020-07-08 15:37:51.317 11742M] SQL: Comparing 10.0.99.192 vs. 10.0.1.0/16 (subnet 255.255.0.0) - !! MATCH !!
[2020-07-08 15:37:51.317 11742M] SQL: Comparing 10.0.99.192 vs. 10.0.1.0/16 (subnet 255.255.0.0) - !! MATCH !!
[2020-07-08 15:37:51.317 11742M] SQL: Comparing 10.0.99.192 vs. 10.0.10.1/24 (subnet 255.255.255.0) - NO MATCH
[2020-07-08 15:37:51.318 11742M] SQL: Comparing 10.0.99.192 vs. 10.0.1.0/16 (subnet 255.255.0.0) - !! MATCH !!
[2020-07-08 15:37:51.318 11742M] Querying gravity database for client 10.0.99.192 (getting groups)
[2020-07-08 15:37:51.319 11742M] Querying gravity database for client 10.0.40.3 (getting best match)
[2020-07-08 15:37:51.319 11742M] SQL: Comparing 10.0.40.3 vs. 10.0.1.0/16 (subnet 255.255.0.0) - !! MATCH !!
[2020-07-08 15:37:51.319 11742M] SQL: Comparing 10.0.40.3 vs. 10.0.1.0/16 (subnet 255.255.0.0) - !! MATCH !!
[2020-07-08 15:37:51.319 11742M] SQL: Comparing 10.0.40.3 vs. 10.0.10.1/24 (subnet 255.255.255.0) - NO MATCH
[2020-07-08 15:37:51.319 11742M] SQL: Comparing 10.0.40.3 vs. 10.0.1.0/16 (subnet 255.255.0.0) - !! MATCH !!
[2020-07-08 15:37:51.320 11742M] Querying gravity database for client 10.0.40.3 (getting groups)
[2020-07-08 15:37:51.321 11742M] Querying gravity database for client 10.0.10.64 (getting best match)
[2020-07-08 15:37:51.321 11742M] SQL: Comparing 10.0.10.64 vs. 10.0.1.0/16 (subnet 255.255.0.0) - !! MATCH !!
[2020-07-08 15:37:51.321 11742M] SQL: Comparing 10.0.10.64 vs. 10.0.1.0/16 (subnet 255.255.0.0) - !! MATCH !!
[2020-07-08 15:37:51.321 11742M] SQL: Comparing 10.0.10.64 vs. 10.0.10.1/24 (subnet 255.255.255.0) - !! MATCH !!
[2020-07-08 15:37:51.322 11742M] SQL: Comparing 10.0.10.64 vs. 10.0.10.1/24 (subnet 255.255.255.0) - !! MATCH !!
[2020-07-08 15:37:51.322 11742M] SQL: Comparing 10.0.10.64 vs. 10.0.10.1/24 (subnet 255.255.255.0) - !! MATCH !!
[2020-07-08 15:37:51.322 11742M] Querying gravity database for client 10.0.10.64 (getting groups)
[2020-07-08 15:37:51.323 11742M] Querying gravity database for client 10.0.10.136 (getting best match)
[2020-07-08 15:37:51.330 11742M] SQL: Comparing 10.0.10.136 vs. 10.0.1.0/16 (subnet 255.255.0.0) - !! MATCH !!
[2020-07-08 15:37:51.331 11742M] SQL: Comparing 10.0.10.136 vs. 10.0.1.0/16 (subnet 255.255.0.0) - !! MATCH !!
[2020-07-08 15:37:51.331 11742M] SQL: Comparing 10.0.10.136 vs. 10.0.10.1/24 (subnet 255.255.255.0) - !! MATCH !!
[2020-07-08 15:37:51.331 11742M] SQL: Comparing 10.0.10.136 vs. 10.0.10.1/24 (subnet 255.255.255.0) - !! MATCH !!
[2020-07-08 15:37:51.331 11742M] SQL: Comparing 10.0.10.136 vs. 10.0.10.1/24 (subnet 255.255.255.0) - !! MATCH !!
[2020-07-08 15:37:51.331 11742M] Querying gravity database for client 10.0.10.136 (getting groups)
[2020-07-08 15:37:51.332 11742M] Querying gravity database for client 10.0.99.33 (getting best match)
[2020-07-08 15:37:51.332 11742M] SQL: Comparing 10.0.99.33 vs. 10.0.1.0/16 (subnet 255.255.0.0) - !! MATCH !!
[2020-07-08 15:37:51.333 11742M] SQL: Comparing 10.0.99.33 vs. 10.0.1.0/16 (subnet 255.255.0.0) - !! MATCH !!
[2020-07-08 15:37:51.333 11742M] SQL: Comparing 10.0.99.33 vs. 10.0.10.1/24 (subnet 255.255.255.0) - NO MATCH
[2020-07-08 15:37:51.333 11742M] SQL: Comparing 10.0.99.33 vs. 10.0.1.0/16 (subnet 255.255.0.0) - !! MATCH !!
[2020-07-08 15:37:51.333 11742M] Querying gravity database for client 10.0.99.33 (getting groups)
[2020-07-08 15:37:51.334 11742M] Querying gravity database for client 10.0.10.84 (getting best match)
[2020-07-08 15:37:51.334 11742M] SQL: Comparing 10.0.10.84 vs. 10.0.1.0/16 (subnet 255.255.0.0) - !! MATCH !!
[2020-07-08 15:37:51.334 11742M] SQL: Comparing 10.0.10.84 vs. 10.0.1.0/16 (subnet 255.255.0.0) - !! MATCH !!
[2020-07-08 15:37:51.334 11742M] SQL: Comparing 10.0.10.84 vs. 10.0.10.1/24 (subnet 255.255.255.0) - !! MATCH !!
[2020-07-08 15:37:51.335 11742M] SQL: Comparing 10.0.10.84 vs. 10.0.10.1/24 (subnet 255.255.255.0) - !! MATCH !!
[2020-07-08 15:37:51.335 11742M] SQL: Comparing 10.0.10.84 vs. 10.0.10.1/24 (subnet 255.255.255.0) - !! MATCH !!
[2020-07-08 15:37:51.335 11742M] Querying gravity database for client 10.0.10.84 (getting groups)
[2020-07-08 15:37:51.336 11742M] Querying gravity database for client 10.0.10.190 (getting best match)
[2020-07-08 15:37:51.336 11742M] SQL: Comparing 10.0.10.190 vs. 10.0.1.0/16 (subnet 255.255.0.0) - !! MATCH !!
[2020-07-08 15:37:51.336 11742M] SQL: Comparing 10.0.10.190 vs. 10.0.1.0/16 (subnet 255.255.0.0) - !! MATCH !!
[2020-07-08 15:37:51.336 11742M] SQL: Comparing 10.0.10.190 vs. 10.0.10.1/24 (subnet 255.255.255.0) - !! MATCH !!
[2020-07-08 15:37:51.336 11742M] SQL: Comparing 10.0.10.190 vs. 10.0.10.1/24 (subnet 255.255.255.0) - !! MATCH !!
[2020-07-08 15:37:51.337 11742M] SQL: Comparing 10.0.10.190 vs. 10.0.10.1/24 (subnet 255.255.255.0) - !! MATCH !!
[2020-07-08 15:37:51.337 11742M] Querying gravity database for client 10.0.10.190 (getting groups)
[2020-07-08 15:37:51.338 11742M] Querying gravity database for client 127.0.0.1 (getting best match)
[2020-07-08 15:37:51.338 11742M] SQL: Comparing 127.0.0.1 vs. 10.0.1.0/16 (subnet 255.255.0.0) - NO MATCH
[2020-07-08 15:37:51.338 11742M] SQL: Comparing 127.0.0.1 vs. 10.0.10.1/24 (subnet 255.255.255.0) - NO MATCH
[2020-07-08 15:37:51.339 11742M] Querying gravity database for client 10.0.1.2 (getting best match)
[2020-07-08 15:37:51.339 11742M] SQL: Comparing 10.0.1.2 vs. 10.0.1.0/16 (subnet 255.255.0.0) - !! MATCH !!
[2020-07-08 15:37:51.339 11742M] SQL: Comparing 10.0.1.2 vs. 10.0.1.0/16 (subnet 255.255.0.0) - !! MATCH !!
[2020-07-08 15:37:51.339 11742M] SQL: Comparing 10.0.1.2 vs. 10.0.10.1/24 (subnet 255.255.255.0) - NO MATCH
[2020-07-08 15:37:51.340 11742M] SQL: Comparing 10.0.1.2 vs. 10.0.1.0/16 (subnet 255.255.0.0) - !! MATCH !!
[2020-07-08 15:37:51.340 11742M] Querying gravity database for client 10.0.1.2 (getting groups)
[2020-07-08 15:37:51.341 11742M] Querying gravity database for client 10.0.1.4 (getting best match)
[2020-07-08 15:37:51.341 11742M] SQL: Comparing 10.0.1.4 vs. 10.0.1.0/16 (subnet 255.255.0.0) - !! MATCH !!
[2020-07-08 15:37:51.341 11742M] SQL: Comparing 10.0.1.4 vs. 10.0.1.0/16 (subnet 255.255.0.0) - !! MATCH !!
[2020-07-08 15:37:51.341 11742M] SQL: Comparing 10.0.1.4 vs. 10.0.10.1/24 (subnet 255.255.255.0) - NO MATCH
[2020-07-08 15:37:51.341 11742M] SQL: Comparing 10.0.1.4 vs. 10.0.1.0/16 (subnet 255.255.0.0) - !! MATCH !!
[2020-07-08 15:37:51.342 11742M] Querying gravity database for client 10.0.1.4 (getting groups)
[2020-07-08 15:37:51.343 11742M] Querying gravity database for client 10.0.1.3 (getting best match)
[2020-07-08 15:37:51.343 11742M] SQL: Comparing 10.0.1.3 vs. 10.0.1.0/16 (subnet 255.255.0.0) - !! MATCH !!
[2020-07-08 15:37:51.343 11742M] SQL: Comparing 10.0.1.3 vs. 10.0.1.0/16 (subnet 255.255.0.0) - !! MATCH !!
[2020-07-08 15:37:51.343 11742M] SQL: Comparing 10.0.1.3 vs. 10.0.10.1/24 (subnet 255.255.255.0) - NO MATCH
[2020-07-08 15:37:51.343 11742M] SQL: Comparing 10.0.1.3 vs. 10.0.1.0/16 (subnet 255.255.0.0) - !! MATCH !!
[2020-07-08 15:37:51.343 11742M] Querying gravity database for client 10.0.1.3 (getting groups)
[2020-07-08 15:37:51.344 11742M] Querying gravity database for client 10.0.10.6 (getting best match)
[2020-07-08 15:37:51.345 11742M] SQL: Comparing 10.0.10.6 vs. 10.0.1.0/16 (subnet 255.255.0.0) - !! MATCH !!
[2020-07-08 15:37:51.345 11742M] SQL: Comparing 10.0.10.6 vs. 10.0.1.0/16 (subnet 255.255.0.0) - !! MATCH !!
[2020-07-08 15:37:51.345 11742M] SQL: Comparing 10.0.10.6 vs. 10.0.10.1/24 (subnet 255.255.255.0) - !! MATCH !!
[2020-07-08 15:37:51.345 11742M] SQL: Comparing 10.0.10.6 vs. 10.0.10.1/24 (subnet 255.255.255.0) - !! MATCH !!
[2020-07-08 15:37:51.345 11742M] SQL: Comparing 10.0.10.6 vs. 10.0.10.1/24 (subnet 255.255.255.0) - !! MATCH !!
[2020-07-08 15:37:51.345 11742M] Querying gravity database for client 10.0.10.6 (getting groups)
[2020-07-08 15:37:51.346 11742M] Compiled 0 whitelist and 0 blacklist regex filters for 13 clients in 33.9 msec

Coro · July 8, 2020, 2:10pm

It doesn't look like there is a functional change in here but the code is more readable. This should be merged @DL6ER

https://github.com/pi-hole/FTL/compare/release/v5.1...tweak/gravity_counting_logging

(but I'm not a C expert)

Why are they wrong? When you

then, to my understanding, there are no enabled regexes any longer so their count is zero.

sqlite3 /etc/pihole/gravity.db "SELECT COUNT(DISTINCT domain) FROM vw_regex_blacklist;"
sqlite3 /etc/pihole/gravity.db "SELECT COUNT(DISTINCT domain) FROM vw_regex_whitelist;"

What does this tell you in both cases?

yubiuser · July 8, 2020, 2:22pm

I know, but switching "back" to new/tre-regex I can immediately reproduce it.

You're right. I changed my reply above.

yubiuser · July 8, 2020, 2:36pm

Still on new/tre-regex.

It doesn't stop if I do the following.

disable default group
pihole restartdns
enable default group

Coro · July 8, 2020, 2:43pm

First logging then counting (line 659|664 is hidden). This order is not changed by this commit.

@yubiuser Can you compare against switching to release/v5.1 ? As you can see from my comparison link, Github tells us that they based their change on this branch. new/tre-regex may be older code if the development code has not been merged into it recently so the change may even be somewhere else and we're not seeing this here.