DNS lookups blocked but not logged

Since upgrading pihole, I found there are periodically websites being blocked, but the dns lookup is not logged by pihole as either being blocked or forwarded. The latest one I whilelisted was:

(.|^)orders-prestigeportraits.com$

Prior to white listing, I would click on the link to view my order from the portrait studio and my web-browser would display an ip address not found. After whitelisting, the link works normally.

But here is the thing, when I search the lookup logs, I don't see the domain as being queried at all.

My guess is this has something to do with 5.0's handling of CNAME records, and it is being logged as a different domain name. But that is strictly a guess on my part.

Is there a way to turn off the CNAME filtering? Then I could quickly learn if this is the issue.

Yes, see this section of the documentation: https://docs.pi-hole.net/ftldns/configfile/

This should not be the case. All received queries should be logged in the query log (consistent with your privacy settings), and if a CNAME blocked it that will be clearly indicated in the query log.

Here is the fun part.

$ grep dartsearch.net /var/log/pihole.log
May 23 18:20:14 dnsmasq[940]: query[A] clickserve.dartsearch.net from 172.31.252 .1 
May 23 18:20:14 dnsmasq[940]: gravity blocked clickserve.dartsearch.net is 0.0.0 .0

So it is actually logged. But when I search for it in the Recent Queries page, there are no results.

It looks like this is a UI bug with the way the log search is implemented, not with actually physically logging this to disk.

This is the exact command you ran? That file should not exist.

The domain is on one of the stock blocklists:

pihole -q clickserve.dartsearch.net
Match found in **https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts** :
clickserve.dartsearch.net
dig clickserve.dartsearch.net +short
0.0.0.0

I see the problem is just a confusing UI:

### Recent Queries (showing up to 100 queries), [show all]

Search:

Show  10 entries

One naturally expects that when you enter a search that means it will stop the search after the most recent 100 matches, and then show the first ten on the current page. But instead it means it will only search the most recent 100 matches. Which pretty much means unless I hit the search in less than 30 seconds after the query, the search will never return a hit.

Instead it seems you have to press "show all". Then after a significant delay the log file is scanned, at that point the search will act as a filter, not a search of all the entries currently listed.

Click "show all" at the top, then select "all" in the pull down menu. Then search for clickserver.dartsearch.net and post a screen snap of what shows up:

No it is not. I actually ran the command in the /var/log directory and then incorrectly added the /var/log into the copy and past text in front of the wrong argument.

The original problem statement was that DNS lookups were blocked but not logged. Is that still the case? Or is the problem that the query log does not behave as you expected and it took more steps to see the blocked query?

Yep. That doesn't surprise me. Blocking that link keeps me from being able to open:

from a google search on google play store.

The one I started this thread with was:

orders-prestigeportraits.com

Searching the full log I see it is blocked by:

pihole.log.3.gz:May 20 16:28:51 dnsmasq[940]: reply click.orders-prestigeportraits.com is

Which is unfortunately, as that is the site for my son's school photos. It looks like the cname is for click.virt.exacttarget.com. Googling that I see exact target is a Salesforce Marketing Cloud tool. Which means we should reasonably expect many fortunate 500 companies will be using this service. Mostly for you to access legit products you purchased, and relevant marketing lists you explicitly signed up for.

While I'm certain there are a few spammers that also use exacttarget.com, it would be a violation of exacttarget's and salesforce licensing agreements. And most spammers aren't going to pay the huge licensing fees for salesforce and ontop of that for exacttarget.com just to have their whole service shut down for a stupid license violation.

Whitelist it and the domain will load. A whitelist entry will also cancel any further CNAME checking for blocks for that domain. Same for the portraits domain.

Yes. Exactly. Except the goal is not to have every user need to whitelist hundreds of websites just to use websites normally. Just as we shouldn't have to blacklist hundreds of websites to block unwanted advertisements.

These features are intended simply as one-off exceptions. So there is any easy way to discover if a listing is incorrect. If a block prevents normal usage of a website clearly that is a bad black to include in the standard lists. So my question again is what is the correct way to report those incorrect entries? Is there a wiki like sight, such as the way we submit corrections and updates to thetvdb entries?

If you are having to whitelist hundreds of domains (not websites) just to use websites normally, this is out of the ordinary, particularly with the stock blocklists.

Go to the page that serves the list you are using, and look for contact information. As an example, Steven Black has a Github page:

If you go to the Wally3K collection (https://firebog.net), and select the link titled "toggle list maintainer sources", the list of blocklists toggles and now includes a link to the maintainers sites.

I was not able to duplicate that. Opening play.google.store in my browser with the stock blocklists (and a few added regex) was not a problem. The following domains were requested when the page was loaded:

* play.google.com
* www.gstatic.com
* lh3.google.com
* fonts.gstatic.com
* apis.google.com
* lh3.googleusercontent.com
* www.google.com
* www.google-analytics.com
* ssl.gstatic.com
* ogs.google.com
* ajax.googleapis.com

Please post the token generated by pihole -d

or do it through the Web interface:

Tools > Generate Debug Log

I can take a look at your blocklists and see if there is anything there that would account for you not being able to load this domain.