Blocked sites without any info in the log

[315]

Expected Behaviour:

A website (or a part of a website) is being blocked, which I'd like not to be blocked.
I could see the website address as blocked in the Query Log.
Later I could whitelist or delete from blacklist.

Actual Behaviour:

A website (or a part of a website) is being blocked, which I'd like not to be blocked.
I cannot see any blocked website at that time in the Query Log.
When I Permanently Disable Pihole the website is loading normally.
When I Enable Pihole again the website is STILL loading normally. --> makes no sense
At a later time the website is being blocked again.

Debug Token:

No actual problem. It happens infrequent. Its difficult to reproduce this behaviour.

Thats why I would like to know, what I can try or activate to more easily reproduce this next time.

[jfb]

@315

When I Permanently Disable Pihole the website is loading normally.
When I Enable Pihole again the website is STILL loading normally. --> makes no sense

When the browser gets that IP when Pi-Hole is disabled, the IP has a finite TTL (time to live). Until that TTL expires, the browser does not have to look up that IP again. After it does expire, then the browser asks Pi-Hole for the IP and the domain is now blocked by Pi-Hole.

How do I determine what domain an ad is coming from?

Block ads or fix broken sites This FAQ is written in the context of finding out how to block an ad that isn't blocked automatically, but the same steps can be used to troubleshoot issues with sites that don't function properly. An example is that sometimes certain payment gateways try to collect metrics on the sale, but these domains are blocked by Pi-hole, so the payment cannot be processed. Also, some links send you through several referrers and if one of them is blocked, you may land on a …

[315]

I read the FAQ, but did not see how it can help me, solve the problem.

I have clicked a site in a search engine that is not loading.

In the pihole.log file the DNS is resolved correctly.
The Query Log only shows this site is not blocked/it is showing green.

How do I find out why this site is not loading?

When I disable Blocking it is loading.

How do I find out how long the TTL is?

[pisome]

Sounds U'll be a "victim" of CNAME-blocking. When a DNS-request is answered with a CNAME (another "via" domain), then your Browser has to resolve it too. If this CNAME is blocked by Pihole, you cannot see this action at this time. This will be a feature in the next version of Pihole.
But CNAME blocking can be disabled via

CNAME_DEEP_INSPECT=false

in pihole-FTL.conf
Once FTL is restartet, the hidden CNAME blocking is gone.

[jfb]

@pisome

If this CNAME is blocked by Pihole, you cannot see this action at this time.

There is no CNAME blocking in Pi-Hole V4.

[DanSchaper]

@pisome

Sounds U’ll be a “victim” of CNAME-blocking.

Just to note, you'll need to learn the new CNAME process if you want to block in the future. Many sites are moving to that trick.

[315]

Now I’m confused.

One says the problem is the CNAME blocking. The other says there is NO CNAME blocking in my version.

What it right?

[deHakkelaar]

https://pi-hole.net/2020/01/19/announcing-a-beta-test-of-pi-hole-5-0/#page-content

[DanSchaper]

What does pihole -v show as the version? If you are not on the beta for 5 then you don't have CNAME blocking.

[315]

PiHole 4.3.2
FTL 4.3.1

Could the beta be a solution?

[315]

I think it could be a problem with my regex blockings, but how can I find out which of them is the problem?

This one?

.*\.g[0-9]+\..*

[315]

Now its getting more incomprehensible:

Yesterday I was so annoyed thats so many sites are not loading and for example the Threema App on iOS could not connect, that I disabled the pihole.

But today Threema App in local network still does NOT connect.

When I use Mobile Data Threema IS connecting.
So must be something in my local network.

The router is not configured to blocking anything.
In the router the Pihole is configured as the upstream DNS (hope thats the right name for it; Internet > DNS-Server).

AND the DHCP is giving all clients the Pihole as local DNS.

In Pihole I use these upstream DNS:

• 46.182.19.48#53
• 194.150.168.168#53
• 2a02:2970:1002::18#53
• 2001:4f8:0:2::14#53

Is there any problem? Are these still used when pihole is disabled?

[315]

It looks like only one device has a problem: iPhone
When I configure a different DNS Server, it works.
When the default DNS Server (Pihole) is configured, its not working.

But why should the Pihole block my iPhone? Can I check it somewhere in the settings?

[315]

Ok, I don't get the problem. So another client for example cannot access paypal.com. In the Query Log its shown "green" as "OK (forwarded) / INSECURE / CNAME (132777.9ms)".

But why is the site not loading? Keeps white...

PS. Do I do something wrong, or why is nobody answering? Do I miss some information? I can give these, when somebody tells me what is needed.

[Bucking_Horn]

@315

Do I do something wrong, or why is nobody answering?

I guess it's just that the information you've provided so far is way too unspecific or vague to allow for any serious recommendations.
So you are getting either partial answers for certain circumstances of your observations that are clearly definable (as e.g. @jfb's explanation of how TTLs cause websites to show ads even when Pi-hole is re-enabled) or just wide guesses at less related simliarities to some future features.

Let me have a try at another, a bit more elaborated guess by taking your most recent post into account, and by taking the discussion back to your starting point: Your topic's heading - Blocked sites without any info in the log.

Now, Pi-hole wouldn't block anything without letting you know.
As you don't see anything blocked in the logs, Pi-hole doesn't block it.

Indeed, you further correlate DNS queries that Pi-hole is showing as successful with pages that do not load.

@315

cannot access paypal.com. In the Query Log its shown “green” as “OK (forwarded) / INSECURE / CNAME (132777.9ms)”.

In that most recent post from just a few minutes ago, you provide evidence of two additional noticeable facts:

1. a DNS query (related to a page not loading) takes an unusually long time to finish - well over two minutes.
2. you are using DNSSEC

If you happen to run Pi-hole on an RPi, using DNSSEC might be the reason for 1) to take so long:
RPis are lacking a real-time clock (aka RTC) used to keep track of a precise time, so they have to re-sync with internet time servers way more frequently. As DNSSEC relies on exact timings, a resync hitting your RPi will also delay resolution of hostnames for that time, or may even prevent sucessful resolution completely.

This failed or delayed resolution may in turn cause your browser to give up on waiting for a hostname to resolve or just waiting endlessly, resulting either in a time-out or some (significant) fraction of your page growing stale.

This explanation would fit in nicely with your inability to put a certain time or a certain domain to your problem that would guarantee reproducabilty of your observations.

On the other hand, it doesn't fit quite well with you still observing strange behaviour even when Pi-hole is switched off completely for a longer period.

The best approach to further isolate your problem would possibly be to
a) forego using DNSSEC for a while and see whether you still experience problems
b) manually configure your problem client (iPhone) to bypass Pi-hole to see whether its divergent behaviour is at all related to Pi-hole

I'd leave it up to you how to best combine or separate these two options

In case you'd need further assistance in dealing with your findings, it'd be probably helpful if you could post your debug token here and provide any interesting additional information about your network configuration.

[315]

Hi Bucking_Horn,

yes I see, thanks for explaining. The reason I didn't post the debug log was, that every time I tried to reproduce a problem, it was not possible and next time working...

Just before your post I already disabled DNSSEC, becuase of some additional google work.
I think that solved most of the problems.

Will see how it works the next days without another external DNS on the client (iPhone).

Thank you!

What still confuses me:
Activating DNSSEC resulted in different behaviour on different device with same websites. Is this normal?

[DanSchaper]

@315

Activating DNSSEC resulted in different behaviour on different device with same websites. Is this normal?

A better way to phrase the question (and to get a developer to help) would be to include:

1. Show us the different behavior of each device
2. What the different devices are and what OS they are running
3. A URL for us to look at and compare to.

If we have to spend time inferring what may be happening or ask a number of questions then our response gets pushed off to when we have time to invest. Right now we're involved in a beta so we have less time than usual.

[315]

Ok thank you.

As DNSSEC is now disabled, I cannot answer these questions anymore.
But next time I will do like this.

Could also DNSSEC still be the problem when Pihole blocking is disabled?
Would make sense then, that there was nearly no effect when I disabled blocking...

[Bucking_Horn]

@315

Could also DNSSEC still be the problem when Pihole blocking is disabled?

Yes.
If you just disable Pi-hole's blocking (as opposed to shutting it down), DNS requests would still be exposed to the same possible penalties by passing through DNSSEC on your RPi.

@315

What still confuses me:
Activating DNSSEC resulted in different behaviour on different device with same websites. Is this normal?

Well, no, not normal, in the sense that it can't be attributed to just employing DNSSEC.
But to be expected in your case.

Time syncs would appear to occur randomly when trying to correlate them with actual internet usage, so delays or failures would affect some random device acessing some random website, but likely happening in defined chronological intervals - and it's always difficult to diagnose (seemingly) random behaviour.

If you have been able to confirm it is indeed DNSSEC causing your troubles, you could try to minimise chances for delays:

a) use a time server that provides reliable and fast resolution for the area of the world where you are located - read more at the NTP pool project (but still exercise due care in picking the optimal time server, which could as well be your ISP's).

b) If your router is capable of it, configure it to advertise itself as a time server in your network, and use your router as your RPi's time server, while making your router use the best time servers for your region.

You can supply a time server on your RPi by editing /etc/systemd/timesyncd.conf (at least when running Raspbian Stretch or Buster).