Can you take a screenshot of the "Timing" tab of the request?
It looks like blocked queries are not being rejected. What about if you go directly to one of the blocked queries?
If I copy the URL of one of the scripts that takes a while to fetch, then paste it into a new tab, then it takes a while and eventually displays a "Problem loading page" message, so a 404. Seems to me it takes quite some time to wait for the blocked page before giving up with a 404.
Is that via HTTPS? How about HTTP?
That's via HTTP. I read on another forum post about HTTPS issues and already configured iptables to REJECT requests on 443
Here's an example of one of the URLs from the list of long queries that I tried opening separately:
I get an immediate ERR_CONNECTION_REFUSED, as it is HTTPS.
Weird, when I pasted that URL in it was not HTTPS. I'll try and paste it again but add some spaces so it's not converted:
http: // pubads.g.doubleclick.net/
I get the block page immediately.
I went into my DNS settings on my computer, and found two entries. One is the IPv4 address of my router and the other is the IPv6 address of my router. If I manually add the IPv4 address of my Pi Hole machine, then everything loads much quicker.
In my router (a FRITZ!Box 6490) I had manually entered the DNS server to be the IPv4 and IPv6 of the Pi Hole device.
Interestingly I can't ping pi.hole from another machine. Is that relevant?
Yes, that means the machine is not using Pi-hole (or is trying to use Pi-hole alongside another DNS server).
Aha! I've cracked it. The FRITZ!Box is a pain to configure... For the benefit of others, I had specified the DNS Server under Internet -> Account Information -> DNS Server but that's not correct. It's necessary to go into Home Network -> Home Network Overview -> Network Settings -> IP4/6 Addresses then enter the IP address of the Pi Hole device under "Local DNS server".
Thanks for your fast help though, Mcat12! It helped me at least narrow down the problem.
I experienced the same issue:
--Slow loading of some websites
--Only from mobile and not from desktop
--On mobile only from browser and not from apps
I used tcpdump to gather a full pageload of the troublesome site from mobile and saw that the mobile device stalls for a minute a couple of times on CDN's.
I used tcpdump to gather the same pageload from desktop and did not see the traffic stalling. Page loaded in 2 secs instead of three minutes.
When comparing the two dumps I see that the traffic for mobile is different from the traffic from desktop. Different CDN's are used as well as the mobile using proxies to supposedly speed up pageloads. Traffic from proxy to CDN seems to be stalling the loads.
I also noticed that Pi Hole Always responded within milliseconds so is not the problem.
I will do further investigation and report back when I know more..
Some further results:
-Some websites load quick, others take minutes.
-Loading the 'slow' websites via 4G loads them quickly.
-Loading the 'slow' websites when PI Hole is disabled loads them quickly.
The above combined with the data from earlier post lets me assume that somewhere in the chain of CDN's a request for ads is blocked by pi-hole and is disrupting this chain of events. This will lead any of the CDN's to wait for a time-out. This causes the slowness.
Next thing I will do is whitelist all the blocked requests for a certain 'slow' page to prove the point...
Will report back..
Whitelisting does not help for this case. The problem is still that a site (google analytics in my case) is taking a minute to respond back, thus holding up the rest of the traffic. No clue why this only happens with mobile requests...
BTW: I tried to ask for the desktop page in the browser but that does not solve the issue...
I did some additional investigation.
I found that the problem is caused, in my case, by the firewall blocking all traffic to the DNS server (a.k.a. Pi-Hole) except for UDP port 53 (DNS port).
What happens is that, for the sites that take a long time to load (fully), the following happens:
The client sent a DNS request. Pi-Hole responds within milli seconds.
Then the client starts firing TCP requests to port 80 of the DNS server. This is blocked by the firewall. The client is waiting for the ACK message. This, of course, never comes. Then after a while the request times out and is resubmitted. After a few resubmissions the next requests are sent to the content provider (using the IP's provided by Pi-Hole). And then the page is loading further.
When I opened the firewall, the same page that took >3 mins to load is suddenly loading in 4 secs with Pi-Hole enabled.
Conclusion:
Some websites need port 80 on the DNS server open to acknowledge requests sent.
The above was, alas, not the whole story....There is more:
I have done some additional research as to why some websites are not loading or not loading fast. Also, why do I need to open TCP port 80/443 to my DNS server (Pi-Hole) for other webistes to work. I think I know why this is. I do, however, not know how to solve it.
What happens:
When a website loading on a client is asking the DNS for the IP Address of a blocked domain my Pi-Hole gives return the DNS server IP Address. See wireshark example below (my DNS server has IP 1.1.1.1):
z-na.amazon-adsystem.com: type A, class IN, addr 1.1.1.1
Subsequently, my browser is going to launch all sorts of requests to port 80/443 (depending on the visiting website config) but, of course, directed to my DNS server. See example below (1.2.3.4 is my phone):
Internet Protocol Version 4, Src: 1.2.3.4, Dst: 1.1.1.1
Transmission Control Protocol, Src Port: 58531 (58531), Dst Port: http (80), Seq: 1, Ack: 1, Len: 496
GET /widgets/onejs?MarketPlace=US&adInstanceId=940dd04f-8090-4fa9-b381-46473d341c2c HTTP/1.1\r\n
Of course the DNS server does not know what to do with this request so what happens is that, again dependent on the visting site, the browser will go into endless re-transmissions. At some point the better designed webistes will time out. Others will wait forever and the webpage will not load.
The potential solution for this would be to have the DNS server return not its own IP address but 0.0.0.0.. This will lead to the immediate 'blackholing' of all further requests without retransmissions. My problem is: I do not know what to do to make that happen. It must be a setting in the /etc/dnsmasq.d/01-pi-hole.conf but which one???
Who can help with advice?
Pi-hole installs lighttpd to provide the block page and a response to queries on port 80. This allows us to show a blocked page when someone visits a blocked domain over HTTP. This is preferable over 0.0.0.0 because that setting will result in errors in place of ads, instead of an empty element when using Pi-hole's block page.
See here for the HTTPS (port 443) issue: