Why Some Pages Load Slow When Using Pi-hole And How To Fix It (for versions < v4.0)

Originally published at: Why Some Pages Load Slow When Using Pi-hole And How To Fix It (for versions < v4.0) - Pi-hole

Update: 2018-08-06

As of v4.0, this should no longer be a problem unless you choose to deviate from the new default blocking mode and return to the IP-based mode.

Update: 2018-05-26

This problem is far less likely to happen when using NXDOMAIN or NULL blocking:

Pi-hole can make your network run faster, however, there are certain situations where Webpages will take a very long time to load (10-60 seconds or more). There are several reasons behind this--most of which can be remedied. This post explain the technical reasons why you may experience slow load times and provides solutions for them.

Understanding How Ads Are Delivered And How Pi-hole Blocks Them

There are several technologies and protocols that are used to deliver content (and consequently advertisements) to your Web browser or other devices. In it's simplest form, a DNS query is made for a domain, a TCP connection is established over port 80, and then an advertisement image (or other asset) is delivered via HTTP.

Pi-hole works by intercepting the DNS query and checking it against a list of known-advertisers. If a match is found, the request is sent to the Pi-hole instead of the server where the ad is actually hosted. In place of the ad, Pi-hole returns a 404 response, indicating there is no file, thus no ad.

Intercepting The Ad

It's this interception of the DNS query, which enables Pi-hole to make your network faster. Since your computer no longer needs to look up where the ad-serving domain is and then download the file from it, the request is caught before it leaves your network; your computer never goes out to the Internet to download the ad, but instead it simply understands that the ad that was requested does not exist, so there is nothing to display.

Other Technologies

But advertisers have caught on and are using different protocols and ports to deliver their advertisements. In part, they would probably make these changes at some point anyway because they offer some advantages over the typical HTTP advertisement. But since ad blockers have risen in popularity, it's likely driven a quicker adoption.

IPv6

Most networks these days support IPv4 and IPv6. If an advertisement fails to deliver over IPv4, it can still be delivered via IPv6. Pi-hole can already use both protocols to block advertisements, but IPv6 is more complex so this is the first place you might run into issues with slowness when using it.

Fixing IPv6 Issues That Cause Slow-Loading Pages

Without going into too much detail, IPv6 has several different types of addresses and your computer likely already has more than one assigned to it. To fix slowness, you'll want to enable Unique Local Addresses (ULA), which is the IPv4 equivalent of an address in the private IP range (i.e 10.x.x.x, 172.16.x.x, 192.168.0.x).

You can typically find a setting in your router to enable ULAs or your network might already be using them. If the IPv6 assigned to your Pi-hole is in the fc00::/7 block, this is a ULA.

Once enabled, add this IP to your setupVars.conf and run pihole -g to apply it.
The main reason using an ULA improves speed is because they will not change, which is good because Pi-hole needs a static address to function properly.

Some ISPs do not hand out static IPv6 addresses. So if you configured Pi-hole with an IPv6 address during installation and that address is later changed by your ISP, you now run into the problem the wrong (i.e. an invalid) IPv6 address in gravity.list. Because Pi-hole looks to this file to find out where to go, your computer cannot find that IP address and the requests time out, causing the long loading times.

ULA has simply the advantage that it is defined purely local (not routable over the Internet) and as such does not change regardless of how the ISP might be having fun with the IPv6 prefix.

HTTPS

Many ads are now being delivered over HTTPS and port 443, the encrypted version of HTTP. This encryption requires the use of a digital certificate to verify you are connecting to the correct server. While the IPv6 fix for slowness was straightforward, the solution with HTTPS is a bit more complex.

Since Pi-hole only knows about the domain being requested and not the protocols being used to access it, Pi-hole will intercept HTTPS advertisements and since the Pi-hole does not have the certificate of the actual server being queried, you may see slow loading times waiting for the request to time out.

Although it is possible to install a self-signed certificate, it's not something we do by default. When doing this, you are creating a Man-in-the-Middle attack, which is not something we want to do to a fresh install.

Fixing HTTPS Issues That Cause Slow-Loading Pages

As mentioned previously, Pi-hole is only handling the DNS queries and doesn't know about the other protocols that are taking place. But we can use iptables to manage these protocols to prevent time-outs allowing Pi-hole work it's magic.

There are several iptables rulesets you can put in place to optimize your Pi-hole's performance. These commands are explained in detail here, but at a high-level, they will prevent your Pi-hole from timing out over HTTP/HTTPS requests on ports 80/443, resulting in a faster browsing experience.

The following commands should fix many of the slowdowns people have been experiencing. You can read more about what these rules do, but you should have an understanding of what they do before running them.

You may also want to save your existing ruleset in case you want to restore them later.

iptables -A INPUT -p tcp --destination-port 443 -j REJECT --reject-with tcp-reset
iptables -A INPUT -p udp --destination-port 80 -j REJECT --reject-with icmp-port-unreachable
iptables -A INPUT -p udp --destination-port 443 -j REJECT --reject-with icmp-port-unreachable

ip6tables -A INPUT -p tcp --destination-port 443 -j REJECT --reject-with tcp-reset
ip6tables -A INPUT -p udp --destination-port 80 -j REJECT --reject-with icmp6-port-unreachable
ip6tables -A INPUT -p udp --destination-port 443 -j REJECT --reject-with icmp6-port-unreachable

These changes won't be applied permanently unless you save them:

iptables-save > /etc/iptables/rules.v4
ip6tables-save > /etc/iptables/rules.v6

The QUIC And The Google

Since a significant portion of Google's income comes from advertising, it's not surprising they do what they can to get their ads delivered to end users. The QUIC protocol is quite cool, but since it was developed by Google, they know how to leverage it to deliver ads in non-traditional ways. Thanks to one user on our forums, the iptables rules mentioned previously include blocking ads over this protocol.

Testing And Verifying After Making These Changes

You'll likely notice a difference if you simply browse the Internet as normal, but if you want to test it out with some static assets, you can try going to one of these URL's (assuming the domain is on your blocklist): We also have a page of different Websites you can visit to generate lots of ads or blocked queries. Something in there is bound to elicit the desired behavior.

If you are still seeing issues with HTTPS, you can use nmap on the machine experiencing the slowness to verify what's actually happening:

nmap --reason pi.hole -p443 -Pn # for TCP
nmap --reason pi.hole -p443 -sU -Pn # for UDP

You will see something like this:

PORT    STATE  SERVICE REASON
443/tcp closed https   reset ttl 64

You will want to look in the REASON/STATE columns. We have seen some instances where iptables is set to block/reject traffic, but nmap shows the port as filtered, not closed.

If this is the case for you, you may have a non-standard configuration on your network and may require further diagnoses. If this is the case, feel free to compile the necessary information and get a hold of us on Discourse and we'll help figure it out.

A Faster Pi-hole

Hopefully this sheds some light on why you might experience slow downs with your Pi-hole and how to fix them. Feel free to reach out to us with any additional questions.
3 Likes

Is there a way how the "not so tech savvy" of us can benefit from this?
Are you including some of these suggestions into the next version of Pi-Hole?

I like it it is currently simple to setup (more like setup-and-forget). Hope you can keep it this way!

These lines should be corrected to:

ip6tables -A INPUT -p udp --destination-port 80 -j REJECT --reject-with icmp6-port-unreachable
ip6tables -A INPUT -p udp --destination-port 443 -j REJECT --reject-with icmp6-port-unreachable

Otherwise you will receive an error message.

Thank, I have updated the post to reflect this.

Firewall rules for HTTPS part are iptables based, but CentOS uses firewalld.
Is it possible to include a part for firewalld, please?

Is it possible that the paths to the config files to make the rules persistent is not correct. On my raspberry pi the directory /etc/iptables/ did not exist. I have created it and saved the configuration to said files. But after a reboot the command iptables -S does not show the new rules.

Edit:
Have found out that I need the package iptables-persistent. Sorry.

We're still trying to track this down, hence these commands from the FAQ.

nmap --reason pi.hole -p443 -Pn # for TCP
nmap --reason pi.hole -p443 -sU -Pn # for UDP

AFAIK, we haven't come up with an exact reason yet.

If you are running pi-hole on CentOS 7, do this--

After a quite a bit of head-banging, I discovered that Red Hat / CentOS switched from iptables to firewalld. Since iptables is much more well-understood, let's revert back to to iptables.

first, disable firewalld with this command:
systemctl disable firewalld

Then install iptables-service by following command:
yum install iptables-services

Then enable iptables as services:
systemctl enable iptables

And now you can add the above iptables commands to your server.

Hope this saves someone from a headache...

I was suffering from the same issue.
So I completely reinstalled pi-hole and did not configure it. Now everything was working as expected.
Then I disabled ipv6 on the raspi itself and disabled aaaa on pi-hole. Now it is still working as expected. (No iptables needed!)

Before I only had aaaa disabled inside pi-hole but did not disable ipv6 on the raspi.
Maybe this was the cause of my issue.

but i think ads arent blocked over ipv6 then

For those still having this issue and are running Pi-Hole on CentOS, use the following Rich Rules WITHOUT having to disable Firewalld:

firewall-cmd --permanent --add-rich-rule='rule family=ipv4 port port=443 protocol=udp reject type=icmp-port-unreachable'

firewall-cmd --permanent --add-rich-rule='rule family=ipv4 port port=80 protocol=udp reject type=icmp-port-unreachable'

firewall-cmd --permanent --add-rich-rule='rule family=ipv4 port port=443 protocol=tcp reject type=tcp-reset'

firewall-cmd --reload

edit: Also, I have ipv6 disabled on pi-hole.

I just tried that out. Those commands only save the current iptables to a file. The files will stay but the current iptable will still be delete on reboot. Those rules.* files need to be reloaded on every reboot to work properly. An easy way to accomplish this is to use this command after saving those rules.* files.:

apt install iptables-persistent

It will pop up confirmation windows to save the rules persistently.
Use this to check whether it worked after a reboot. It will display the current iptables content:

iptables -L

I had to do this, to get no errors when I used the iptables-save commands (pi zero w, raspbian):

sudo mkdir /etc/iptables
sudo su
sudo iptables-save > /etc/iptables/rules.v4
sudo ip6tables-save > /etc/iptables/rules.v6
exit
sudo apt install iptables-persistent