What Determines Memory Usage?

How does FTL Determine its Memory Usage?

Expected Behaviour:

The memory usage be reflective of the domain amount.

Actual Behaviour:

I downloaded every block list I had onto my computer. There are totaling to 18million something unique domains. All this does not even take up 1 gig on my Windows machine (about 0.8 gb). When I just add ~5million domains to the Pi-hole, I'm getting a very high memory usage, about 85%. The hard drive is a fresh 16 gig card that only has Pi-hole installed on it with Raspbian Desktop.

I have also have some Regex blocking enabled.

What contributes to this--what I consider--high memory usage for not a lot of domains?

5 million domains is a lot of domains. The memory usage is RAM usage, not disk storage usage. Also consider that the space the domain takes up in the list is not the same amount of space it takes up in memory, because dnsmasq and FTL store metadata about each domain.

In addition to Mcat12 comments, with NULL blocking (the default in Pi-Hole), each of the domains in gravity is carried twice (once for the A record and once for the AAAA record). If you shift your blocking mode to NXDOMAIN, this cuts the memory requirement for the gravity list roughly in half.

How much RAM is on your Pi-Hole host platform?

I'm running it on a Raspberry Pi 3 B+, which should have 1gb of RAM. I'll look into switching to NXDOMAIN.

Thanks, that makes sense. Is there any way to cut down on RAM usage?

You also might want to re-evaluate your block lists. After some point you are adding domains that you will never visit in your lifetime, and in my experience many of the domains on existing block lists are not active on the internet (a dig returns NXDOMAIN). You may find a few hundred thousand variations of youtube subdomains, which aren't particularly helpful since the subdomains change regularly.

Open your /etc/pihole/gravity.list and give it a look - you may be surprised what you find.

I fully agree with what @jfb and @Mcat12 said, five million domains is way too much even for 24/7 web surfers. I'm still using the default lists (~ 140,000 domains) and don't see any ads anywhere, but I agree that the mileage may vary depending on what pages you're browsing.

The domains are stored in RAM to allow quick access. If we'd have to read in 0.8 GB for every incoming domain requests, this would probably mean a few dozen seconds (up to a few minutes depending on the SD card rating) for each requests to be answered. This would be totally unacceptable.

1 Like

Several.

  1. Reduce the number of domains on blocklist. Each domain takes memory.
  2. Change blocking mode to NXDOMAIN (note that in V4.2 the memory use of NULL will be reduced to the same as NXDOMAIN, but that is not yet in the current version).

This topic was automatically closed 21 days after the last reply. New replies are no longer allowed.