no, it is not on the blocking list
What is "it" ?
And below question ?
Solved now ?
No as I did not add a php reference to the blocking list :-/
There seems to be a limit for the gravity db file size, because I have enough free disk space.
Na dont think so.
Probably some php/curl limit if the link holds too many domains.
I dont know.
Wait for a dev to have a look.
Try post the jinxed blocklist URL here that your trying to add.
And post a new debug token for the devs to have a look.
What is the exact output from pihole -g -r?
Debug Token: https://tricorder.pi-hole.net/mcf2yuo096
Blocklist:
1 https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts 1 2020-01-24 12:50:26 2020-02-14 19:39:30 Migrated from /etc/pihole/adlists.list
2 https://mirror1.malwaredomains.com/files/justdomains 1 2020-01-24 12:50:26 2020-02-14 19:39:30 Migrated from /etc/pihole/adlists.list
3 http://sysctl.org/cameleon/hosts 1 2020-01-24 12:50:26 2020-02-14 19:39:30 Migrated from /etc/pihole/adlists.list
4 https://s3.amazonaws.com/lists.disconnect.me/simple_tracking.txt 1 2020-01-24 12:50:26 2020-02-14 19:39:30 Migrated from /etc/pihole/adlists.list
5 https://s3.amazonaws.com/lists.disconnect.me/simple_ad.txt 1 2020-01-24 12:50:26 2020-02-14 19:39:30 Migrated from /etc/pihole/adlists.list
6 https://hosts-file.net/ad_servers.txt 1 2020-01-24 12:50:26 2020-02-14 19:39:30 Migrated from /etc/pihole/adlists.list
10 https://hosts-file.net/grm.txt 1 2020-01-24 13:07:21 2020-02-14 19:39:30
11 https://reddestdream.github.io/Projects/MinimalHosts/etc/MinimalHostsBlocker/minimalhosts 1 2020-01-24 13:07:21 2020-02-14 19:39:30
12 https://raw.githubusercontent.com/StevenBlack/hosts/master/data/KADhosts/hosts 1 2020-01-24 13:07:21 2020-02-14 19:39:30
13 https://raw.githubusercontent.com/StevenBlack/hosts/master/data/add.Spam/hosts 1 2020-01-24 13:07:21 2020-02-14 19:39:30
17 https://www.dshield.org/feeds/suspiciousdomains_Low.txt 1 2020-01-24 13:07:21 2020-02-14 19:39:30
18 https://www.dshield.org/feeds/suspiciousdomains_Medium.txt 1 2020-01-24 13:07:21 2020-02-14 19:39:30
19 https://www.dshield.org/feeds/suspiciousdomains_High.txt 1 2020-01-24 13:07:21 2020-02-14 19:39:30
20 https://www.joewein.net/dl/bl/dom-bl-base.txt 1 2020-01-24 13:07:21 2020-02-14 19:39:30
21 https://raw.githubusercontent.com/matomo-org/referrer-spam-blacklist/master/spammers.txt 1 2020-01-24 13:07:21 2020-02-14 19:39:30
22 https://hostsfile.org/Downloads/hosts.txt 1 2020-01-24 13:07:21 2020-02-14 19:39:30
23 https://someonewhocares.org/hosts/zero/hosts 1 2020-01-24 13:07:21 2020-02-14 19:39:30
24 https://raw.githubusercontent.com/Dawsey21/Lists/master/main-blacklist.txt 1 2020-01-24 13:07:21 2020-02-14 19:39:30
25 https://raw.githubusercontent.com/vokins/yhosts/master/hosts 1 2020-01-24 13:07:21 2020-02-14 19:39:30
26 http://winhelp2002.mvps.org/hosts.txt 1 2020-01-24 13:07:21 2020-02-14 19:39:30
27 https://hosts.nfz.moe/basic/hosts 1 2020-01-24 13:07:21 2020-02-14 19:39:30
28 https://raw.githubusercontent.com/RooneyMcNibNug/pihole-stuff/master/SNAFU.txt 1 2020-01-24 13:07:21 2020-02-14 19:39:30
29 https://ssl.bblck.me/blacklists/hosts-file.txt 1 2020-01-24 13:07:21 2020-02-14 19:39:30
30 https://hostsfile.mine.nu/hosts0.txt 1 2020-01-24 13:07:21 2020-02-14 19:39:30
32 https://adblock.mahakala.is 1 2020-01-24 13:07:21 2020-02-14 19:39:30
33 https://adaway.org/hosts.txt 1 2020-01-24 13:07:21 2020-02-14 19:39:30
35 https://raw.githubusercontent.com/anudeepND/blacklist/master/adservers.txt 1 2020-01-24 13:07:21 2020-02-14 19:39:30
39 https://pgl.yoyo.org/adservers/serverlist.php?hostformat=hosts&showintro=0&mimetype=plaintext 1 2020-01-24 13:07:21 2020-02-14 19:39:30
40 https://raw.githubusercontent.com/StevenBlack/hosts/master/data/UncheckyAds/hosts 1 2020-01-24 13:07:21 2020-02-14 19:39:30
42 https://raw.githubusercontent.com/bigdargon/hostsVN/master/hosts 1 2020-01-24 13:07:21 2020-02-14 19:39:31
43 https://raw.githubusercontent.com/jdlingyu/ad-wars/master/hosts 1 2020-01-24 13:07:21 2020-02-14 19:39:31
46 https://gitlab.com/quidsup/notrack-blocklists/raw/master/notrack-blocklist.txt 1 2020-01-24 13:07:21 2020-02-14 19:39:31
47 https://raw.githubusercontent.com/StevenBlack/hosts/master/data/add.2o7Net/hosts 1 2020-01-24 13:07:21 2020-02-14 19:39:31
48 https://raw.githubusercontent.com/crazy-max/WindowsSpyBlocker/master/data/hosts/spy.txt 1 2020-01-24 13:07:21 2020-02-14 19:39:31
49 https://raw.githubusercontent.com/Kees1958/WS3_annual_most_used_survey_blocklist/master/w3tech_hostf 1 2020-01-24 13:07:21 2020-02-14 19:39:31
50 https://www.github.developerdan.com/hosts/lists/ads-and-tracking-extended.txt 1 2020-01-24 13:07:21 2020-02-14 19:39:31
51 https://hostfiles.frogeye.fr/firstparty-trackers-hosts.txt 1 2020-01-24 13:07:21 2020-02-14 19:39:31
52 https://hostfiles.frogeye.fr/multiparty-trackers-hosts.txt 1 2020-01-24 13:07:21 2020-02-14 19:39:31
53 https://raw.githubusercontent.com/Perflyst/PiHoleBlocklist/master/android-tracking.txt 1 2020-01-24 13:07:21 2020-02-14 19:39:31
54 https://raw.githubusercontent.com/Perflyst/PiHoleBlocklist/master/SmartTV.txt 1 2020-01-24 13:07:21 2020-02-14 19:39:31
55 https://raw.githubusercontent.com/Perflyst/PiHoleBlocklist/master/AmazonFireTV.txt 1 2020-01-24 13:07:21 2020-02-14 19:39:31
57 https://s3.amazonaws.com/lists.disconnect.me/simple_malvertising.txt 1 2020-01-24 13:07:21 2020-02-14 19:39:31
59 https://hosts-file.net/exp.txt 1 2020-01-24 13:07:21 2020-02-14 19:39:31
60 https://hosts-file.net/emd.txt 1 2020-01-24 13:07:21 2020-02-14 19:39:31
61 https://hosts-file.net/psh.txt 1 2020-01-24 13:07:21 2020-02-14 19:39:31
62 https://mirror.cedia.org.ec/malwaredomains/immortal_domains.txt 1 2020-01-24 13:07:21 2020-02-14 19:39:31
63 https://www.malwaredomainlist.com/hostslist/hosts.txt 1 2020-01-24 13:07:21 2020-02-14 19:39:31
64 https://bitbucket.org/ethanr/dns-blacklists/raw/8575c9f96e5b4a1308f2f12394abd86d0927a4a0/bad_lists/M 1 2020-01-24 13:07:21 2020-02-14 19:39:31
67 https://phishing.army/download/phishing_army_blocklist_extended.txt 1 2020-01-24 13:07:21 2020-02-14 19:39:31
68 https://gitlab.com/quidsup/notrack-blocklists/raw/master/notrack-malware.txt 1 2020-01-24 13:07:21 2020-02-14 19:39:31
70 https://raw.githubusercontent.com/StevenBlack/hosts/master/data/add.Risk/hosts 1 2020-01-24 13:07:21 2020-02-14 19:39:31
72 https://gitlab.com/curben/urlhaus-filter/raw/master/urlhaus-filter-hosts.txt 1 2020-01-24 13:07:21 2020-02-14 19:39:31
73 https://raw.githubusercontent.com/DandelionSprout/adfilt/master/Alternate%20versions%20Anti-Malware% 1 2020-01-24 13:07:21 2020-02-14 19:39:31
74 https://raw.githubusercontent.com/HorusTeknoloji/TR-PhishingList/master/url-lists.txt 1 2020-01-24 13:07:21 2020-02-14 19:39:31
76 https://zerodot1.gitlab.io/CoinBlockerLists/hosts_browser 1 2020-01-24 13:07:21 2020-02-14 19:39:31
Output pihole -g -r
pi@raspberrypi:~ $ pihole -g -r
[✓] Restoring from migration backup
[i] Creating new gravity database
[i] Migrating content of /etc/pihole/adlists.list into new database
[i] Migrating content of /etc/pihole/regex.list into new database
[i] Neutrino emissions detected...
[✓] Pulling blocklist source list into range
[✓] Preparing new gravity database
[i] Target: https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts
[✓] Status: No changes detected
[i] Target: https://mirror1.malwaredomains.com/files/justdomains
[✓] Status: No changes detected
[i] Target: http://sysctl.org/cameleon/hosts
[✓] Status: No changes detected
[i] Target: https://s3.amazonaws.com/lists.disconnect.me/simple_tracking.txt
[✓] Status: No changes detected
[i] Target: https://s3.amazonaws.com/lists.disconnect.me/simple_ad.txt
[✓] Status: No changes detected
[i] Target: https://hosts-file.net/ad_servers.txt
[✓] Status: No changes detected
[✓] Storing downloaded domains in new gravity database
[✓] Building tree
[✓] Swapping databases
[✓] Flushing DNS cache
[i] Number of gravity domains: 146359 (123826 unique domains)
[i] Number of exact blacklisted domains: 0
[i] Number of regex blacklist filters: 0
[i] Number of exact whitelisted domains: 0
[i] Number of regex whitelist filters: 0
[✓] Cleaning up stray matter
[✓] DNS service is running
[✓] Pi-hole blocking is Enabled
And is everything working now with the new database?
yes, but I cannot add more addlists
How can I solve it?
How are you trying to add lists?
By the webinterface.
Somewhat like 717436 (501059 unique domains) does seem to be the maxium.
Can I change the database size somehow?
Does it show any error or how do you know it doesn't work?
Hint: Remember that you'll have to run pihole -g after modifying the set of blocking lists.
The domains are not automatically updated as this would cause a too large delay if done automatically.
Adding more lists makes the database larger, having fewer makes it smaller. There are no limits for anything defined in Pi-hole.
I get the same error:
[✗] Unable to build gravity tree in /etc/pihole/gravity_temp.db
Error: database or disk is full
Adding more lists makes the database larger, having fewer makes it smaller. There are no limits for anything defined in Pi-hole.
This can't be true. If I remove some blocking lists, I can build the database, if I add another list, I will get the error again.
It is. We have a lot of users having several million domains on their blocking lists without ever having had any issue even on 8 GB SD cards.
I'm sure we'll get there in the end, but I wonder if there is any other issue with the file system on your device.
Removed file system testing instructions
edit Ignore the filesystem test instructions I wrote here before.
while building the tree does never mean the database is too large. It always means: the disk is full.
And this IS the case for you, as I've just seen that
your temp directory is too small to hold the growing tree structure. We intentionally store the tree piecewise on-disk (instead of entirely in-memory) while it is growing to avoid issues with the typically severely limited amount of RAM on SBC devices.
Solution: Enlarge your temp directory (it should not have been this small to even begin with!) and your issue will be resolved.
1 Like
Hi,
for some reasons, /var/tmp and /tmp have been mounted (see Make /tmp location configurable? - #8 by jpgpi250) on my rpi3. I removed them from my /etc/fstab, and now it works, no filesize problem anymore.
Thank you!
2 Likes
For anyone trying to leave the /var/tmp, /tmp in tmpfs (to save SD card life) and get this to work, you might want to check your permissions. Somehow something was causing the /tmp to have 0755 which didn't allow anyone except root to write stuff. I followed this gist Fix ignored mode of tmpfs mount on /tmp and everything works as it should now.
This is my fstab. I will increase the size on demand if I encounter any errors with writing again.
tmpfs /tmp tmpfs defaults,noatime,nosuid,size=100M 0 0
tmpfs /var/tmp tmpfs nodev,nosuid,size=100M 0 0
Your posted gist is not a valid and working link.
Thanks! Edited & fixed.