unable to access docs.google.com

Looking at some of your 305 subscribed blocklists, I find many that aren't in hosts format. You are basically importing junk into your gravity list, and this is causing you problems.

https://sslbl.abuse.ch/blacklist/sslblacklist.csv
https://feodotracker.abuse.ch/downloads/ipblocklist.csv
http://data.phishtank.com/data/online-valid.csv
...
and there are many more.

This list in particular is fairly representative of some on your block list. It clearly states not to use as a blocklist, and it does not contain domain names - it is a list of IP's that are of no use to Pi-Hole.

http://www.dshield.org/ipsascii.html?limit=10000

# ipsascii.html
# DO NOT USE AS BLOCKLIST
# you may obtain longer/shorter lists by adding '?limit=1234' to the URL
# "1234" is the number of IPs returned. You can specify a number between 10
# and 10,000.
#   NOTE: Please use your API instead. See https://isc.sans.edu/api
#
#  IMPORTANT: Include your e-mail address in the user agent if you are using
#             a script to access this page.
#
# created: Fri, 15 Mar 2019 20:34:43 +0000# Date in GMT. YYYY-MM-DD format.
077.072.085.026	220891	9442	2018-12-27	2019-02-15
185.176.027.118	246484	9395	2019-02-02	2019-03-15
185.254.123.012	88391	8854	2019-01-10	2019-02-18
193.032.160.035	112718	8745	2019-01-18	2019-03-15

If I may ask, what is your purpose in blocking almost 10 million domains? And, how are you screening the block lists to which you subscribe? Have you looked at the contents of any of those lists, or are you just finding them listed online and adding them to your blocklists?

1 Like

Thank you for all the prompt replies. I have deleted lists that aren't giving data in the format required or data that is required (IPs as opposed to hostname, which DNS is queried for).

I took the lists I could find from the internet (most are from threads on discourse.pi-hole.net).

I have manually ran an update post deletion of the list. Is there a specific command to "rebuild" gravity list?

Thank you.

pihole -g rebuilds gravity.

Ah thank you very much. it is the same switch as updategravity.

I will post output of tail, head and grep commands in sometime. I have deleted lost of lists too. You gave me a text file, how do I import it to my list?

I didn't give you a file to import into any lists.

https://ncloud.zaclys.com/index.php/s/MPYSjLkb8cE8gPN/download

This link invoked a download which is a text file with ~400,000 domains. All having loopback address in front of them.

tail -n20 /etc/pihole/gravity.list

|http:*=$popup,domain=grammarist.com
|https:$domain=adfarm1.adition.com
|https:$popup,third-party,domain=eztv.ag|flashx.tv|imagerar.com|imgbox.com|mywebtv.info|sendvid.com|thevideo.me|uptobox.com|uptostream.com|yts.ag
|https:$popup,third-party,domain=fmovies.is
|https:$subdocument,third-party,domain=2ad.in|adf.ly|adfoc.us|adjet.biz|adv.li|ay.gy|j.gs|linkbucksmedia.com|q.gs|sh.st|thevideo.me|tvbximak.com|u.bb
|https:$third-party,xmlhttprequest,domain=123movies-proxy.ru|123movies.cz|123movies.gs|123movies.is|123movies.live|123movies.net|123movies.net.ru|123movies.ru|123movies.vc|123movieshd.net|123movieshd.tv|dropapk.com|estream.to|hdmusic99.in|otorrents.com|solarmovie.sc|wholecloud.net
|javascript:*settimeout*location.href$popup
|javascript^$popup,domain=1337x.to|biology-online.org|bitvid.sx|cloudtime.to|eztv.ag|eztv.tf|eztv.yt|flashx.tv|gorillavid.in|letwatch.us|movpod.in|nowvideo.sx|nowvideo.to|wholecloud.net
▒眠症改善.com
▒眠症改善.jpn.com
▒氧水.com
▒イフラボ.com
▒・ダーマラボ99.net
▒ethèrwallét.com
ètherwället.com
ƒluwdu.cn
дольщикиспб.рф
лоу-пак.com
шляхтен.рф
ラジオショッピング.jp

head -n20 /etc/pihole/gravity.list

!akamai.smartadserver.com
!cdn1.smartadserver.com
!diff.smartadserver.com
!diff1.smartadserver.com
!diff2.smartadserver.com
!diff3.smartadserver.com
!diff4.smartadserver.com
!diff5.smartadserver.com
!eqx.smartadserver.com
!im2.smartadserver.com
!itx5-publicidad.smartadserver.com
!itx5.smartadserver.com
!mobile.smartadserver.com
!rtb-csync.smartadserver.com
!smartadserver.com
!tcy.smartadserver.com
!ww1003.smartadserver.com
!ww1270.smartadserver.com
!ww129.smartadserver.com

sudo grep https: /etc/pihole/gravity.list | tail -n30

a[onmousedown^="this.href='https:
|https:$domain=adfarm1.adition.com
|https:$popup,third-party,domain=eztv.ag|flashx.tv|imagerar.com|imgbox.com|mywebtv.info|sendvid.com|thevideo.me|uptobox.com|uptostream.com|yts.ag
|https:$popup,third-party,domain=fmovies.is
|https:$subdocument,third-party,domain=2ad.in|adf.ly|adfoc.us|adjet.biz|adv.li|ay.gy|j.gs|linkbucksmedia.com|q.gs|sh.st|thevideo.me|tvbximak.com|u.bb
|https:$third-party,xmlhttprequest,domain=123movies-proxy.ru|123movies.cz|123movies.gs|123movies.is|123movies.live|123movies.net|123movies.net.ru|123movies.ru|123movies.vc|123movieshd.net|123movieshd.tv|dropapk.com|estream.to|hdmusic99.in|otorrents.com|solarmovie.sc|wholecloud.net

None of those are valid domains, nor valid entries that will work with Pi-hole.

The link was to the block list that contained the invalid domain .docs.google.com that is causing your problem. This is one of many lists you are using that have improper domains, and this list should be removed from your blocklist.

The head and tail of your gravity list shows that your gravity list contains improperly formatted domains that won't work with Pi-Hole. We only looked at the very ends of the gravity list, since that's where odd characters show up. The search for "https" in the gravity list shows further evidence - the text "https:" should not appear in a hosts format list.

The solution for you - completely remove all your block lists. Put back the seven original blocklists (link below), then selectively add properly formatted block lists one at a time. There is no value in randomly grabbing junk lists from the internet - this is preventing your Pi-Hole from working properly as evidenced by your post.

https://discourse.pi-hole.net/t/how-can-i-restore-pi-holes-default-ad-lists/4683/3?u=ramset

Properly formatted lists are found here:

Great! I agree dumping lists that the system does not parse (understand) is making the system fragile instead of strong.

is there a single command to delete all the lists? Or do I have to delete them manually through the web administration portal?

From the terminal:

sudo rm /etc/pihole/adlists.list

pihole -g -f

Thanks, I have rebuilt the system. :slight_smile:

I would still like to highlight that even with default list blocks other internal lists from updating. Is there a way we can diagnose and fix this?

[i] Target: s3.amazonaws.com (simple_ad.txt)
[✗] Status: s3.amazonaws.com is blocked by https://adblock.mahakala.is. Using DNS on 8.26.56.26 to download https://s3.amazonaws.com/lists.disconnect.me/simple_ad.txt
[✗] Status: https://s3.amazonaws.com/lists.disconnect.me/simple_ad.txt (200000)
[✗] List download failed: using previously cached list

[i] Target: hosts-file.net (ad_servers.txt)
[✗] Status: hosts-file.net is blocked by https://adblock.mahakala.is. Using DNS on 8.26.56.26 to download https://hosts-file.net/ad_servers.txt
[✓] Status: No changes detected

[i] Target: hosts-file.net (grm.txt)
[✗] Status: hosts-file.net is blocked by https://adblock.mahakala.is. Using DNS on 8.26.56.26 to download https://hosts-file.net/grm.txt
[✓] Status: No changes detected

[i] Target: s3.amazonaws.com (simple_malvertising.txt)
[✗] Status: s3.amazonaws.com is blocked by https://adblock.mahakala.is. Using DNS on 8.26.56.26 to download https://s3.amazonaws.com/lists.disconnect.me/simple_malvertising.txt
[✗] Status: https://s3.amazonaws.com/lists.disconnect.me/simple_malvertising.txt (304000)
[✗] List download failed: using previously cached list

[i] Target: hosts-file.net (exp.txt)
[✗] Status: hosts-file.net is blocked by https://adblock.mahakala.is. Using DNS on 8.26.56.26 to download https://hosts-file.net/exp.txt
[✓] Status: No changes detected

[i] Target: hosts-file.net (emd.txt)
[✗] Status: hosts-file.net is blocked by https://adblock.mahakala.is. Using DNS on 8.26.56.26 to download https://hosts-file.net/emd.txt
[✓] Status: No changes detected

[i] Target: hosts-file.net (psh.txt)
[✗] Status: hosts-file.net is blocked by https://adblock.mahakala.is. Using DNS on 8.26.56.26 to download https://hosts-file.net/psh.txt
[✓] Status: No changes detected

How have you rebuilt the system?

https://adblock.mahakala.is is not one of the default block lists.

What is the output of:

cat /etc/pihole/adlists.list

wc -l /etc/pihole/gravity.list

Yes I rebuilt the system. I formatted the memory card, reinstalled Pi-Hole and went with default list.

I have carefully started adding new ones though. I am not sure which command to use to weed out incompatible entries so I kept doing:

tail -n20 /etc/pihole/gravity.list
sudo grep https: /etc/pihole/gravity.list | tail -n30

Both of which did not give any output. I considered this to be a good sign. I have added few custom list but only after posting the thread. At the time I ran my first update there were no custom lists added. After the first update, I started getting these errors (I always update via SSH). -- Ignoring the auto-update.

wc -l /etc/pihole/gravity.list
1 /etc/pihole/gravity.list

cat /etc/pihole/adlists.list
https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts
https://mirror1.malwaredomains.com/files/justdomains
http://sysctl.org/cameleon/hosts
https://zeustracker.abuse.ch/blocklist.php?download=domainblocklist
https://s3.amazonaws.com/lists.disconnect.me/simple_ad.txt
https://hosts-file.net/ad_servers.txt
https://hosts-file.net/grm.txt
https://reddestdream.github.io/Projects/MinimalHosts/etc/MinimalHostsBlocker/minimalhosts
https://raw.githubusercontent.com/StevenBlack/hosts/master/data/KADhosts/hosts
https://raw.githubusercontent.com/StevenBlack/hosts/master/data/add.Spam/hosts
https://v.firebog.net/hosts/static/w3kbl.txt
https://v.firebog.net/hosts/BillStearns.txt
https://www.dshield.org/feeds/suspiciousdomains_Low.txt
https://www.dshield.org/feeds/suspiciousdomains_Medium.txt
https://www.dshield.org/feeds/suspiciousdomains_High.txt
https://www.joewein.net/dl/bl/dom-bl-base.txt
https://raw.githubusercontent.com/matomo-org/referrer-spam-blacklist/master/spammers.txt
https://hostsfile.org/Downloads/hosts.txt
https://someonewhocares.org/hosts/zero/hosts
https://raw.githubusercontent.com/Dawsey21/Lists/master/main-blacklist.txt
https://raw.githubusercontent.com/vokins/yhosts/master/hosts
http://winhelp2002.mvps.org/hosts.txt
https://hostsfile.mine.nu/hosts0.txt
https://v.firebog.net/hosts/Kowabit.txt
https://adblock.mahakala.is
https://adaway.org/hosts.txt
https://v.firebog.net/hosts/AdguardDNS.txt
https://raw.githubusercontent.com/anudeepND/blacklist/master/adservers.txt
https://v.firebog.net/hosts/Easylist.txt
https://pgl.yoyo.org/adservers/serverlist.php?hostformat=hosts;showintro=0
https://raw.githubusercontent.com/StevenBlack/hosts/master/data/UncheckyAds/hosts
https://www.squidblacklist.org/downloads/dg-ads.acl
https://v.firebog.net/hosts/Easyprivacy.txt
https://v.firebog.net/hosts/Prigent-Ads.txt
https://gitlab.com/quidsup/notrack-blocklists/raw/master/notrack-blocklist.txt
https://raw.githubusercontent.com/StevenBlack/hosts/master/data/add.2o7Net/hosts
https://raw.githubusercontent.com/crazy-max/WindowsSpyBlocker/master/data/hosts/spy.txt
https://raw.githubusercontent.com/Perflyst/PiHoleBlocklist/master/android-tracking.txt
https://raw.githubusercontent.com/Perflyst/PiHoleBlocklist/master/SmartTV.txt
https://v.firebog.net/hosts/Airelle-trc.txt
https://s3.amazonaws.com/lists.disconnect.me/simple_malvertising.txt
https://hosts-file.net/exp.txt
https://hosts-file.net/emd.txt
https://hosts-file.net/psh.txt
https://mirror.cedia.org.ec/malwaredomains/immortal_domains.txt
https://www.malwaredomainlist.com/hostslist/hosts.txt
https://bitbucket.org/ethanr/dns-blacklists/raw/8575c9f96e5b4a1308f2f12394abd86d0927a4a0/bad_lists/Mandiant_APT1_Report_Appendix_D.txt
https://v.firebog.net/hosts/Prigent-Malware.txt
https://v.firebog.net/hosts/Prigent-Phishing.txt
https://phishing.army/download/phishing_army_blocklist_extended.txt
https://gitlab.com/quidsup/notrack-blocklists/raw/master/notrack-malware.txt
https://ransomwaretracker.abuse.ch/downloads/RW_DOMBL.txt
https://ransomwaretracker.abuse.ch/downloads/CW_C2_DOMBL.txt
https://ransomwaretracker.abuse.ch/downloads/LY_C2_DOMBL.txt
https://ransomwaretracker.abuse.ch/downloads/TC_C2_DOMBL.txt
https://ransomwaretracker.abuse.ch/downloads/TL_C2_DOMBL.txt
https://v.firebog.net/hosts/Shalla-mal.txt
https://raw.githubusercontent.com/StevenBlack/hosts/master/data/add.Risk/hosts
https://www.squidblacklist.org/downloads/dg-malicious.acl
https://raw.githubusercontent.com/HorusTeknoloji/TR-PhishingList/master/url-lists.txt
https://v.firebog.net/hosts/Airelle-hrsk.txt
https://github.com/chadmayfield/pihole-blocklists/raw/master/lists/pi_blocklist_porn_all.list
https://raw.githubusercontent.com/chadmayfield/pihole-blocklists/master/lists/pi_blocklist_porn_top1m.list
https://zerodot1.gitlab.io/CoinBlockerLists/hosts
https://tspprs.com/dl/tracking
https://tspprs.com/dl/crypto
https://zerodot1.gitlab.io/CoinBlockerLists/list.txt
https://raw.githubusercontent.com/chadmayfield/my-pihole-blocklists/master/lists/pi_blocklist_porn_all.list

Sorry? I did not understand that?

What is the command to see if the list I've added (or all present) are being parsed properly and Pi-Hole is able to understand them.

This list that you have added (not one of the default lists) is known to contain a large number of false positives, including domains that serve other blocklists.

Note that despite this list blocking other lists, Pi-Hole has bypassed itself to download the requested list.

You are adding lists that won't block any more domains that you will ever run across, but will cause a large number of false positives and cause you trouble.

Just to quickly let you know, the reason download successded (pihole update) -- it is because I disabled Pi-Hole before moving ahead. :slight_smile:

Before I close this thread, is there a way I can verify which list are parsing correctly and which are causing issues to Pi-Hole's functioning?

This topic was automatically closed 21 days after the last reply. New replies are no longer allowed.