Just dropping in to say hello and intro myself. I love the community layout. Clean and Professional.
I especially love the product. Kicking myself for not trying it sooner than I did. I was wondering how you can tell which app is generating queries to a specific url/fqdn. Is there some repository that maps urls to applications or companies. Most Google URLs are easy to identify.
Other than that, for now, I am just a consumer. Perhaps, I'll be able to contribute at some point in the future. I do have more questions but for now I am looking around and seeing what people have to say and what issues they are having.
There is no master list that I know of. Sometimes a Google search will return a useful result if you search for that domain, but this is hit/miss. The Reddit r/pihole forum frequently gets requests for "what is causing these domain requests."
The first step is to determine which client is creating the request. Then it's a process of looking at what software is running on the client, and look at the patterns in the request (you can do this by tailing the pihole.log - Admin GUI > tools). There is also the element of time - after a while you will see repeated requests and patterns that help you narrow it down.
Here is a good FAQ on how to find where an ad is coming from. You can use these techniques in reverse as well to figure out what is requesting what. DNSThingy Chrome extension is a good tool.
The top 10 request on the main page would be a good start to see what is going on in your lan.
Match the requesting Ip's to a device.
If there is an app on that device with lots of ads, You could start with blacklisting the domains that are requested as soon as you start the app.
Block until the ads are gone or until the app stops working.
It's time consuming, but there is plenty of blacklist availble already from people that have done that steps for you. Just add them and see if it helps.
This is how I do this mostly. Add the backlists a much as possible.
If an app breaks or webppage breaks, se what is blocked and whiteltst.
Thanks for the replies guys. I appreciate it. I actually did run into the link you provided. I hadn't tried out the DNS Thingy yet since I was [and still am] a bit taken back by all the chatter on my LAN. A bit of information overload.
For the most part identifying the clients submitting dns queries is pretty straight forward. Identifying urls is also pretty easy for 95% of the domain queries. But it's just that 5% [less actually] that have weird names. I ran into one the other day that didn't look familiar at all. Had to use Arin Whois online db and still wasn't able to discern which online entity or application was generating those queries. The most I could discover is that it was some online marketing agency which I suppose they are tens of thousands of those who all probably have multiple clients and the clients all probably have multiple apps.
So I've been pretty heavy-handed in black-listing domains that I don't know and then looking for which application is broken as a result and then switching the domain from black to white list.
A bit of sub-plot here: It appears that a datastore is created by PiHole. One that survives a log flush. If the schema of that database could be expanded to include Applications known to query those domains and companies associated with those domains.... Well, .... heh... that seems like it could be some very vital/valuable information. I mean, ... that's not even considering the affect on "fake news" that could have. I suspect that kind thinking could lead to marketing companies paying the public for the right to track us. Crazy! I know.
Anyway, sorry for the long winded response. One final thought... and this is just an observation. I had no clue how chatty all the various connected devices are. My primary laptop is the #1 client but clients #2 - #5 were my cellular phone and television sets. My phone just sits there and there is no open applications but theirs a ton of stuff running in the background that are flooding the PiHole DNS server. I even double checked that there were no apps running in the background on android. But there my phone was, in sleep mode, screen black...and it was banging away and querying various domains. I just check and there were 72 queries in the last 6 minutes. The TVs were worse. I fixed the TVs though. I had the dhcp server hand out a bogus dns-server address of 0.0.0.0 just to their specific MAC addresses. I probably could have just disabled networking on the TVs especially since I am running Kodi on Raspberry Pis and connect to the tvs that way. [The RPi's are absolutely quiet by comparison to the TVs and my phone] ... but handing out bogus dns-server address seemed like a more centralized solution. One that I can use and expand in the future should the need arise.
Anyway, that's all for now. Sorry for the novel...
Yes: unbelievable how much data is send by devices, even in standby.
Even when locking/unlocking an iphone there is data send to apple.
Scary thoughs....how much do "they" know.?
So if you see so many requests from your android Phone. : wack'm and blacklist and test if nothing breaks.. and share the lists with the community.
Yep! Already dabbled and tinkered with the sqlite db behind Pi-Hole after reading another article that provided the steps to perform a sort of 'factory reset' on Pi-Hole. Also tried on the "Dark Theme" for the web interface that I found in another conversation.
I tinker with my Emby Media Center's db which sits on a DBServer that also supports one of the Fantasy Football Leagues I'm a member of. Ive been told that I broke a lot of toys when I was a kid. Then put them back together just so I would know how they worked. Aside from breaking things, I guess some things don't change. I'm sure a lot of us in tech can relate.
Anyway, all that to say, I don't have a problem with compiling [and sharing] a list of domains and the applications and companies that are associated with them. Though,... I wonder if I surf enough of the web to compile a list that was meaningful.
I'm just thinking out loud here, but I imagine a meaningful list would be a consolidation of several thousand users. Then, if each user collected/shared their lists, there would still be wild, varied levels of details that each person provides.
I suspect that Pi-hole provides an API for exactly this reason. So that someone with the know-how, willingness and time could develop an addon/plugin that would could deploy agents [for unix, windows, android, etc] to collect an app inventory that also included listening ports from all connected nodes on a users LAN.
While I have experience doing similar things on Wintel platforms. Other platforms would require a learning curve.
Anyway, these are just rambling, musings .. I'm sure that security and anti-virus corporations probably already have such lists....
This sounds like something that would go nicely in the enhancement category. Similar to different themes, running speed check, etc. I don't see this as something the developers have time to work on.
I'm sure a lot of us in tech can relate.