Potential Work-around for spammy devices & apps

General
1

Note: I ran some searches on this problem and didn't notice anyone suggesting this solution, so I figured I would share it. If it's been done before and I just missed it, then I apologize.

Preface
There seems to be some apps/devices that are a bit spammy if they get blocked. Notably, roku smart TVs and related devices are a big offender, but I've also noticed that some app on my wife's smartphone apparently likes to get noisy when it doesn't get its way.

The two URLs I've experienced as major offenders are:
scribe.logs.roku.com (roku devices)
api2.branch.io (smartphone apps)

Since there's probably more out there than just the two as offenders, I thought I'd share my work-around to help reduce how chatty they are, just in case it helped someone else. Maybe it's not that big of a deal to you, and you just whitelist them. But I'm stubborn, and I refuse to let them win, so I've come up with an over-engineered 'solution'.

Concept
The concept is to make the device or app think it has a good domain name, and then waste time trying to connect to something that's not real. Normally, the pihole software replies with a 0.0.0.0 response telling the device that it couldn't be found, and so the device requests again and again - as though it's throwing a tantrum until it gets its way.

What to do
Practically speaking, here's what you do:

  • Set up a custom DNS entry under Local DNS --> DNS Records in the web interface for scribe.logs.roku.com (or whatever offending URL you have)
  • Point it at an IP address that you know will never resolve to anything real. (I chose 192.168.192.168)
  • Then whitelist that domain so that it resolves to the new IP instead of getting a blocked message.

I also like to set a block in my router for that domain as well, so I know it's not going to leave my network, just in case the device tries to bypass my pihole entirely, but that's a separate issue.

EDIT:---------------
I recently discovered a better way to do the above, using Pihole's regex functions. Here's two blacklist examples to get you started:

scribe\.logs\.roku\.com;reply=192.168.192.168
api2\.branch\.io;reply=192.168.192.168

Note that you can use the ;reply= notation to reply with any other kind of blocking message as well, making this method far more versatile than my previous one.
(ex: "scribe.logs.roku.com;reply=nxdomain" would reply with the nxdomain blocking mode for queries matching "scribe.logs.roku.com".)
For information on the ";reply=" regex options, look here: docs.pi-hole.net/regex/pi-hole/
For information on blocking modes, look here: docs.pi-hole.net/ftldns/blockingmode/
---------------------End EDIT

After running this for a couple of weeks, I've noticed somewhere between 1/4th and 1/2 the number of requests to these domains on my network, (depending on time of day, how much the devices were used, and other factors) and the massive spikes causing my roku to get a DNS timeout have stopped.

Potential Improvements
The only thing I can think of that would improve the functionality of this is if you could direct it at a web server that would always respond with an HTTP 503 message. Ideally one with a REALLY long delay specified in the Retry-After header. This would hopefully make the device think it successfully connected, but that it just had to wait a bit. Hours if we're lucky. But even just a few minutes of wait time would put a huge dent in this spam.
The obvious roadblock to this is having a trusted certificate for a domain you don't own so that the device will easily connect over SSL to get the message. There may be a way to trick a device or app into using plain HTTP instead, however, so I wouldn't rule this out entirely.

It would also be cosmetically nice if there was support for blocking certain URLs in this way baked into pihole. As it stands, these URLs now show as ALLOWED when I know they're being blocked. But that's a minor cost for saving my network from an internal DDoS caused by these spammy devices. I'll put in a proper feature request for this if this kind of rude behavior becomes more common in modern apps & devices.

Maybe some of the geniuses in here can improve on this technique further? What do you guys think?

1 Like
2

I really like this idea, but I tried what you proposed and the frequency of requests to scribe.logs.roku.com seems to be unchanged (several times per minute per device).

3

I was looking at my stats after letting it run this way for at least a day or more.
There's so many requests per minute without this change that cutting them in half or even in quarters isn't going to make a noticeable difference if you're just eyeballing it. Because you're still getting a ton of requests - it's just somewhat less - not a total elimination of the problem.

When messing about with this, I also tried assigning the IP of the roku itself, and the IP of the pihole. Both of those seemed to cut down the requests a bit at first, but over time it returned it nearly the same as the unmodified amount.
Make absolutely sure that the IP you pick to use for these domain names is going to attempt to connect and time out - not return a 404 or some similar error or generate a message from your router like "this host cannot be reached" or something.
You can test various IPs by plugging them into your browser in place of a domain. The longer your browser takes to tell you there's a problem, the better. Which browser you use shouldn't matter for this test, but just in case: I'm using Vivaldi, which is a chrome variant.

I recommend using one that's in a private range similar to yours, but not in your private range.
EX: I have an IP range for our home network of 192.168.2.0/24 - so I used 192.168.192.168 as the address.

This is giving me the best results so far, but I'm looking for better. If you figure out something that works better, or if this continues to not work for you like it has for me, please let me know. I don't want to go handing out advice without knowing it only works half of the time or something.

Thanks.

4

I had similar problem in my network where all the phones with Fitbit app installed wanting to connect to logx.optimizely.com (blocked in gravity) generating crap ton of DNS queries over and over. Fitbit app is continually going nuts every 30 secs trying to connect to logx.optimizely.com to log stuff.

I did a similar thing like you did to resolve that host to my webserver IP on the same network by adding a host entry for logx.optimizely.com in /etc/hosts. This does not help since (as you already figured) once you got past solving DNS query failure, you will hit the next problem in line i.e. trusted certificate for the domain in question which is impossible to solve. So I ended up allowing this since it is impossible to stop this nonsense as Fitbit is relentless.

Would be great if someone can figure out a solution that would stop this altogether. I do see Fitbit logging crap ton of info on this external server where the json payload it sends contains info from the phone like list of all paired bluetooth devices and bunch of other stuff!

1 Like
5

Aselvan, would you mind testing something for me?

I've been reading documentation and generally educating myself more on DNS tech, so I recently discovered both that you can implement different blocking modes, and that you can set a blocking mode only for a specific domain using the built-in regex feature in pihole.

To that end, here's a list of regex filters for your problem in the order I think they'll be most effective:

logx\.optimizely\.com;reply=192.168.192.168
logx\.optimizely\.com;reply=refused
logx\.optimizely\.com;reply=nxdomain
logx\.optimizely\.com;reply=none
logx\.optimizely\.com;reply=ip

Of course, this will require you to disable the whitelist you currently have for it, and some of these options may actually make the problem worse temporarily. So, I understand if you don't want to, or can't afford to do that right now.
However, I hope to eventually make a guide on how best to handle this problem, and any information you can get from testing these will eventually go into making that as solid as possible. In either case, your input has been and continues to be much appreciated!

I'm currently testing several of these options on the two domains I've been seeing this problem with, and I'm noticing varying results for each.

See the edit on my original post for more information/links if you're interested.

1 Like
6

I'd be glad to test it but unfortunately it won't help in my case w/ Fitbit as it relentlessly tries until it succeeds with "200 OK" for the REST API call it makes to siphon out users personal data in the name of "A/B testing" or logging or whatever! It is not feasible to force Fitbit to make a plain call instead of https in which case I can easily get around it by preventing it making connection or even do tarpit which may cause other issues for the legit use of the app. Oh well.

Back to your question... you just wanted to know if the regex you posted returns a static IP or refuse? I am not following the order you are talking about, can you elaborate more and I will test it for you.

7

You don't have to use the IP address one if you already have strong reason to think it won't work.

I'm curious if you see a reduction in DNS spam with the other ones enabled as blacklist rules. Different devices sometimes react differently to each kind of blocking, from my own testing at least. The default blocking style is reply=nodata so you can try reply=nxdomain, reply=none, and reply=refused and see if any of those will make the fitbit give up.

Something else (I'm just now reading about) you can try that will almost assuredly reduce DNS traffic:
Add BLOCK_TTL=60 to your /etc/pihole/pihole-FTL.conf file.
You can put whatever you feel comfortable with as a number there. The default is '2' - raising it to 60 makes it a full minute.

This setting is supposed to apply a TTL (Time To Live) to the DNS responses for blocked domains. In this case, I'm recommending a TTL of 60 seconds. The upside with this in effect is that devices are supposed to wait a full minute before requesting a blocked DNS name again. The downside is that it'll take a full minute before any new whitelist rules come into effect.

You can bump this up to an hour or more if you want to really cut it down, but be warned that will also make new whitelist entries take that long to engage as well.

8

For my case, ip (piholes IP), or x.x.x.x are out. As per none (no answer) DNS resolution will go to fallback DNS if they receive no answer, so that is out as well since I have couple of fallbacks in case Pi-hole for some reason not available. I will try the nxdomain or refused to see what the behaviour is later this weekend.

The BLOCK_TTL sounds interesting to try as well. Thanks for the tips!

9

This is typically not the case. DNS servers are usually not used in strict order. Clients are free to use any DNS server available to them at any time.

10

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.