Hide/Block HTTPS Requests

[pihole2]

This is happening on all of my iOS devices. Just not the iPAD-Pro. It happens on several apps, just not one in particular.

An example of an application where this happens is "CheapCharts".

Also, this has been happening since I've been using Pi-Hole, which is about 4 years now.

[DL6ER]

@pihole2

this has been happening since I've been using Pi-Hole, which is about 4 years now.

Interesting, so it was likely even be happening before.

According to

What’s new at Firebase Summit 2018

over 1.5 million apps are actively using Firebase every month.

Another interesting point is that this bug seems to be present only on Apple devices but not on Android.

Anyway, there is nothing we can do about this but, fortunately, this misbehavior is completely harmless.

I'm not sure if a shortcut to hide them is meaningful, thoughts @moderators ?

[jfb]

My comment regarding the iPad Pro was not well thought out. Disregard.

I run a number of Pi-holes on my network, serving different subsets of clients. My wife's stuff is on some of them, and my stuff is on others. Reasons.

Here is what I found when I checked the most recent six days of dnsmasq logs - I see these from each of our IOS devices, hers more than mine.

zgrep "https://" /var/log/pihole.log* | grep query

/var/log/pihole.log.5.gz:Oct 25 14:55:25 dnsmasq[11039]: query[A] https://app-measurement.com/sdk-exp from 192.168.0.125
/var/log/pihole.log.5.gz:Oct 25 12:50:22 dnsmasq[23030]: query[A] https://app-measurement.com/sdk-exp from 192.168.0.126
/var/log/pihole.log.5.gz:Oct 25 06:10:46 dnsmasq[23030]: query[A] https://app-measurement.com/sdk-exp from 192.168.0.126
/var/log/pihole.log.4.gz:Oct 26 16:08:17 dnsmasq[23030]: query[A] https://app-measurement.com/sdk-exp from 192.168.0.126
/var/log/pihole.log.4.gz:Oct 26 15:25:51 dnsmasq[14596]: query[A] https://app-measurement.com/sdk-exp from 192.168.0.134
/var/log/pihole.log.4.gz:Oct 26 12:43:05 dnsmasq[14596]: query[A] https://app-measurement.com/sdk-exp from 192.168.0.134
/var/log/pihole.log.4.gz:Oct 26 09:08:43 dnsmasq[11039]: query[A] https://app-measurement.com/sdk-exp from 192.168.0.125
/var/log/pihole.log.4.gz:Oct 26 08:22:08 dnsmasq[14596]: query[A] https://app-measurement.com/sdk-exp from 192.168.0.134
/var/log/pihole.log.3.gz:Oct 27 19:32:23 dnsmasq[11039]: query[A] https://app-measurement.com/sdk-exp from 192.168.0.125
/var/log/pihole.log.3.gz:Oct 27 18:06:06 dnsmasq[23030]: query[A] https://app-measurement.com/sdk-exp from 192.168.0.126
/var/log/pihole.log.3.gz:Oct 27 16:01:55 dnsmasq[23030]: query[A] https://app-measurement.com/sdk-exp from 192.168.0.126
/var/log/pihole.log.3.gz:Oct 27 12:51:51 dnsmasq[11039]: query[A] https://app-measurement.com/sdk-exp from 192.168.0.125
/var/log/pihole.log.3.gz:Oct 27 12:16:12 dnsmasq[14596]: query[A] https://app-measurement.com/sdk-exp from 192.168.0.134
/var/log/pihole.log.3.gz:Oct 27 10:28:53 dnsmasq[14596]: query[A] https://app-measurement.com/sdk-exp from 192.168.0.134
/var/log/pihole.log.3.gz:Oct 27 10:15:05 dnsmasq[23030]: query[A] https://app-measurement.com/sdk-exp from 192.168.0.126
/var/log/pihole.log.3.gz:Oct 27 05:39:00 dnsmasq[23030]: query[A] https://app-measurement.com/sdk-exp from 192.168.0.126
/var/log/pihole.log.2.gz:Oct 28 21:15:29 dnsmasq[11039]: query[A] https://app-measurement.com/sdk-exp from 192.168.0.125
/var/log/pihole.log.2.gz:Oct 28 16:07:46 dnsmasq[23030]: query[A] https://app-measurement.com/sdk-exp from 192.168.0.126
/var/log/pihole.log.2.gz:Oct 28 14:53:57 dnsmasq[23030]: query[A] https://app-measurement.com/sdk-exp from 192.168.0.126
/var/log/pihole.log.2.gz:Oct 28 12:40:09 dnsmasq[11136]: query[A] https://app-measurement.com/sdk-exp from 192.168.0.134
/var/log/pihole.log.2.gz:Oct 28 08:31:42 dnsmasq[23030]: query[A] https://app-measurement.com/sdk-exp from 192.168.0.126
/var/log/pihole.log.1:Oct 29 21:19:14 dnsmasq[23030]: query[A] https://app-measurement.com/sdk-exp from 192.168.0.126
/var/log/pihole.log.1:Oct 29 19:38:22 dnsmasq[23030]: query[A] https://app-measurement.com/sdk-exp from 192.168.0.126
/var/log/pihole.log.1:Oct 29 12:54:50 dnsmasq[23030]: query[A] https://app-measurement.com/sdk-exp from 192.168.0.126
/var/log/pihole.log.1:Oct 29 11:10:58 dnsmasq[23030]: query[A] https://app-measurement.com/sdk-exp from 192.168.0.126
/var/log/pihole.log.1:Oct 29 10:47:38 dnsmasq[11136]: query[A] https://app-measurement.com/sdk-exp from 192.168.0.134
/var/log/pihole.log.1:Oct 29 08:09:55 dnsmasq[23030]: query[A] https://app-measurement.com/sdk-exp from 192.168.0.126
/var/log/pihole.log:Oct 30 11:27:37 dnsmasq[8589]: query[A] https://app-measurement.com/sdk-exp from 192.168.0.132
/var/log/pihole.log:Oct 30 11:26:03 dnsmasq[23030]: query[A] https://app-measurement.com/sdk-exp from 192.168.0.126
/var/log/pihole.log:Oct 30 07:13:04 dnsmasq[11039]: query[A] https://app-measurement.com/sdk-exp from 192.168.0.125

IP's as follows:
125 - her iPad Pro IOS 15
126 - her iPhone 12 IOS 15
132 - my iPad IOS 15
134 - my iPhone 7 Plus IOS 15

@DL6ER

Is there a second query somewhere requesting the domain without https://

In my case, yes:

/var/log/pihole.log.1:Oct 29 08:09:55 dnsmasq[23030]: query[A] https://app-measurement.com/sdk-exp from 192.168.0.126
/var/log/pihole.log.1:Oct 29 08:09:56 dnsmasq[23030]: query[A] app-measurement.com from 192.168.0.126
/var/log/pihole.log.1:Oct 29 08:09:56 dnsmasq[23030]: query[type=65] app-measurement.com from 192.168.0.126
...
/var/log/pihole.log.1:Oct 29 11:10:58 dnsmasq[23030]: query[A] https://app-measurement.com/sdk-exp from 192.168.0.126
/var/log/pihole.log.1:Oct 29 11:10:59 dnsmasq[23030]: query[A] app-measurement.com from 192.168.0.126
/var/log/pihole.log.1:Oct 29 11:10:59 dnsmasq[23030]: query[type=65] app-measurement.com from 192.168.0.126
...
/var/log/pihole.log.1:Oct 29 12:54:50 dnsmasq[23030]: query[A] https://app-measurement.com/sdk-exp from 192.168.0.126
/var/log/pihole.log.1:Oct 29 12:54:50 dnsmasq[23030]: query[A] app-measurement.com from 192.168.0.126

[jfb]

Since this is not really a feature request at this point (we have an option to handle HTTPS queries via regex), I moved this to Community Help.

[pihole2]

The reason I made it a feature request was if something could be implemented as below.

This is from the Pi-Hole Remote app. It does hide these from the query log when viewing through it.

[jfb]

I'll move it back to a Feature Request. It will accumulate votes like other open feature requests.

[DL6ER]

@pihole2

It does hide these from the query log when viewing through it.

This is interesting and also somewhat misleading. As it says "hide HTTPS queries" under the title "TYPE", I'd have expected it to do exactly this: Hide queries with type HTTPS.

However, what you describe is that it hides all queries (also A type ones) when then domain starts in `https://... This is obviously something different. Or does it hide both kinds of queries?

[Bucking_Horn]

I think that is a misunderstanding:
Pi-hole cannot hide any HTTPS requests, as it never even receives them.

Pi-hole is only receiving DNS requests - in your case, a request for an improper domain name of https://app-measurement.com/sdk-exp. Like any domain, that can be blocked (via they ways already discussed) and/or excluded via Pi-hole's API / Web interface UI.

The true fix for this would be for the app and kit developers to adjust their incorrect DNS requests.

[yubiuser]

@DL6ER

I'm not sure if a shortcut to hide them is meaningful, thoughts @moderators ?

@Bucking_Horn

The true fix for this would be for the app and kit developers to adjust their incorrect DNS requests.

I would not add an specific option to hide the mal-configured https://app-measurement.com/sdk-exp nor HTTPS (TYPE 65) requests. If at all, this feature would need to be way broader:

Option to ignore domains from appearing in the Query Log

Some services are polling several times in a minute. All these entries are showing up in the Query Log. There should be an option to hide or ignore/exclude domains from the Query Log viewer.

Option to not log selected client(s)

I'd like to have one (or more) clients not show up in the logs. They do not to be logged at all. They need not to bypass the pihole. It is great that the pihole filter applies to them, I just want their 'noise' removed from the logs. Is it possible already, or would this be a new feature?

Hide Specific Groups on Dashboard

I created a "block all requests" group and added my IOT devices to it (because who needs their NAS phoning home, right?). they don't like it when I put baby in the corner and fill up my dashboard with their squealing. it would be nice to be able to hide this group from the main dashboard so I can get better detail on my other devices.

[jfb]

Since this query type is unique to Apple IOS, my preference would be to not implement this feature request. We don't selectively hide other requests that are unique to other platforms - root server queries from routers, etc.

[pihole2]

It hides both types of queries.

[pihole2]

It may be unique to iOS devices, however, they are very common devices. Unfortunately, when you have 6 iOS devices, this appears within the query log quite often.

As far as asking for apps to update, it's happening on so many apps. Are we asking any app that uses app-measurement.com to update the SDK? It's not limited to one or two apps.

I understand you don't hide root server queries. This would help keep the query log clean and easy to view. It's not something that would need to be a default option, but would be nice if there was an option to hide A query's showing HTTPS in the query log.

[jfb]

@pihole2

It hides both types of queries.

It does not. The setting in the Pi-hole Remote app only hides queries of type HTTPS. The queries you see with the leading https:// are type A queries.

[pihole2]

It was hiding it for me. There was a new release today, not sure if this behavior changed. If you are the developer of the app, thanks! This is how I thought of the idea because of it being hid via the app.

Even if it is not hiding it now, I was using this as an example of what could potentially be added.

[jfb]

@pihole2

It was hiding it for me

I am not associated with that app in any way, although I do use it. It has no method to not display type A queries that lead with https://. It has the option to not display DNS queries of type HTTPS (since July 2021).

[Bucking_Horn]

@pihole2

but would be nice if there was an option to hide A query's showing HTTPS in the query log.

That's already possible:

@Bucking_Horn

Like any domain, that can be blocked (via the ways already discussed) and/or excluded via Pi-hole's API / Web interface UI.

[MichaIng]

Interesting issue. Instead of an option to hide queries with specific hostnames (which is already possible via generic filter, as stated), an idea would be to hide failed queries with invalid hostnames in general, i.e. which contain invalid characters like slashes /. But I'd still be against it to encourage users debugging the faulty client. I mean you might have found a bug in an iOS core library here which may be responsible for a lot of unnecessary traffic and system loads all over the world . So this should possibly be reported to iOS devs .

[DL6ER]

@MichaIng

So this should possibly be reported to iOS devs .

I fully agree. Even more because

@DL6ER

over 1.5 million apps are actively using Firebase every month

so this is affecting a really large number of devices and generates a lot of useless traffic. It's always hard to get any real numbers but the energy this extra traffic is wasting may even compare to the energy demand of several households. We are used to pay Internet as a flatrate price these days, however, it should not be forgotten that the entire Internet infrastructure uses a lot of energy.

Back in 2011 (ten years ago), Raghavan and Ma estimated

the Internet uses 84 to 143 gigawatts of electricity every year, which amounts to between 3.6 and 6.2 percent of all electricity worldwide. Taking emergy into account, the total comes up to 170 to 307 gigawatts.

and more recently,

The Internet will use a fifth of all the world’s electricity by 2025

(source).

For the price you are paying for Apple products, I expect no less than exceptional and user-friendly support.