They likely have their webserver configured to respond to any hostname or a URI of '*'. If you tried to go to http://dogpile.com/not_a_page it would first redirect you to http://duckduckgo.com/not_a_page and then to https://duckduckgo.com/not_a_page (I don't think duckduckgo is on the HSTS preload list though.) And finally you'd be sent to https://duckduckgo.com/index.html or how ever they handle 404s since not_a_page wouldn't exist. Or any page at dogpile.com/<pagehere>.
Interestingly, I just added a local cname record on my pi-hole:
which leads to the following result (that I wouldn't think was possible)
1 Like
It looks like duckduckgo.com was already in the cache before you added the CNAME.
Still, this really doesn't do much. It would not be of any use other than a bare URL with no paths on it.
I think what you are seeing is from a combination of CNAME and duckduckgo.com having a 404 page that is the same as their index page.
A better check would be a curl -IL http://dogpile.com and see what the 301/302/303/404 pages are that are being served.
1 Like
Now it gets really strange. Cleared my system cache, closed my browser, did what you suggested, result:
1 Like
Not unexpected. They are basically taking any unknown URI as a search term and rewriting it to be a web search.
Edit: that's the
part of my comment.
OK, but no warnings, no errors, The browser gets (user input) Dogpile.com, duckduckgo shows a result...
Pi-hole had the A record for duckduckgo.com in it's cache when the query for dogpile.com was sent. Otherwise dig dogpile.com would return:
dig dogpile.com @127.0.0.1
; <<>> DiG 9.11.5-P4-5.1+deb10u3-Raspbian <<>> dogpile.com @127.0.0.1
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 54470
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;dogpile.com. IN A
;; ANSWER SECTION:
dogpile.com. 2 IN CNAME duckduckgo.com.
;; Query time: 1 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
;; WHEN: Sat Apr 17 19:09:22 BST 2021
;; MSG SIZE rcvd: 68
And no A/AAA record provided.
tried another AGH redirect, www.demorgen.be -> www.standaard.be (some local newspapers). This time, firefox isn't able to show the content, not even after clear cache, restart firefox or reload (CTRLF5).
Looks like duckduckgo uses some real fancy code to handle all of these problems, succeeding to show the page anyway...
As I initially concluded, looks like it works (AGH) for some protocols, Redirection uses CNAME but it does return a valid IP address for the target, dnsmasq doesn't return an IP address...
1 Like
Yeah, they do a 301 internally for any queries to be updated.
dan@raspberrypi:~ $ dig dogpile.com
; <<>> DiG 9.11.5-P4-5.1+deb10u3-Raspbian <<>> dogpile.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 52958
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;dogpile.com. IN A
;; ANSWER SECTION:
dogpile.com. 2 IN CNAME duckduckgo.com.
duckduckgo.com. 2 IN A 52.250.42.157
;; Query time: 1 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
;; WHEN: Sat Apr 17 19:26:58 BST 2021
;; MSG SIZE rcvd: 84
dan@raspberrypi:~ $ curl -IL http://dogpile.com/not_a_file
HTTP/1.1 301 Moved Permanently
Server: nginx
Date: Sat, 17 Apr 2021 18:27:05 GMT
Content-Type: text/html
Content-Length: 162
Connection: keep-alive
Location: https://duckduckgo.com/not_a_file
Permissions-Policy: interest-cohort=()
Content-Security-Policy: default-src 'none' ; connect-src https://duckduckgo.com https://*.duckduckgo.com https://3g2upl4pq6kufc4m.onion/ ; manifest-src https://duckduckgo.com https://*.duckduckgo.com https://3g2upl4pq6kufc4m.onion/ ; media-src https://duckduckgo.com https://*.duckduckgo.com https://3g2upl4pq6kufc4m.onion/ ; script-src blob: https://duckduckgo.com https://*.duckduckgo.com https://3g2upl4pq6kufc4m.onion/ 'unsafe-inline' 'unsafe-eval' ; font-src data: https://duckduckgo.com https://*.duckduckgo.com https://3g2upl4pq6kufc4m.onion/ ; img-src data: https://duckduckgo.com https://*.duckduckgo.com https://3g2upl4pq6kufc4m.onion/ ; style-src https://duckduckgo.com https://*.duckduckgo.com https://3g2upl4pq6kufc4m.onion/ 'unsafe-inline' ; object-src 'none' ; worker-src blob: ; child-src blob: https://duckduckgo.com https://*.duckduckgo.com https://3g2upl4pq6kufc4m.onion/ ; frame-src blob: https://duckduckgo.com https://*.duckduckgo.com https://3g2upl4pq6kufc4m.onion/ ; form-action https://duckduckgo.com https://*.duckduckgo.com https://3g2upl4pq6kufc4m.onion/ https://duck.co ; frame-ancestors 'self' ; base-uri 'self' ; block-all-mixed-content ;
X-Frame-Options: SAMEORIGIN
X-XSS-Protection: 1;mode=block
X-Content-Type-Options: nosniff
Referrer-Policy: origin
Expect-CT: max-age=0
Expires: Sun, 17 Apr 2022 18:27:05 GMT
Cache-Control: max-age=31536000
X-DuckDuckGo-Locale: en_US
HTTP/2 200
server: nginx
date: Sat, 17 Apr 2021 18:27:06 GMT
content-type: text/html; charset=UTF-8
vary: Accept-Encoding
server-timing: total;dur=13;desc="Backend Total"
x-duckduckgo-results: 1
strict-transport-security: max-age=31536000
permissions-policy: interest-cohort=()
content-security-policy: default-src 'none' ; connect-src https://duckduckgo.com https://*.duckduckgo.com https://3g2upl4pq6kufc4m.onion/ ; manifest-src https://duckduckgo.com https://*.duckduckgo.com https://3g2upl4pq6kufc4m.onion/ ; media-src https://duckduckgo.com https://*.duckduckgo.com https://3g2upl4pq6kufc4m.onion/ ; script-src blob: https://duckduckgo.com https://*.duckduckgo.com https://3g2upl4pq6kufc4m.onion/ 'unsafe-inline' 'unsafe-eval' ; font-src data: https://duckduckgo.com https://*.duckduckgo.com https://3g2upl4pq6kufc4m.onion/ ; img-src data: https://duckduckgo.com https://*.duckduckgo.com https://3g2upl4pq6kufc4m.onion/ ; style-src https://duckduckgo.com https://*.duckduckgo.com https://3g2upl4pq6kufc4m.onion/ 'unsafe-inline' ; object-src 'none' ; worker-src blob: ; child-src blob: https://duckduckgo.com https://*.duckduckgo.com https://3g2upl4pq6kufc4m.onion/ ; frame-src blob: https://duckduckgo.com https://*.duckduckgo.com https://3g2upl4pq6kufc4m.onion/ ; form-action https://duckduckgo.com https://*.duckduckgo.com https://3g2upl4pq6kufc4m.onion/ https://duck.co ; frame-ancestors 'self' ; base-uri 'self' ; block-all-mixed-content ;
x-frame-options: SAMEORIGIN
x-xss-protection: 1;mode=block
x-content-type-options: nosniff
referrer-policy: origin
expect-ct: max-age=0
expires: Sat, 17 Apr 2021 18:27:07 GMT
cache-control: max-age=1
x-duckduckgo-locale: en_US
So this is where the browser is told that it needs to change any requests from http://dogpile.com/....... to https://duckduckgo.com/....... and send a new request for that URI.
Way to go duckduckgo!!!
Anyway, for other protocols, AGH returns a usable IP address with the CNAME, which the application than uses to display the data (NTP example).
Is this something Simon needs to add, to remain competitive with AGH, something DL6ER can add in pihole-FTL, or simply ignore, e.g. use AGH if you need this?
The number of cases where this won't work and people will expect it to work is so much larger than the times it would work.
It's up to Simon on how he'd prefer to go but I don't see it being helpful since 1 out of 50 people would be helped by it and the 49 other people would need a course in protocols and explanations why it may work for NTP but not for TLS or anything else that requires FQDN for the service. And doesn't depend on the server being able to accommodate invalid requests.
NextDNS does a TLS blockpage that some people like but you won't see it in Pi-hole since that requires clients to have a Certificate Authority installed that allows for complete MITM. You have to install NextDNS's CA files and allow them to have ultimate ability to create TLS certificates on your clients.
I managed to get below hack working because the NTP protocol doesnt communicate domain/hostnames.
Below would be impossible for like example HTTPS(SNI) and any other protocol using TLS .. except if you setup MITM 
Considering below "limitations":
pi@ph5a:~ $ man dnsmasq
[..]
--cname=<cname>,[<cname>,]<target>[,<TTL>]
Return a CNAME record which indicates that <cname> is really
<target>. There are significant limitations on the target; it
must be a DNS name which is known to dnsmasq from /etc/hosts (or
additional hosts files), from DHCP, from --interface-name or
from another --cname. If the target does not satisfy this cri‐
teria, the whole cname is ignored. The cname must be unique, but
it is permissible to have more than one cname pointing to the
same target. Indeed it's possible to declare multiple cnames to
a target in a single line, like so: --cname=cname1,cname2,target
If the time-to-live is given, it overrides the default, which is
zero or the value of --local-ttl. The value is a positive inte‐
ger and gives the time-to-live in seconds.
I first created a dnsmasq config file 99-my-settings.conf containing a local TXT DNS record definition on the Pi-hole host to make embedded dnsmasq "authoritative" for the 1.pool.ntp.org domain:
pi@ph5a:~ $ sudo tee /etc/dnsmasq.d/99-my-settings.conf <<< $'txt-record=1.pool.ntp.org,"Dummy record to become authoritative"'
txt-record=1.pool.ntp.org,"Dummy record to become authoritative"
Test syntax:
pi@ph5a:~ $ pihole-FTL --test -c /etc/dnsmasq.d/99-my-settings.conf
dnsmasq: syntax check OK.
And reloaded to apply:
pi@ph5a:~ $ sudo service pihole-FTL reload
pi@ph5a:~ $
Created a CNAME via the web GUI: time.android.com --> 1.pool.ntp.org :
http://pi.hole/admin/cname_records.php
And grabbed my laptop client to test:
dehakkelaar@laptop:~$ host -t txt 1.pool.ntp.org
1.pool.ntp.org descriptive text "Dummy record to become authoritative"
dehakkelaar@laptop:~$ host time.android.com
time.android.com is an alias for 1.pool.ntp.org.
1.pool.ntp.org has address 94.198.159.11
1.pool.ntp.org has address 185.255.55.20
1.pool.ntp.org has address 5.79.75.37
1.pool.ntp.org has address 108.61.164.200
dehakkelaar@laptop:~$ sudo ntpdate -q time.android.com
server 108.61.164.200, stratum 2, offset -0.001624, delay 0.03429
server 94.198.159.11, stratum 2, offset -0.001078, delay 0.03870
server 185.255.55.20, stratum 2, offset -0.009562, delay 0.05182
server 5.79.75.37, stratum 2, offset -0.001605, delay 0.03404
18 Apr 20:03:01 ntpdate[6084]: adjust time server 5.79.75.37 offset -0.001605 sec
But ...
why not push your own NTP server(s) to the clients via DHCP ?
pi@ph5a:~ $ pihole-FTL -- --help dhcp
Known DHCP options:
[..]
42 ntp-server
[..]
pi@ph5a:~ $ sudo tee /etc/dnsmasq.d/99-my-settings.conf <<< $'dhcp-option=option:ntp-server,10.0.0.3'
dhcp-option=option:ntp-server,10.0.0.3
pi@ph5a:~ $ pihole-FTL --test -c /etc/dnsmasq.d/99-my-settings.conf
dnsmasq: syntax check OK.
pi@ph5a:~ $ sudo service pihole-FTL reload
pi@ph5a:~ $
pi@ph5a:~ $ pihole-FTL dhcp-discover
Scanning all your interfaces for DHCP servers
[..]
ntp-server: 10.0.0.3
You could also install the ntp package on your Pi-hole host(s) and push that one via DHCP:
pi@ph5a:~ $ apt show ntp
[..]
Description: Network Time Protocol daemon and utility programs
NTP, the Network Time Protocol, is used to keep computer clocks
accurate by synchronizing them over the Internet or a local network,
or by following an accurate hardware receiver that interprets GPS,
DCF-77, NIST or similar time signals.
.
This package contains the NTP daemon and utility programs. An NTP
daemon needs to be running on each host that is to have its clock
accuracy controlled by NTP. The same NTP daemon is also used to
provide NTP service to other hosts.
.
For more information about the NTP protocol and NTP server
configuration and operation, install the package "ntp-doc".
Or maybe your already running an NTP service on your LAN:
pi@ph5a:~ $ sudo nmap -sU -p123 --open 10.0.0.0/24
Starting Nmap 7.70 ( https://nmap.org ) at 2021-02-28 16:41 CET
Nmap scan report for 10.0.0.3 (10.0.0.3)
Host is up (0.00062s latency).
PORT STATE SERVICE
123/udp open ntp
MAC Address: 00:11:32:xx:xx:xx (Synology Incorporated)
Nmap scan report for 10.0.0.9 (10.0.0.9)
Host is up (0.00093s latency).
PORT STATE SERVICE
123/udp open ntp
MAC Address: B8:27:EB:xx:xx:xx (Raspberry Pi Foundation)
Nmap scan report for 10.0.0.11 (10.0.0.11)
Host is up (0.00050s latency).
PORT STATE SERVICE
123/udp open|filtered ntp
MAC Address: 00:1E:0B:xx:xx:xx (Hewlett Packard)
Nmap done: 256 IP addresses (10 hosts up) scanned in 8.10 seconds
Pew ... it became a bit lengthy
1 Like
Hmm never mind.
Seems the CNAME only works while the 1.pool.ntp.org A records are still cached by pihole-FTL:
$ dig a time.android.com @10.0.0.2
[..]
;; ANSWER SECTION:
time.android.com. 2 IN CNAME 1.pool.ntp.org.
1.pool.ntp.org. 17 IN A 185.255.55.20
1.pool.ntp.org. 17 IN A 5.79.75.37
1.pool.ntp.org. 17 IN A 81.173.63.225
1.pool.ntp.org. 17 IN A 213.154.236.182
$ dig a time.android.com @10.0.0.2
[..]
;; ANSWER SECTION:
time.android.com. 2 IN CNAME 1.pool.ntp.org.
1.pool.ntp.org. 1 IN A 213.154.236.182
1.pool.ntp.org. 1 IN A 185.255.55.20
1.pool.ntp.org. 1 IN A 5.79.75.37
1.pool.ntp.org. 1 IN A 81.173.63.225
$ dig a time.android.com @10.0.0.2
[..]
;; ANSWER SECTION:
time.android.com. 2 IN CNAME 1.pool.ntp.org.
Not sure why ... yet.
Could be a bug or something I dont know 
Again, I never asked for HTTPS. I know, that my request would crash the encryption at those protocols completely.
But Dan made clear, that I have a big mistake in the way how HTTP works. And I fully understand why it would be a bad idea to implement. I am taking my request back. No need anymore. It would crash to much.
I tried to mark every useful post as solution.
Thanks for every participation at this post!
1 Like
Yeah confirmed, if I copy over the A records for 1.pool.ntp.org to my local Pi-hole setup, it works bc the records dont exist only in cache:
pi@ph5a:~ $ cat /etc/dnsmasq.d/99-my-settings.conf
cname=time.android.com,1.pool.ntp.org
host-record=1.pool.ntp.org,94.198.159.11,185.255.55.20,5.79.75.37,108.61.164.200
But that defeats the purpose of getting the IP's updated when they change in the pool unfortunately
EDIT: removed txt-record bc A records for target exists already.
EDIT2: Ow I noticed the host-record= definitions need to be on separate lines or else only one IP would show:
pi@ph5a:~ $ cat /etc/dnsmasq.d/99-my-settings.conf
cname=time.android.com,1.pool.ntp.org
host-record=1.pool.ntp.org,94.198.159.11
host-record=1.pool.ntp.org,185.255.55.20
host-record=1.pool.ntp.org,5.79.75.37
host-record=1.pool.ntp.org,108.61.164.200
pi@ph5a:~ $ host -t a time.android.com localhost
Using domain server:
Name: localhost
Address: ::1#53
Aliases:
time.android.com is an alias for 1.pool.ntp.org.
1.pool.ntp.org has address 185.255.55.20
1.pool.ntp.org has address 5.79.75.37
1.pool.ntp.org has address 108.61.164.200
1.pool.ntp.org has address 94.198.159.11
dehakkelaar@laptop~$ sudo ntpdate -q time.android.com
server 5.79.75.37, stratum 2, offset 0.001129, delay 0.03499
server 108.61.164.200, stratum 2, offset 0.000806, delay 0.03580
server 94.198.159.11, stratum 2, offset 0.001800, delay 0.03883
server 185.255.55.20, stratum 2, offset -0.009495, delay 0.05605
19 Apr 00:04:52 ntpdate[29500]: adjust time server 5.79.75.37 offset 0.001129 sec