Some remarks/requests regarding v2.12

General
1

The new v2.12 seems to be doing ok, however, I ran into the following issues.

  • Settings page: Since I'm also running DNScrypt-proxy, neither /etc/dnsmasqd/01-pihole.conf nor /etc/pihole/setupVars.conf contain a SERVER setting (removed, as instructed, during setup of DNScrypt-proxy). Suggestion: detect if DNScrypt-proxy is running and ignore the missing SERVER setting. Currently, settings can not be saved.
  • DNSsec: As I researched and documented here, raspbian jessie needs a higher version of dnsmasq (minimum version 2.74 - see the dnsmasq changelog) for it to properly handle DNSsec requests. I assume it's pointless to enable DNSsec, if you haven't upgraded dnsmasq. Suggestion: detect the version of dnsmasq, before showing the option in the settings page.
  • Hardware real time clock: since a pi doesn't have a real time clock, I have added dnssec-no-timecheck to /etc/dnsmasqd/01-pihole.conf, explanation here. Suggestion: determine the hardware platform and optionally add this setting.
2

You suggest to disable the possibility to change DNS servers on the Settings page when dnscrypt is detected, is that correct? I do not use it, how to detect it reliably?

It is not easy to detect the version of dnsmasq from the web interface (I don't necessarily run dnsmasq -v). Users will ask why they cannot find it although we say we implemented it - However, I might simply disable the check box.

Why is that an issue? The Pi should always get its time via NTP.

3

These are just suggestions, you may have better ideas.

I'm currently investigating to NOT use DNScrypt-loader at all, and go for dnscrypt directly, using these instructions, however the problem remains, the SERVER settings in /etc/pihole/setupVars.conf must be commented out. If you could use something like ps -e | grep dnscrypt to detect dnscrypt or dnscrypt-loader as an alternative to checking the SERVER settings, that would solve the problem. You already did a similar thing in /opt/pihole/piholeLogFlush.sh to detect if logrotate is available.

Disabling the check box doesn't sound like a good idea either. A better approach would be to replace the checkbox and description with a message that explains the absence of the checkbox

As the man page explains, NTP is not able to get the correct time before dnsmasq is loaded (no DNS available). This is particullarry true for users that don't run the pi 24/7 (you even added a line to cron for them). Adding the line ensures dnsmasq doesn't generate false BOGUS replies.
--dnssec-no-timecheck
DNSSEC signatures are only valid for specified time windows, and should be rejected outside those windows. This generates an interesting chicken-and-egg problem for machines which don't have a hardware real time clock. For these machines to determine the correct time typically requires use of NTP and therefore DNS, but validating DNS requires that the correct time is already known. Setting this flag removes the time-window checks (but not other DNSSEC validation.) only until the dnsmasq process receives SIGHUP. The intention is that dnsmasq should be started with this flag when the platform determines that reliable time is not currently available. As soon as reliable time is established, a SIGHUP should be sent to dnsmasq, which enables time checking, and purges the cache of DNS records which have not been throughly checked.

4

Just a reminder, any changes to that file are likely to get overwritten in the next update! If you want to add additional flags to dnsmsaq's config, then may I suggest adding them to a file such as /etc/dnsmasq.d/02-customconfig.conf

5

I think you can no longer just replace /etc/dnsmasqd/01-pihole.conf when planning an upgrade, as the DNSSEC stings are added optionally, using a script. A better way may be to parse the configuration file, replacing or adding the required settings and leaving the remainder untouched (just like you do with setupVars.conf)
just my 2 cents....

6

Do you mean the dnssec and trust-anchor= lines added when ticking the DNSSEC option of the web interface? If so, those are accounted for during an upgrade. What we don't account for are custom changes made by users (such as adding the line dnssec-no-timecheck)

7

Apologies. This is why you guys are developers, I'm nothing but an observer. Anyhow, I keep an offline copy off all the files I change, this to be able to track changes. No problem here.

8

Hey, no worries man! We may be developers, but we are by no means infallible :slight_smile:, in fact we can sometimes develop tunnel-vision on certain aspects of the code, so it's nice to have things looked at from another perspective.

If you put your custom settings in another file, you wont need to worry about keeping track of them!

9

I've ceated 05-dnssec.conf in /etc/dnsmasq.d, containing
dnssec
trust-anchor=.,19036,8,2,49AAC11D7B6F6446702E54A1607371607A1A41855200FD2CE1CDDE32F24E8FB5
dnssec-no-timecheck
This means I can no longer use the settings page to save my settings, but that was already impossible, due to the setup of DNScrypt
04-dnscrypt.conf in /etc/dnsmasq.d contains
server=127.10.10.1#5553
server=127.10.10.2#5554

10

Let me say: You don't need the Settings page if you are fine with editing config files by hand. Remember that the Settings page is there for the majority of the users. If we would consider any possible setup it could soon become unmanageable for us. I see the problem that an update will mess around with the settings but that might be something we can tackle.

Let's assume you manually specify the DNS server in setupVars.conf to be PIHOLE_DNS_1=Nothing. We could detect that during an update and skip putting any DNS server lines into the 01-pihole.conf. Is there anything else that is getting destroyed with an update?

11
  1. setupVars.conf:
    As per instructions, when setting up DNScrypt, the SERVER setting(s) are commented out. Any update (pihole -up) seems to remove commented lines, but variables NOT known to pihole are left alone. Example: I've added a setting ALLOW_WHITELISTING=true, this to manipulate the availability of the whitelist button in the blockpage. This variable survives any configuration change / upgrade. The code for this in /var/www/html/pihole/index.php is:
    <a class='safe33' href='javascript:history.back()'>Go back</a>
    <?php if((isset($setupVars["ALLOW_WHITELISTING"])) && ($setupVars["ALLOW_WHITELISTING"] == 1)){ ?>
    <a class='safe33' id="whitelisting">Whitelist this page</a>
    <?php } ?>
    <a class='safe33' href='javascript:window.close()'>Close window</a>
    Of course, this is lost during an upgrade.
  2. /etc/dnsmasq.d configuration files:
    I currently have 4 dnsmasq configuration files (not using DHCP). Maybe you could add these files, by name, to setupVars.conf, and get the appropriate settings from the configuration files e.g.
    PIHOLE=/etc/dnsmasq.d/01-pihole.conf
    DHCP=/etc/dnsmasq.d/???
    WILDCARD=/etc/dnsmasq.d/03-pihole-wildcard.conf
    DNSCRYPT=/etc/dnsmasq.d/04-dnscrypt.conf
    DNSSEC=/etc/dnsmasq.d/05-dnssec.conf
    This means you look for any DNSSEC settings in the appropriate file, look for SERVER settings in the DNScrypt file, even if DNScrypt isn't running. You're already doing that for the wildcard settings, except the filename is fixed (not opposed to that either, just need the naming convention).
    For the SERVER settings, this means you need to leave the port number and separator alone, or allow this as valid input in the settings page!!!
    I don't need the settings page, except maybe only once to determine the result of a setting on the settings page (e.g. bogus-priv comes from the option "never forward non-FQDNs", if I'm right)
    This may also be in line with this pull request;
    I didn't check this out yet, but there is competition out there. Sticking with pi-hole:slight_smile:, smarter developers...
12

Picking up on my suggestion to add dnssec-no-timecheck, for use on systems without a hardware clock, such as a raspberry pi. I've investigated this further and came to the unpleasant surprise that it takes my pi (running the latest raspbian jessie lite, all packages updated) about 15 minutes to achieve time synchronization, this after a simple reboot.
As I see the question coming, here is my NTP configuration, default configuration, just changed the servers to europe
server 0.europe.pool.ntp.org iburst
server 1.europe.pool.ntp.org iburst
server 2.europe.pool.ntp.org iburst
server 3.europe.pool.ntp.org iburst

This is what I did (no extra packages required - ntp-wait comes out of the box)

created /etc/cron.d/ntpcheck, containing:
@reboot root PATH="$PATH:/home/pi/" /home/pi/ntpcheck.sh

created /home/pi/ntpcheck.sh, containing:

#!/bin/bash
echo "Waiting for Time Synchronization (NTP)..."
if [[ $EUID -ne 0 ]]; then
# you are NOT root (not @reboot), no sleep, system already initialized...
:
else
# you are root (@reboot), sleep, let the system initialize...
sleep 60s
fi
/usr/sbin/ntp-wait -s 10
RETVAL=$?

if [ "$RETVAL" != "0" ];then
echo "Time NOT synchronized (NTP)!!!"
exit 1
else
echo "Time synchronized established (NTP)."
/bin/kill -HUP $(ps -e | grep 'dnsmasq' | awk '{print $1}')
fi

remarks:

  • the sleep function is absolutely required, the system seems NOT ready if you remove it, the script simply fails, not even an email will be sent, as the mail system cannot find the domain.
    Jan 31 18:02:53 raspberrypi sSMTP[504]: Unable to locate smtp.gmail.com
    Jan 31 18:02:53 raspberrypi sSMTP[504]: Cannot open smtp.gmail.com:587
  • according to the man page, ntp-wait retries a 1000 times, the -s parameter specifies there is a 10 second delay between each attempt.
  • neither the ntp deamon, nor ntp-wait, send a SIGHUP signal, that would trigger dnsmasq, so we have to generate it, using kill.

Now we analyze the pihole log:
at boot time, dnsmasq loads, generating the following messages:

Jan 31 18:29:32 dnsmasq[631]: started, version 2.76 cachesize 10000
Jan 31 18:29:32 dnsmasq[631]: compile time options: IPv6 GNU-getopt DBus i18n IDN DHCP DHCPv6 no-Lua TFTP conntrack ipset auth DNSSEC loop-detect inotify
Jan 31 18:29:32 dnsmasq[631]: DNSSEC validation enabled
Jan 31 18:29:32 dnsmasq[631]: DNSSEC signature timestamps not checked until first cache reload
Jan 31 18:29:32 dnsmasq[631]: warning: ignoring resolv-file flag because no-resolv is set
Jan 31 18:29:32 dnsmasq[631]: using nameserver 127.10.10.4#5554
Jan 31 18:29:32 dnsmasq[631]: using nameserver 127.10.10.3#5553
Jan 31 18:29:32 dnsmasq[631]: using nameserver 127.10.10.2#5552
Jan 31 18:29:32 dnsmasq[631]: using nameserver 127.10.10.1#5551
Jan 31 18:29:32 dnsmasq[631]: read /etc/hosts - 9 addresses
Jan 31 18:29:32 dnsmasq[631]: read /etc/pihole/local.list - 2 addresses
Jan 31 18:29:42 dnsmasq[631]: read /etc/pihole/gravity.list - 599517 addresses

In the mean time, while we're waiting, run ps aux | grep sh, you will notice the script is in the list twice; the first reference comes from cron, the second one is the actual running script.

As the script finally completes (nervously waiting - receiving a mail with the expected echo messages), time is synchronized and the SIGHUP signal is sent, resulting in the following messages in the pihole log:

Jan 31 18:45:15 dnsmasq[631]: now checking DNSSEC signature timestamps
Jan 31 18:45:16 dnsmasq[631]: read /etc/hosts - 9 addresses
Jan 31 18:45:16 dnsmasq[631]: read /etc/pihole/local.list - 2 addresses
Jan 31 18:45:25 dnsmasq[631]: read /etc/pihole/gravity.list - 599517 addresses

As you can see, the SIGHUP signal also triggers a reload of the various lists.

My concerns, regarding DNSSEC remain, as my log is filled with INSECURE (pages do load), ABANDONED (pages don't load), BOGUS (pages don't load). I only saw SECURE a few times and can't even replicate this.

Another concern of mine are the strange replies, that keep coming up, notice the uppercase character in the domain name. Examples (some of many):
Jan 31 18:50:58 dnsmasq[631]: reply aus5.mozilla.orG is
Jan 31 19:23:00 dnsmasq[631]: reply ocsp.int-x3.letsencrypt.orG is
Jan 31 19:26:09 dnsmasq[631]: reply cdn.bizible.coM is
Jan 31 19:26:09 dnsmasq[631]: reply www.cloudflare.coM is 198.41.215.162

The strange thing here is: when using dig, the domain is resolved as expected
pi@raspberrypi:~ $ dig www.cloudflare.com @127.10.10.3 -p 5553

; <<>> DiG 9.9.5-9+deb8u9-Raspbian <<>> www.cloudflare.com @127.10.10.3 -p 5553
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 25162
;; flags: qr rd ra ad; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;www.cloudflare.com. IN A

;; ANSWER SECTION:
www.cloudflare.com. 15 IN A 198.41.214.162
www.cloudflare.com. 15 IN A 198.41.215.162

;; Query time: 54 msec
;; SERVER: 127.10.10.3#5553(127.10.10.3)
;; WHEN: Tue Jan 31 19:58:19 CET 2017
;; MSG SIZE rcvd: 79

the output is identical when using dig www.cloudflare.com @127.0.0.1 -p 53

Eagerly awaiting your thoughts...

13

Thanks for all your in-depth research on this topic. However, please understand that we don't want to go through the hassle of implementing scripts that do all the things you describe in your various posts (including manual downloading, compiling and installing the recent version of dnsmasq).

However, our strategy will be different: We will implement a clear warning for users enabling DNSSEC from the web UI Settings page. It should be clear to them that DNSSEC neither widely supported nor expected to flawlessly work just out-of-the-box.

14

I am continuing my investigation, currently mailing with the author of dnsmasq, regarding DNSSEC issues.

I would like to know where the dnsmasq restart instructions are located (when changing a configuration, add a whitelist or blacklist, add or remove a wildcard or updategravity), this to implement ntpcheck (updated - see above) for personal use only, so this is not a request to implement this in pi-hole.

15

Look at advanced/Scripts/webpage.sh