Pi-hole v3.3 Released: It's "Extra" Special

Originally published at: https://pi-hole.net/2018/02/14/pi-hole-v3-3-released-its-extra-special/

Update 2018-02-20 18:05


Hi All, After a few days of pulling out our hair and troubleshooting this whitelisting issue that some of you have reported, we're finally getting to the bottom of it.

The good news is, whitelisting is not completely broken. You can still whitelist domains from the cli with no issues by calling pihole -w [domain-to-whitelist]. The issue only affects whitelisting from the admin page (whitelist page, query log, and block page).

Take a look over this pull request where /u/promofaux has attempted to explain what is going on. Though, we're a bit confused ourselves, and any insight from the community would be greatly appreciated!

There are a couple of options, we can either revert the change that broke it, or use the fix in the above pull request. Whichever way we go, rest assured that we are working hard internally to make sure that we have the bug well and truly squashed, and will try to get a fix out as soon as we can (and really, take that soon™ in the Blizzard sense of the word).

In the mean time, do not attempt to whitelist from the web admin, it wont work... apologies for any inconvenience this causes.

In other news, we have updated the to include instructions on how you may possibly be able to update your version of dnsmasq to be able to update to Pi-hole 3.3

Update 2018-02-18 06:12

If you're running Raspbian Jessie and you updated Pi-hole to v3.3, you likely ran into issues. This is because the version of dnsmasq that ships with it does not support the log-queries=extra option, which we use in v3.3.

You have two options to resolve this: revert Pi-hole to a previous version or upgrade dnsmasq manually.

Option one: downgrade Pi-hole to the previous version

Instructions for this can be found here.

Option two: install the version of dnsmasq that supports the extra flag (v2.76)

Please note, you should only try this on Rasbpian Jessie and do so at your own risk (but in our opinion the risk is low)

First step: Download more recent version of dnsmasq compiled for Raspbian Jessie from the official sources

wget https://archive.raspberrypi.org/debian/pool/main/d/dnsmasq/dnsmasq-base_2.76-5+rpi1_armhf.deb
wget https://archive.raspberrypi.org/debian/pool/main/d/dnsmasq/dnsmasq_2.76-5+rpi1_all.deb

Second step: Ensure requirements are fulfilled

sudo apt-get install libnetfilter-conntrack3 libmnl0

Third step: Install downloaded packages

sudo dpkg -i dnsmasq-base_2.76-5+rpi1_armhf.deb
sudo dpkg -i dnsmasq_2.76-5+rpi1_all.deb

Fourth step: Verify it worked:

dnsmasq -v

should return:

Dnsmasq version 2.76  Copyright (c) 2000-2016 Simon Kelley
Compile time options: IPv6 GNU-getopt DBus i18n IDN DHCP DHCPv6 no-Lua TFTP conntrack ipset auth DNSSEC loop-detect inotify

You should now be able to use Pi-hole v3.3 on Raspbian Jessie.

Update 2018-02-14 18:43 (version issues and not working after update)

If you're running Rasbian Jessie, your version of dnsmasq will not work with this release, so you'll need to revert to the previous versions:
cd /etc/.pihole
sudo git fetch --tags
sudo git checkout v3.2.1
cd /var/www/html/admin
sudo git fetch --tags
sudo git checkout v3.2.1
pihole -r
pihole checkout ftl v2.13.2

The Release


This release takes full advantage of dnsmasq's extra logging feature, which means you'll get 100% accurate log analysis. This release also includes full DNSSEC support, Teleporter enhancements, several important security fixes, as well as some other tweaks. This blog post will focus on the main features of this release, but if you want a detailed breakdown, the full changelogs can always be found at changes.pi-hole.net.

100% Accurate Logs

We bumped FTL to v3.0 because it's even faster and will now interpret dnsmasq's log files with 100% accuracy. This is a good thing, but if you have custom scripts or any thing else dealing with the log file, it's important you know about this change before updating. We are enabling one of dnsmasq's additional options, which changes the way we have previously written to the logs.

Under the hood, we are enabling --log-queries=extra, which provides more information in the log files:

-q, --log-queries
Log the results of DNS queries handled by dnsmasq. Enable a full cache dump on receipt of SIGUSR1. 
If the argument "extra" is supplied, ie --log-queries=extra then the log has extra information at the start of each line. 
This consists of a serial number which ties together the log lines associated with an individual query, and the IP address of the requestor.

Why Weren't The Logs 100% Accurate Before This?

For performance, we have been using (and will continue to use) asynchronous logging (--log-async), which has been the default on Pi-hole for some time. Since Pi-hole began on low power devices like the Raspberry Pi, this feature prevented the DNS server from locking up if it was trying to process and log a lot of traffic at the same time.

A side effect of this was that the the response to the query was not always chronologically logged. It didn't happen that often and FTL was still able to decipher it quite well. So if you still want to run an older version of Pi-hole, the FTL engine is still 99.9% accurate.

The extra logging feature was not enabled before because until now many distributions did not have a new enough dnsmasq to support it.

Using the extra logging feature, each log entry has a unique identifier, which lets us match up the logged response and query. For the average user, it's not even something they would likely notice.

We'll still be using the asynchronous logging for it's performance benefits, and we'll let FTL handle the parsing of the log since we can keep track of it better now.

What Should I Know About The New Logging?

  • After updating, the old-style log files will not be readable by FTL
  • If you have previously disabled the database, you must re-enable it for at least 24 hours of history before updating if you want to see that day's previous stats (this should only affect a small percentage of users)
  • The last 24 hours of stats are read directly from the database now and not pihole.log.1 (this is where we snagged some extra performance from)

Disable Logging Without Flushing

We have also added a button to disable logging without flushing the log files.

Full DNSSEC Support

If your version of dnsmasq doesn't support DNSSEC or was compiled without it, this won't work, but most newer versions of the package should work fine with it enabled.

You will also now see a DNSSEC column in the query log, which will display the status of individual queries (if enabled).

Teleporter Enhancements

Teleporter will now export your Audit Log.

Security Fixes And Other Notes

We'd like to thank Denis Andzakovic for notifying us about some security vulnerabilities, which are now fixed in this release.

The cosmetic version issues on the Web interface should be fixed now.

We were previously using .local as a fallback TLD for DHCP generated domains. We changed this to .lan so as not to conflict with Multicast DNS.

Contributing

If you're interested in helping contribute to Pi-hole, there are several ways to help. One roadblock new users had when trying to submit pull requests, was to decipher the space/black hole-themed function names; we have renamed these in the codebase to help make it easier to understand.

Stickers!

Many of you have been asking if we have stickers for sale. Not currently, but if you upvote our request here, they will become available on for sale Unixstickers.com (and we'll get a portion of the sales to help further development of Pi-hole).

Also, if you're presenting about Pi-hole or attending some other event where Pi-hole will be discussed, contact us and we'd be happy to send you some.

1 Like

Getting 404 when pihole try to update FTL

Thanks, it should be fixed now.

1 Like

7 posts were split to a new topic: Domains blocked number decreased/dnsmasq not showing queries

Note that log-queries=extra will not work on all platforms, yes I know it is for the Pi but just FYI.

# service dnsmasq start
[....] Starting DNS forwarder and DHCP server: dnsmasq
dnsmasq: extraneous parameter at line 37 of /etc/dnsmasq.d/01-pihole.conf
 failed!
1 Like

A post was split to a new topic: Error: Unable to get latest release location from GitHub

So, if we went back to the earlier version, will the interface advice us when a new version is available? or we gonna be stuck in the vDev version?

When we have something confirmed working we will make an announcement here and on our other platforms.

1 Like

I did the vDev version fix and now i see that there is a way to manually upgrade. I tried changing the versions in the steps but am i stuck in vDev versions? How can i go back after manually upgrading dnsmasq?

I am having the same issue, did you find a way to do an update back to live?

Just waiting for @DanSchaper announcement as he said to do a manually upgrade to official release with the fix.
At the moment i'm staying in Dev (HEAD, v3.2.1-0-ge602008)

I expect we will have instructions when the bugfix release is out. However, running pihole checkout master will allow you to update (and if you already tried updating to 3.3, this will checkout out 3.3).

2 Likes

A post was split to a new topic: Unable to update web interface v3.3

Just FYI: Raspbian Jessie user on Pihole 3.2. I waited with updating and just came back to check status on how to move to 3.3. Read the post on how to update dnsmasq. Started by trying "dnsmasq -v" to see which version I have now, so I can later validate updating it to 2.76 but to my surprise it is already reported as being 2.76. And I surely haven't done any of the steps in that guide.

I am having dnsmasq-troubles on my Synology DS-413j, which is obviously running a Jessie-Debian on "armel-architecture".

Consequently this will fail to install.
I'm a completely ignorant Linux-Dummie and have not the slightest grasp of what I'm doing here.
I have google for dnsmasq-base 2.76-5 and armel, but have not found anything.

On
https://archive.raspberrypi.org/debian/pool/main/d/dnsmasq/
there is no arm-thingy of any sort except the "armhf", which is not compatible.

Somwhow I have managed to remove dnsmasq completey and can't find a way to install it again.

curl -L https://install.pi-hole.net | bash

is not installing it, either.

Any hint on how to get basic functionality of pihole again until your new version with ftldns will be final?

Unless you can find at least version 2.73 of dnsmasq for Debian Jessie, you should update to Debian Stretch or downgrade to Pi-hole 3.2.1

Thanks a lot, it was a headache why pi-hole not working with OMV.

Hello,
It's been some time being on 3.2.1 and I cannot update any further on the RiPi 3. Is pihole done for RiPi 3 now or still working on a proper fix where everything runs and install successfully

We are working on 4.0 and have had that in beta for a bit. The 4.0 release will fix the inconsistencies with dnsmasq. I'm not sure why you are having problems with the RPI 3 as the problem is with Raspbian Jessie, with Raspbian Stretch you can run 3.3 on the RPi 3?