Feedback for "Allow defining clients by their MAC address, host name and networking interface"

Yesterday evening I updated the FTL-branch to the most recent version, and... it worked again! FTL now listens on for IPv4 TCP DNS requests. I did not work after the last update (the fix to the locking-problem), or after repeated new setups from the scratch. I am bit puzzled.

For the sake of completeness this is (and was before performing the latest update) the content of /etc/dnsmasq.d/01-pihole.conf:

addn-hosts=/etc/pihole/local.list
addn-hosts=/etc/pihole/custom.list


localise-queries


no-resolv



cache-size=10000

log-queries
log-facility=/var/log/pihole.log

local-ttl=2

log-async




server=8.8.8.8
server=8.8.4.4
interface=ens3
server=/use-application-dns.net/

No other files are in this directory.

It worked quite well until just now, when FTL crashed:

[2020-06-25 13:00:01.340 4838] !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
[2020-06-25 13:00:01.340 4838] ---------------------------->  FTL crashed!  <----------------------------
[2020-06-25 13:00:01.340 4838] !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
[2020-06-25 13:00:01.340 4838] Please report a bug at https://github.com/pi-hole/FTL/issues
[2020-06-25 13:00:01.340 4838] and include in your report already the following details:
[2020-06-25 13:00:01.340 4838] FTL has been running for 46876 seconds
[2020-06-25 13:00:01.340 4838] FTL branch: new/mac_clients
[2020-06-25 13:00:01.340 4838] FTL version: vDev-73473ac
[2020-06-25 13:00:01.340 4838] FTL commit: 73473ac
[2020-06-25 13:00:01.340 4838] FTL date: 2020-06-24 20:44:04 +0200
[2020-06-25 13:00:01.340 4838] FTL user: started as pihole, ended as pihole
[2020-06-25 13:00:01.340 4838] Compiled for x86_64 (compiled on CI) using gcc (Debian 6.3.0-18+deb9u1) 6.3.0 20170516
[2020-06-25 13:00:01.340 4838] Received signal: Segmentation fault
[2020-06-25 13:00:01.340 4838]      at address: 0x18
[2020-06-25 13:00:01.340 4838]      with code: SEGV_MAPERR (Address not mapped to object)
[2020-06-25 13:00:01.341 4838] Obtained lock for _FTL_new_query() (/root/project/src/dnsmasq_interface.c:480)
[2020-06-25 13:00:01.341 4838] **** new UDP query[AAAA] "jenson.api.swiftkey.com" from ens3:fd00::c1a6:8f8e:24f:ff4 (ID 16366, FTL 22494, /root/project/src/dnsmasq/forward.c:1556)
[2020-06-25 13:00:01.341 4838] getOverTimeID(1593083100): 138
[2020-06-25 13:00:01.341 4838] Adding "jenson.api.swiftkey.com" (len 24) to buffer. next_str_pos is 56439
[2020-06-25 13:00:01.341 4838] jenson.api.swiftkey.com is not known
[2020-06-25 13:00:01.341 4838] Getting sqlite3_stmt** 0x558a8c3f6de0[15] --> 0x558a8c3fdf18
[2020-06-25 13:00:01.342 4838] domain_in_list("jenson.api.swiftkey.com", 0x558a8c3fdf18, whitelist): 0
[2020-06-25 13:00:01.342 4838] Getting sqlite3_stmt** 0x558a8c3f6e10[15] --> 0x558a8c3feb18
[2020-06-25 13:00:01.342 4838] domain_in_list("jenson.api.swiftkey.com", 0x558a8c3feb18, blacklist): 0
[2020-06-25 13:00:01.342 4838] Getting sqlite3_stmt** 0x558a8c401480[15] --> 0x558a8c3fe688
[2020-06-25 13:00:01.361 4838] domain_in_list("jenson.api.swiftkey.com", 0x558a8c3fe688, gravity): 0
[2020-06-25 13:00:01.362 4838] Removed lock in _FTL_new_query() (/root/project/src/dnsmasq_interface.c:622)
[2020-06-25 13:00:01.362 4838] Waiting for lock in _FTL_forwarded() (/root/project/src/dnsmasq_interface.c:709)
[2020-06-25 13:00:01.362 4838] Obtained lock for _FTL_forwarded() (/root/project/src/dnsmasq_interface.c:709)
[2020-06-25 13:00:01.362 4838] **** forwarded jenson.api.swiftkey.com to 9.9.9.9 (ID 16366, /root/project/src/dnsmasq/forward.c:549)
[2020-06-25 13:00:01.362 4838] Removed lock in _FTL_forwarded() (/root/project/src/dnsmasq_interface.c:815)
[2020-06-25 13:00:01.372 4838] Backtrace:
[2020-06-25 13:00:01.373 4838] B[0000]: 0x558a8bcdf2c9, /usr/bin/pihole-FTL(+0x342c9) [0x558a8bcdf2c9]
[2020-06-25 13:00:01.389 4838] B[0001]: 0x7febddfe3730, /lib/x86_64-linux-gnu/libpthread.so.0(+0x12730) [0x7febddfe3730]
[2020-06-25 13:00:01.389 4838] B[0002]: 0x558a8bd64151, /usr/bin/pihole-FTL(+0xb9151) [0x558a8bd64151]
[2020-06-25 13:00:01.392 4838] B[0003]: 0x558a8bde7f7c, /usr/bin/pihole-FTL(+0x13cf7c) [0x558a8bde7f7c]
[2020-06-25 13:00:01.396 4838] B[0004]: 0x558a8bde8f39, /usr/bin/pihole-FTL(+0x13df39) [0x558a8bde8f39]
[2020-06-25 13:00:01.399 4838] B[0005]: 0x558a8bd4e0e7, /usr/bin/pihole-FTL(+0xa30e7) [0x558a8bd4e0e7]
[2020-06-25 13:00:01.403 4838] B[0006]: 0x558a8bd4eaeb, /usr/bin/pihole-FTL(+0xa3aeb) [0x558a8bd4eaeb]
[2020-06-25 13:00:01.406 4838] B[0007]: 0x558a8bdb9b9b, /usr/bin/pihole-FTL(+0x10eb9b) [0x558a8bdb9b9b]
[2020-06-25 13:00:01.416 4838] B[0008]: 0x558a8bdcf5bf, /usr/bin/pihole-FTL(+0x1245bf) [0x558a8bdcf5bf]
[2020-06-25 13:00:01.419 4838] B[0009]: 0x558a8bdd417f, /usr/bin/pihole-FTL(+0x12917f) [0x558a8bdd417f]
[2020-06-25 13:00:01.423 4838] Waiting for lock in _FTL_dnssec() (/root/project/src/dnsmasq_interface.c:1364)
[2020-06-25 13:00:01.423 4838] Obtained lock for _FTL_dnssec() (/root/project/src/dnsmasq_interface.c:1364)
[2020-06-25 13:00:01.423 4838] **** got DNSSEC details for jenson.api.swiftkey.com: 2 (ID 16366, /root/project/src/dnsmasq/forward.c:1237)
[2020-06-25 13:00:01.423 4838] Removed lock in _FTL_dnssec() (/root/project/src/dnsmasq_interface.c:1404)
[2020-06-25 13:00:01.423 4838] Waiting for lock in _FTL_reply() (/root/project/src/dnsmasq_interface.c:869)
[2020-06-25 13:00:01.423 4838] Obtained lock for _FTL_reply() (/root/project/src/dnsmasq_interface.c:869)
[2020-06-25 13:00:01.423 4838] **** got reply jenson.api.swiftkey.com is (NODATA) (ID 16366, /root/project/src/dnsmasq/cache.c:480)
[2020-06-25 13:00:01.423 4838]      Flags: F_FORWARD F_NEG F_IPV6 
[2020-06-25 13:00:01.423 4838] Removed lock in _FTL_reply() (/root/project/src/dnsmasq_interface.c:1019)
[2020-06-25 13:00:01.425 4838] B[0010]: 0x558a8bdd46d5, /usr/bin/pihole-FTL(+0x1296d5) [0x558a8bdd46d5]
[2020-06-25 13:00:01.428 4838] Waiting for lock in _FTL_new_query() (/root/project/src/dnsmasq_interface.c:480)
[2020-06-25 13:00:01.429 4838] Obtained lock for _FTL_new_query() (/root/project/src/dnsmasq_interface.c:480)
[2020-06-25 13:00:01.429 4838] **** new UDP query[AAAA] "jenson.api.swiftkey.com" from ens3:192.168.2.105 (ID 16367, FTL 22495, /root/project/src/dnsmasq/forward.c:1549)
[2020-06-25 13:00:01.429 4838] getOverTimeID(1593083100): 138
[2020-06-25 13:00:01.429 4838] jenson.api.swiftkey.com is not known
[2020-06-25 13:00:01.429 4838] Getting sqlite3_stmt** 0x558a8c3f6de0[4] --> 0x558a8c3e5258
[2020-06-25 13:00:01.429 4838] domain_in_list("jenson.api.swiftkey.com", 0x558a8c3e5258, whitelist): 0
[2020-06-25 13:00:01.429 4838] Getting sqlite3_stmt** 0x558a8c3f6e10[4] --> 0x558a8c3e5068
[2020-06-25 13:00:01.429 4838] domain_in_list("jenson.api.swiftkey.com", 0x558a8c3e5068, blacklist): 0
[2020-06-25 13:00:01.429 4838] Getting sqlite3_stmt** 0x558a8c401480[4] --> 0x558a8c512688
[2020-06-25 13:00:01.429 4838] domain_in_list("jenson.api.swiftkey.com", 0x558a8c512688, gravity): 0
[2020-06-25 13:00:01.429 4838] Removed lock in _FTL_new_query() (/root/project/src/dnsmasq_interface.c:622)
[2020-06-25 13:00:01.431 4838] **** got cache answer for jenson.api.swiftkey.com /  / <unknown> (ID 16367, /root/project/src/dnsmasq/rfc1035.c:1797)
[2020-06-25 13:00:01.431 4838]      Flags: F_FORWARD F_NEG F_IPV6 
[2020-06-25 13:00:01.431 4838] Waiting for lock in _FTL_cache() (/root/project/src/dnsmasq_interface.c:1221)
[2020-06-25 13:00:01.431 4838] Obtained lock for _FTL_cache() (/root/project/src/dnsmasq_interface.c:1221)
[2020-06-25 13:00:01.431 4838] Removed lock in _FTL_cache() (/root/project/src/dnsmasq_interface.c:1314)
[2020-06-25 13:00:01.433 4838] B[0011]: 0x558a8bdd481a, /usr/bin/pihole-FTL(sqlite3_prepare_v2+0x1a) [0x558a8bdd481a]
[2020-06-25 13:00:01.434 4838] Waiting for lock in _FTL_new_query() (/root/project/src/dnsmasq_interface.c:480)
[2020-06-25 13:00:01.434 4838] Obtained lock for _FTL_new_query() (/root/project/src/dnsmasq_interface.c:480)
[2020-06-25 13:00:01.434 4838] **** new UDP query[AAAA] "jenson.api.swiftkey.com" from ens3:fd00::c1a6:8f8e:24f:ff4 (ID 16368, FTL 22496, /root/project/src/dnsmasq/forward.c:1556)
[2020-06-25 13:00:01.434 4838] getOverTimeID(1593083100): 138
[2020-06-25 13:00:01.434 4838] jenson.api.swiftkey.com is known as not to be blocked
[2020-06-25 13:00:01.434 4838] Removed lock in _FTL_new_query() (/root/project/src/dnsmasq_interface.c:622)
[2020-06-25 13:00:01.435 4838] **** got cache answer for jenson.api.swiftkey.com /  / <unknown> (ID 16368, /root/project/src/dnsmasq/rfc1035.c:1797)
[2020-06-25 13:00:01.435 4838]      Flags: F_FORWARD F_NEG F_IPV6 
[2020-06-25 13:00:01.435 4838] Waiting for lock in _FTL_cache() (/root/project/src/dnsmasq_interface.c:1221)
[2020-06-25 13:00:01.435 4838] Obtained lock for _FTL_cache() (/root/project/src/dnsmasq_interface.c:1221)
[2020-06-25 13:00:01.435 4838] Removed lock in _FTL_cache() (/root/project/src/dnsmasq_interface.c:1314)
[2020-06-25 13:00:01.437 4838] Waiting for lock in _FTL_new_query() (/root/project/src/dnsmasq_interface.c:480)
[2020-06-25 13:00:01.437 4838] Obtained lock for _FTL_new_query() (/root/project/src/dnsmasq_interface.c:480)
[2020-06-25 13:00:01.437 4838] **** new UDP query[AAAA] "jenson.api.swiftkey.com" from ens3:192.168.2.105 (ID 16369, FTL 22497, /root/project/src/dnsmasq/forward.c:1549)
[2020-06-25 13:00:01.437 4838] getOverTimeID(1593083100): 138
[2020-06-25 13:00:01.437 4838] jenson.api.swiftkey.com is known as not to be blocked
[2020-06-25 13:00:01.438 4838] Removed lock in _FTL_new_query() (/root/project/src/dnsmasq_interface.c:622)
[2020-06-25 13:00:01.439 4838] **** got cache answer for jenson.api.swiftkey.com /  / <unknown> (ID 16369, /root/project/src/dnsmasq/rfc1035.c:1797)
[2020-06-25 13:00:01.439 4838]      Flags: F_FORWARD F_NEG F_IPV6 
[2020-06-25 13:00:01.439 4838] Waiting for lock in _FTL_cache() (/root/project/src/dnsmasq_interface.c:1221)
[2020-06-25 13:00:01.439 4838] Obtained lock for _FTL_cache() (/root/project/src/dnsmasq_interface.c:1221)
[2020-06-25 13:00:01.439 4838] Removed lock in _FTL_cache() (/root/project/src/dnsmasq_interface.c:1314)
[2020-06-25 13:00:01.440 4838] B[0012]: 0x558a8bcecf0f, /usr/bin/pihole-FTL(getDatabaseHostname+0x19f) [0x558a8bcecf0f]
[2020-06-25 13:00:01.442 4838] Waiting for lock in _FTL_new_query() (/root/project/src/dnsmasq_interface.c:480)
[2020-06-25 13:00:01.442 4838] Obtained lock for _FTL_new_query() (/root/project/src/dnsmasq_interface.c:480)
[2020-06-25 13:00:01.442 4838] **** new UDP query[A] "jenson.api.swiftkey.com" from ens3:fd00::c1a6:8f8e:24f:ff4 (ID 16370, FTL 22498, /root/project/src/dnsmasq/forward.c:1556)
[2020-06-25 13:00:01.442 4838] getOverTimeID(1593083100): 138
[2020-06-25 13:00:01.443 4838] jenson.api.swiftkey.com is known as not to be blocked
[2020-06-25 13:00:01.443 4838] Removed lock in _FTL_new_query() (/root/project/src/dnsmasq_interface.c:622)
[2020-06-25 13:00:01.443 4838] Waiting for lock in _FTL_forwarded() (/root/project/src/dnsmasq_interface.c:709)
[2020-06-25 13:00:01.443 4838] Obtained lock for _FTL_forwarded() (/root/project/src/dnsmasq_interface.c:709)
[2020-06-25 13:00:01.443 4838] **** forwarded jenson.api.swiftkey.com to 9.9.9.9 (ID 16370, /root/project/src/dnsmasq/forward.c:549)
[2020-06-25 13:00:01.443 4838] Removed lock in _FTL_forwarded() (/root/project/src/dnsmasq_interface.c:815)
[2020-06-25 13:00:01.448 4838] B[0013]: 0x558a8bcdc37b, /usr/bin/pihole-FTL(+0x3137b) [0x558a8bcdc37b]
[2020-06-25 13:00:01.451 4838] B[0014]: 0x558a8bcdc55d, /usr/bin/pihole-FTL(resolveClients+0x12d) [0x558a8bcdc55d]
[2020-06-25 13:00:01.455 4838] B[0015]: 0x558a8bcdc88d, /usr/bin/pihole-FTL(DNSclient_thread+0xdd) [0x558a8bcdc88d]
[2020-06-25 13:00:01.460 4838] B[0016]: 0x7febddfd8fa3, /lib/x86_64-linux-gnu/libpthread.so.0(+0x7fa3) [0x7febddfd8fa3]
[2020-06-25 13:00:01.460 4838] B[0017]: 0x7febddf094cf, /lib/x86_64-linux-gnu/libc.so.6(clone+0x3f) [0x7febddf094cf]
[2020-06-25 13:00:01.460 4838] ------ Listing content of directory /dev/shm ------
[2020-06-25 13:00:01.460 4838] File Mode User:Group  Filesize Filename
[2020-06-25 13:00:01.461 4838] rwxrwxrwx root:root 260 .
[2020-06-25 13:00:01.461 4838] rwxr-xr-x root:root 3K ..
[2020-06-25 13:00:01.461 4838] rw------- pihole:pihole 4K FTL-per-client-regex
[2020-06-25 13:00:01.461 4838] rw------- pihole:pihole 33K FTL-dns-cache
[2020-06-25 13:00:01.461 4838] rw------- pihole:pihole 12K FTL-overTime
[2020-06-25 13:00:01.461 4838] rw------- pihole:pihole 2M FTL-queries
[2020-06-25 13:00:01.461 4838] rw------- pihole:pihole 4K FTL-upstreams
[2020-06-25 13:00:01.462 4838] rw------- pihole:pihole 340K FTL-clients
[2020-06-25 13:00:01.462 4838] rw------- pihole:pihole 98K FTL-domains
[2020-06-25 13:00:01.462 4838] rw------- pihole:pihole 57K FTL-strings
[2020-06-25 13:00:01.462 4838] rw------- pihole:pihole 12 FTL-settings
[2020-06-25 13:00:01.462 4838] rw------- pihole:pihole 124 FTL-counters
[2020-06-25 13:00:01.462 4838] rw------- pihole:pihole 48 FTL-lock
[2020-06-25 13:00:01.462 4838] ---------------------------------------------------
[2020-06-25 13:00:01.462 4838] Thank you for helping us to improve our FTL engine!
[2020-06-25 13:00:01.462 4838] FTL terminated!

A pihole restardns made everything functional again.

Yep, I think I found where the issue was, and have prepared a fix for this here:

1 Like

Is it still running? We're currently haunting another issue with incorrect database pointers when a massive number of TCP clients is spawned. The new/mac_client branch may not be fully up-to-date with respect to development so this may have already been fixed.

Yes, still running since 2 days now. Maybe it was an unrelated glitch. It seems my issues are resolved for now. I will continue using new/mac_client and report new issues.

For now, thank you for your help and excellent work!

1 Like

Hi,
As version 5.1.1 is out and there are some bugs already fixed in development branch, i wanted to ask you to update the branch with latest commits?
Especially updating of DNS settings and conditional forwarding is broken at the moment within this branch.

Thanks for your hard work! I like this project very much and as dual-stack user this branch is very useful (happy wife, happy life :smile: )

:slight_smile:

I second this request.

See

Thanks for the feedback and hard work, we will wait for it :smile:

Currently I am on:

pi@pihole:~ $ pihole -v
  Pi-hole version is development v5.0-134-g60f0ab51 (Latest: v5.1.1)
  AdminLTE version is new/mac_clients v5.0-403-ga901489a (Latest: v5.1)
  FTL version is new/mac_clients vDev-661dad9 (Latest: v5.1)

Am I understanding that now the best way to keep up with this version to switch over to the dev branch? Is that by doing:

pihole checkout dev

or do I need to do each component:

pihole checkout core dev
pihole checkout web dev
pihole checkout ftl dev

Or should I stick on this branch for now? I know there was DB changes with this, so don't want to corrupt things by switching out of this branch prematurely.

It has not been merged now. I'm sure, DL6ER will post a message here when he merged the branches into development.

For now you should stay on this branch if you don't want to loose the ability to define clients by MAC, hostname, etc. and want to avoid to delete your long-term-database.

This two will be sufficient.

Current status is: Waiting for FTL#845 to get merged.

Then I will update all the features branches floating around. And I will also mark them as ready-to-be-merged into development.

Depending on the team's review abilities this may happen within minutes to hours to days. I hope that we can release FTL v5.1.1 tomorrow which also means you will get all the updated branches tomorrow.

I could already do it now, however, there are two drawbacks:

  1. I have double amount of work (I don't care too much about this, but see no. 2)
  2. There would be a needless extra merge commit. It is just cleaner to wait for release/v5.1.1 to be done in order to reduce the number of merge commits floating around. They are just noise. However,. given the complexity of the currently discussed branches, a rebase would result in even (much) more work than an ordinary merge which is why I'll prefer the latter.

I hope this makes my motivation clearer. It's not about me simply not wanting to do it.

Sorry if my post came across in any way other than just seeking clarification. I totally understand that in the world of software development, things take time. I also get that cleaner is better, and would wish extra work on no one.

I do want to take a moment to thank you (and the rest of the team) on putting this all together. Specifically this feature is a really great one, that is helping to fine tune my setup for the different use cases in my household.

2 Likes

I don't think your post comes across as demanding in any way. Re-reading his reply, I don't think he received it this way, either, so all good.

There seems to be a lot of work going on behind the scenes. They added another optimization yesterday and still have one more bugfix prepared and even an update for the dnsmasq core to the (today released!!) dnsmasq v2.82.

How cool it this!

Also, @DanSchaper commented that the update of dnsmasq will push FTL to v5.2 so v5.1.1 will maybe be skipped. We can stay tuned for more awesome Pi-hole things to come.

This feature branch (new/mac_clients) has been updated to the latest version of FTL (release/v5.2).

@DL6ER Thank you very much! Will the AdminLTE also be updated so changes (for example Conditional Forwarding on DNS page) can be updated?

Thanks for your hard work :slight_smile:

Yes, however, the AdminLTE branch is still awaiting two more PRs to be reviewed + merged, first.

Alright, thank you very much for your fast reply :slight_smile:

PRs have been merged, so I just updated the web branch as well now. Please test if adding custom IPs still works, I had to resolve some merge conflicts and I did so with a "best efforts" approach. I'm limited to my mobile development system right now so I cannot really do much to test my changes. If adding custom IPs in the dropdown select menu still works (Groups -> Clients page), then everything went well.

Just tried it, seems to work! Thank you very much :slight_smile:

Thank you very much for pushing the new release to this. pihole is working exceedingly well now. From my experience testing release/5.2, it seems to fix alot of stuff.