Feedback for "Allow defining clients by their MAC address, host name and networking interface"

This is still a complex task to do because we have to communicate this through forks as well when there are TCP requests. It is by no means trivial, except for when we do this once per minute for every client.

It is not quite that, they are not necessarily able to contact whatever they want. They will still be member of the default group so whether gravity and which parts of the black and whitelist will apply for them depends on the individual user configuration.

Yes, I want things to work perfect. Anything else will trigger support requests and is hard to explain. We would, at least, have to put up a warning like: Note that MAC and host name client recognition may involve a certain delay and not be available instantaneously after connecting your devices.

Something along these lines.

However, I fee the need for more feedback also from others how this could get realized. And I'm interested both in technical as well as user point of views of this.

This is what I imagined. Would that create to much overhead to do it for all clients? But it might be less complex code to maintain.

Would be wise to put up that warning at the clients page.

I already try to invite users to this topic if they ask for that feature. Hopefully they will give some feedback as well.
I see no need to rush this feature... if it makes it to 5.2 or 5.3 is soon enough.

I am currently testing, I am in need of something like this as my Network rotates ipv6 address assignment constantly.

So Far, I am running into no problems, I did get a SQlite error when flushing the network table. seemed Minor though because everything appears to be working.

I am seeing this behavior as well. I am using Safari on OS X.

Unrelated to the MAC changes, but it appears that the theme settings are not saving for me (choosing light or dark). It changes when I hit save, but as soon as I navigate away, it reverts back to light theme. And if I go back to the settings, it is back on the default light setting.

I checked the settings conf file, and I don't see an entry being created for the theme settings. Changing the boxed layout works, and updates settings as expected. So it seems it is just the theme setting that isn't saving.

I ran into this problem while adding clients, I was at about 50 or so clients and a big red message popped up saying 1 of 0 Unique ID's, and FTL went down entirely. I had to restart it. After restarting the issue went away.

Also got this error.

root@raspberrypi:~# pihole restartdns reload-lists
  [✗] kill: usage: kill [-s sigspec | -n signum | -sigspec] pid | jobspec ... or kill -l [sigspec]

Other than what I just mentioned, I am currently running 68 clients linked by mac address (Pihole not acting as DHCP, instead conditional forwarding to the router with router handing out Pihole as acting lan DNS). All clients working properly with list respective list assigned to each one.

Would be good to post the error here so it can be fixed.

I see that too. It worked for a while but no it doesn't anymore. I also had it before using the current development branch

Tried switching back to

pihole checkout ftl development or master

but DNS resolution stopped after a few seconds.

[2020-05-17 16:26:00.228 4073] SQLite3 message: no such column: name in "SELECT name FROM network WHERE id = (SELECT network_id FROM network_addresses WHERE ip = ?);" (1)
[2020-05-17 16:26:00.228 4073] getDatabaseHostname("10.0.10.64") - SQL error prepare: SQL logic error

Switching to this branch and DNS resolution is restored.

new/mac_clients is not compatible with master or development, once you changed to new you committed to it.

Edit: Unless you delete the database used in new/mac_clients and restore from a backup or create a new one.

Thanks for the explanation.

You can go back.
Here are the steps that worked for me.
First I removed all mac listed clients off of my Clients List.

Then I used
pihole checkout master

This is where i noticed after the update
pihole would not start.

Then I edited /etc/resolv.conf and added

Nameserver 8.8.8.8

I used Pihole option

pihole -r

Then I ran

pihole arpflush

followed by

pihole flush

After pihole flush the pihole was up and running!

I did all of this with out having to use my backup.

Pihole is very resilient at repairing itself.

When you are done remove the arbitrary Nameserver from /etc/resolv.conf or reboot it will remove itself.

@DL6ER

Thanks for the workaround. I have no need to switch branches so I stick with the current.

I switched to using this on one of my other pi's. It seemed to me to use a little bit more memory than the initial 5.0 master branch, so I am running it on a faster pi. Did you experience any extra memory/cpu constraints using the mac_clients?

No, but I haven't paid attention. Pihole is the only software running on the device with only 15 clients. Memory and CPU was never a problem.

yea i have a ton of clients about 72. i don't think that should have caused an issue though as dns traffic is pretty basic.I wasn't using it for dhcp.

Yes.

I will check this.

Make sure you ran

pihole checkout development

as well

Can you give us the corresponding lines from /var/log/pihole-FTL.log (there should be a bug report + include a few lines from before the !!!!!! lines)?

Some more CPU cycles will be needed when

  • a new client is seen for the first time, and
  • when you do any group-related changes (add domains, adlists, clients, modify groups, etc.)

because every time we need to check the groups to be applied for the clients. And with more freedom (not only IPs but also MAC addresses and soon host names), more research has been done to be sure that a client is truly not configured by any means.
Outside of the two events above, you shouldn't see an effect.

I had a few minutes for coding and have hostnames as client identifier ready for FTL. I will push what I have, however, be prepared that the web interface may not work as expected in a the affected places (network overview and groups->clients) until I updated this as well. I'll report back.

1 Like

You are onto some great changes. So far so good with mac listings. Cant wait to test hostnames.

Just pushed the updates. I hope it works, two things which may need to be discussed:

  1. We should validate entered host names in some way. I chose the inverse way of looking for what should clearly not be in a host name. So far, this set includes
    < > ; "
    
    This will hopefully eliminate possible exploitation issues while still allowing a lot of stuff. Note that garbage in the database will not harm FTL so the validator doesn't need to be strict. It should just prevent users from being able to enter stuff that can hurt them (e.g., when they copy some Javascript code from a malicious page into this field).
  2. The dropdown suggests MAC addresses. For devices we're aware of but don't have the MAC address for (connected through VPN, etc.), we furthermore suggest IP address.
    So far, I don't suggest host names (I don't think we should do it).

I pushed a potential fix for this as well. It is a bit theoretical as - for a yet unknown reason - I wasn't able to reproduce this.

1 Like

Maybe I should point out again that setting DEBUG_CLIENTS=true in /etc/pihole/pihole-FTL.conf will log details for any known client. This may be helpful, especially when trying if this new feature is behaving as I designed and you expect it to work.

Example 1 (this client is not configured at all):

[2020-05-18 15:50:10.151 27808] Querying gravity database for client with IP 192.168.3.219...
[2020-05-18 15:50:10.151 27808] SQL: Comparing 192.168.3.219 vs. 192.168.0.101 (subnet 255.255.255.255) - NO MATCH
[2020-05-18 15:50:10.151 27808] SQL: Comparing 192.168.3.219 vs. 192.168.0.102 (subnet 255.255.255.255) - NO MATCH
[2020-05-18 15:50:10.151 27808] SQL: Comparing 192.168.3.219 vs. 192.168.0.103 (subnet 255.255.255.255) - NO MATCH
[2020-05-18 15:50:10.152 27808] --> No record for 192.168.3.219 in the client table
[2020-05-18 15:50:10.152 27808] Querying gravity database for MAC address of 192.168.3.219...
[2020-05-18 15:50:10.152 27808] Found database hardware address 192.168.3.219 => 18:5e:0f:a7:45:7c
[2020-05-18 15:50:10.153 27808] --> Querying client table for "18:5e:0f:aa:44:7c"
[2020-05-18 15:50:10.153 27808] --> There is no record for "18:5e:0f:aa:4a:7c" in the client table
[2020-05-18 15:50:10.153 27808] Querying gravity database for host name 192.168.3.219...
[2020-05-18 15:50:10.154 27808] --> No result.
[2020-05-18 15:50:10.154 27808] Gravity database: Client 18:5e:0f:aa:44:7c not found. Using default group

Example 2 (this client is configured through its host name):

[2020-05-18 15:50:10.146 27808] Querying gravity database for client with IP 127.0.0.1...
[2020-05-18 15:50:10.147 27808] SQL: Comparing 127.0.0.1 vs. 192.168.0.101 (subnet 255.255.255.255) - NO MATCH
[2020-05-18 15:50:10.147 27808] SQL: Comparing 127.0.0.1 vs. 192.168.0.102 (subnet 255.255.255.255) - NO MATCH
[2020-05-18 15:50:10.147 27808] SQL: Comparing 127.0.0.1 vs. 192.168.0.103 (subnet 255.255.255.255) - NO MATCH
[2020-05-18 15:50:10.147 27808] --> No record for 127.0.0.1 in the client table
[2020-05-18 15:50:10.147 27808] Querying gravity database for MAC address of 127.0.0.1...
[2020-05-18 15:50:10.148 27808] Found database hardware address 127.0.0.1 => ip-127.0.0.1
[2020-05-18 15:50:10.148 27808] --> Skipping mock-device hardware address lookup
[2020-05-18 15:50:10.148 27808] Querying gravity database for host name 127.0.0.1...
[2020-05-18 15:50:10.149 27808] Found database host name 127.0.0.1 => localhost
[2020-05-18 15:50:10.149 27808] --> Querying client table for "localhost"
[2020-05-18 15:50:10.150 27808] --> Found record for "localhost" in the client table (ID 14)
[2020-05-18 15:50:10.150 27808] Querying gravity database for client 127.0.0.1 (getting groups)
[2020-05-18 15:50:10.150 27808] Gravity database: Client localhost found. Using groups [0].