I love that v5.0 has added the ability to manage individual clients. In setting up the clients on my network I've noticed that it would be awesome to be able to scroll through the Network page (where I can see MAC addresses and basic usage information) and add known clients right from there (maybe just have the "comment" field from the client page visible on the Network page and if the user populates it then it gets auto-added to the client table?). With the current implementation I have taken to opening the client list in one tab and the Network page in the other tab and flipping back and forth trying to add all the pertinent clients to the list.
Why? All the clients from the network page should be contained in the drop-down list on the clients page. Isn't this the case for you?
This could be useful as none of the devices from my VLAN show up here (and I thought they did in the prior version 4.x.x...not sure). My dual pi-hole setup can service VLAN requests, firewall rules allow my primary LAN devices to access the entire VLAN address range, but VLAN only allows certain functions like port 53 routed back to the Pi-holes. I think this is fairly common for those employing a guest network/IoT network of similar. Not sure how easy it would be to grab the MAC ID over routed traffic. Would be nice to have but not essential. Thanks.
EDIT: Also on the network page, I noticed there are many very old devices that will never come back. Is there a way to remove those or set an expire time (say after 30 days they roll off)? I should say, too, thanks for a great product. 
You'd need to deploy an external application to do it. MAC addresses are dropped at the router. The router replaces the requestor MAC with its own MAC and then forwards the packet to the new network segment. Response traffic is addressed to the router MAC, the router gets the packet and rewrites the MAC back to the requestor. It's called a broadcast domain, once you segment that domain then the router or a DHCP-helper type application.
Got it, wasn't sure if there was some type of ICMP-like request that could be done for that info across broadcast domains. The custom DNS feature is great for this purpose on the query and dashboard pages. Cheers.
No, this is technically not possible and was never possible before. You can manually add clients to the network table in this case, however, we do not offer a GUI for doing so. Once they are added in the network (and network_addresses) tables of /etc/pihole/pihole-FTL.db, they will also show up in the drop-down menu of the client group management page.
I have adding devices that are seen by FTL which are more than one hop away (like your VLANs) on my ToDo, however, it has low priority. The network overview started as a tool showing you the devices on the local network so you can see if some of them are not using the Pi-hole (reddish background color). Extending this beyond the physically connected network seems to be advantageous due to how it is used now, however, I will first have to see how to implement this as devices are currently not allowed to exist without a proper (and unique!) MAC address.
We will likely have to craft up mock-addresses which are known to not exist so the GUI can exclude them from being shown.
Thanks for the clarification, makes sense. Thinking more about it from an admin view, mac id isn't even necessary (at least in my use), I would look to this page to see what's on the network, it's useful information with the stats (dates, query count). In conjunction with the Custom DNS info and IP addr, that would be plenty for host ID. Pi-hole knows the unique IP addresses hitting it...the problem might be if the same client is not using a static IP and it's changing (i.e., not unique like mac id). I don't think this is a problem from an admin perspective, because if it is, the IP could be made static. Just thinking out-loud.
I noticed yesterday adding devices to Custom DNS it allowed me to add two entries for the same IP addr (I wanted to change the name of one, so thought it might overwrite rather than delete/re-add), then removing one removed both. Not sure if that's by design...just mentioning in case not.
Again thanks.
Yes, this is exactly the issue why they're not already in there. The MAC needs to be unique in the table, but we can likely "just" use the IP address as pseudo-MAC.
The issue is more with the statistics. Given many users have a non-deterministic DHCP server (handing out IP addresses sequentially in the order of appearance of the device), it is a commonly seen situation that a smartphone gets the IP address 192.168.0.2 which was associated to a laptop on the day before. And this just because the device was switched on before the laptop on this day. All the statistics that were counted for the laptop yesterday would now be added to the same "device" because of the same IP address. I don't think I like this...
Note that the Pi-hole's internal DHCP server is in fact deterministic, i.e., tries to assign the same dynamic IP address to the same devices every time. This is also the reason why the first device can receive a pretty high terminal address such as 192.168.0.216 because we computed this IP from the hash of the MAC address of the requesting device. This to make per-client filtering possible even with devices that may do not even support (or honor) static assignments. I, for instance, do not have a single static IP (only exception is the Pi-hole itself) in my network. Everything else is dynamic.
There is not only a lack in validation, the entire logic seems to need a restructuring. Already on my todo list. ETA sometime next week.
Please try if
pihole checkout ftl new/all_clients_network_table
fills your network table with all the clients you are missing from it. Instead of only those known to the kernel (ARP/neigh cache), we now also populate the table with all remaining clients known to FTL. Obviously, there will be no MAC address available here and also the interface information is too difficult to obtain, so they are missing. I will have to invest a little bit of work into hiding the mock hardware addresses in the web interface but it should already work now.
Note that this change does not implement validation for the input fields in any way. This has to come separately (and later).
2 Likes
I'm not sure if it is a FTL's refresh/cache issue (as the network table is not populate immediately) but I have a client that is now twice in the list.
No ideas why it is this client and not all others too....
Current Pi-hole version is v4.3.5-444-gf617ed2
Current AdminLTE version is v4.3.2-436-g56de14a
Current FTL version is vDev-ae3418a
I assume the device has not (yet) been used today, right? Hence, it was in FTL's memory but not in the ARP cache (the entry expired over night) and FTL added it to the table as something the ARP cache didn't know about. This is maybe expected side effect of what we do now.
I pushed a small modification skipping the addition of clients with zero (active) queries. This may relieve this special situation. However, we cannot know if we see one and the same thing without the MAC address.
Please update FTL to the latest version and remove the entry you're seeing there with
sudo sqlite3 /etc/pihole/pihole-FTL.db "DELETE FROM network WHERE hwaddr = 'ip-10.0.1.84';"
No need to notify FTL of this change, it will update the network table a few minutes later.
Right 
But there are others that haven't queried pihole today and haven't got a duplicated entry.
Why are you using the arp cache as comparison instead of pihole's network table? This would circumvent the problem with clients not seen for a while (no info in the ARP cache) because they are still in the network overview? I guess there is a drawback I don't see right now... 
This reminds me that I've seen a feature request somewhere to be able to delete single entries from the network table via web UI 
Because we don't know if a device that was not seen in weeks with some IP address 192.168.0.3 is still the same as it was a long time ago. Most users have routers with sequential IP address associating DHCP servers, i.e., the IP a device gets solely depends on the order of switching the devices on at this day. This makes the entire identify devices by their IP addresses idea pretty useless and is why I am still cautious with this approach.
Yes, this is because the last query of this device was three days ago, i.e., it isn't present any more in Pi-hole's memory.
edit: I pushed another small change that should finally eliminate the double addition of devices with the same IP address. This should resolve most of my concerns.
I hear you, but I don't think we want to push the release of v5.0 much longer. There has to be something left for v5.1
Clearly understand. This is why it should be used with static DHCP leases.
Hehe
My general take on networking tell me: Never use static DHCP leases. It's also why Pi-hole's integrated DHCP server is a deterministic one. But that's another story.
I went back and forth about having this as an option for power users (disabled by default), but then I think it wouldn't make any difference as "normal" users wouldn't have any devices in their networks which are not also known to the Pi-hole (except lo, a.k.a. 127.0.0.1 or ::1 on the Pi-hole server itself).
If there is once enough time I would really like to hear the reason for that.
My guess is: 99% of the users won't notice any differences (except of 127.0.0.1) so you could also skip the option and enable it by default. The ones that use VLAN/subnetting will likely be happy to see their device and will use the function anyway if they know about it. You would save them a step to enable the configuration.
Yes, this is my plan.
Networking is also about simplicity. KISS is a good principle to keep things clean. Other reasons:
-
It is much easier to remember addresses than IPs when they are meaningful.
-
Using names like
chromecastis much easier than trying to remember that it was10.0.30.254after not having used this in months. In your case, it is not only the last octet to be remembered, it is even more so prone to errors. DNS is a central service and should be relied on enough to assume it is stable. -
Switching behavior to host names already now will make the transition to dual-stacked (at at some point, hopefully, IPv6-only) systems a lot easier.
-
The IP address can change without affecting the end user in case you run internal services and want to move them from one to another device. This may be less important on home-style networks, but, as you know, one IP can identify a single server and yet have many names associated to it. One server can host multiple "virtual hosts" and only using a host name, they can be distinguished.
There are probably some more but this is what came to my mind without thinking too much about it. And typing on the phone is not one of my strengths, either, so I'll just stop here. I think this request is resolved once we merge this into the beta code. Anything else we're missing here (except the necessary web changes to hide mock-addresses)?
Made an account to say I just stumbled onto this and it worked great for me.
I recently migrated to a docker container bridged setup of pihole, unbound and dhcphelper relay running together. Once that was up I noticed the network table page populated with just the IP of the dhcp relay interface.
Running the above command populated around 20 hostnames and their details for the exception of obviously the MAC address.
Thank you! Looking forward to it being added officially in upcoming releases!
1 Like
Something went off again. Not sure when it happend, my guess would be when the device 10.0.1.84 was turned on first time today. It is the same device we had the issues earlier today. 10.0.1.1. is the Gatway/DHCP server.
nanopi@nanopi:~$ pihole -v
Pi-hole version is v4.3.5-444-gf617ed2 (Latest: v4.4)
AdminLTE version is v4.3.2-436-g56de14a (Latest: v4.3.3)
FTL version is vDev-560685a (Latest: v4.3.1)
Not that I see. I have one more idea but this can wait for v5.1 (will there be a beta as well?)
Thanks for your explanation and thoughts.