Is this still the case?
The network table had to be reset as we're now explicitly linking host names to IP addresses and this data was not available from the existing data. However, the host names should be re-populated rather soon (at least for clients known to FTL). I will work on getting host names for clients not directing queries at FTL. However, this is more lower priority as it is just the icing on the cake whereas it will not be trivial to add this concerning all the datalocks which need to be carefully aligned for this.
Furthermore, we remove records which are older than 24 hours (we flag them as "expired"). Do you think this is bad/needs tweaking?
You can customize the time after which IP addresses and host names are removed through pihole-FTL.conf using MAXNETAGE=24.0 to specify 24 hours. You can also set 72.0 for three days, etc.
I also changed the debug flag you have to set to true from DEBUG_DATABASE=true to DEBUG_CLIENTS=true. This will be a lot less chattier as you're likely not interested in the remaining database noise. I edited my post above to show this new flag.
pi@pihole:~ $ pihole checkout development
Please note that changing branches severely alters your Pi-hole subsystems
Features that work on the master branch, may not on a development branch
This feature is NOT supported unless a Pi-hole developer explicitly asks!
Have you read and understood this? [y/N] y
[i] Requested option "development" is not available
or run what @technicalpyro suggested but you'll need to re-checkout web and FTL afterwards, so the first command is preferred.
Well, interesting, this killed the system-provided function inet_pton. I was expecting this to be robust against invalid input but this doesn't seem to hold true. I will implement a check for IP addresses before calling this kernel function. You triggered an edge-case with an empty CIDR. Thanks for trying exactly this. It is now fixed. Please try further things to break it
The idea is to do two things:
Try to obtain a host name for clients newly added to the table which are unknown to FTL
Re-resolve all host names from all addresses once an hour.
No. 2 wasn't working because I wrapped everything into a single transaction and forgot to actually commit the changes to the database. This is now fixed as well. Please see if they populate after some time (as said, it may take up to the next full hour).
Can confirm. Took some time, but now I see all names for clients where Local DNS Records exist.
What I get now might be normal debug output.
[2020-05-19 19:00:00.677 17215] Res: no more rows available
[2020-05-19 19:00:00.678 17215] Res: no more rows available
[2020-05-19 19:00:00.679 17215] Res: no more rows available
[2020-05-19 19:00:00.679 17215] Res: no more rows available
[2020-05-19 19:00:00.680 17215] Res: no more rows available
[2020-05-19 19:00:00.681 17215] Res: no more rows available
[2020-05-19 19:00:00.682 17215] Res: no more rows available
[2020-05-19 19:00:00.683 17215] Res: no more rows available
[2020-05-19 19:00:00.684 17215] Res: no more rows available
[2020-05-19 19:00:00.685 17215] Res: no more rows available
[2020-05-19 19:00:00.688 17215] Res: no more rows available
[2020-05-19 19:00:00.689 17215] Res: no more rows available
[2020-05-19 19:00:00.690 17215] Res: no more rows available
[2020-05-19 19:00:00.691 17215] Res: no more rows available
[2020-05-19 19:00:00.692 17215] Res: no more rows available
[2020-05-19 19:00:00.693 17215] Res: no more rows available
192.168.4.3 is my phone connected through Wireguard. We get both the interface as well as the host name. The latter is defined through local DNS records. It shows the interface wg0 as my config file is /etc/wireguard/wg0.conf. If you'd name your wireguard interface differently, then you may see other things here (like home or work).
Note: For OpenVPN connections you'd see tun0 etc.
We also get interface and host name of 127.0.0.1.
The third device actually exists physically. Its record was complete even before my changes.
I know, I can already hear it ... Client selection by interface...
Your pace is awesome. Soon there will be no features left to be implemented....
It get fairly complicated with all the possibilities now. What is the order clients will be assigned? IP, MAC, Hostname, Interface. The moment one property fits all the others are skipped? Although the explanation is long already this might be worth putting there.
What is missing now is the "interface" in the explanation and tooltip hint.
And the "warning" only says "IP" and "MAC".
I can already here it... support wildcards......and vendor
Yes. The table I put from top to bottom. The first hit counts. IP is more important than MAC address which is more important than hostname which is more important than interface.
Yeah, I did not update the web interface at all for this. Still do be done.
wildcards: Already implemented with IP/CIDR. I don't think wildcards will be needed for MACs, host names or interfaces.
vendor: Why would I add rules for a specific vendor to be blocked? I see no reasoning here. If you want to block specific, e.g., Samsung domains to prevent Samsung devices from talking home ... you should probably just block them for all devices.
edit I think what I added in here can be very useful. If we go ahead and allow more and more, then users will use it even when they should not. Because a lot of users tend to overcomplicate things when there is no need to. It takes a lot of experience to see the simple solution, sometimes.
I will edit the priority into my table above.
Another, rather small update to this experimental branch: As the ip address is a UNIQUE field in the network_addresses table, new assignments of a previously seen IP address will automatically move an already known old address to the new device. This makes removing old entries somewhat less important. However, the new possibility to specify clients by host names may make it more important to be able to remember devices not seen for longer so they can get recognized sooner.
Hence, we disable the automated removal for now, but leave the FTL config option MAXNETAGE so users can re-enable this if they feel a need for it (setting it to a positive value will re-enable automatic removal of entries).
We'll have to see how this goes.
Automated rescanning of client groups a certain time after their first query (if their host names are only known a few minutes later, etc.) is still a to-do.