IP addresses on the dashboard are not replaced by names from the network table if they cannot be associated with a MAC address *immediately*

[Daxtorim]

Expected Behaviour:

IP addresses without a known hostname associated to them should be replaced with another hostname of the same device from the network table, if possible. The corresponding setting (NAMES_FROM_NETDB) defaults to TRUE.

Actual Behaviour:

Sometimes IP addresses are not replaced by names from the network table.

In the case of the green underlined address, it is already filed under an existing entry which has several known hostnames associated with it. The hostnames are not all identical, however. Some addresses have the hostname that was set in the device's settings connected to them, other addresses are using a generic Android-X.fritz.box, where X is a digit.

---

It gets more interesting with the red underlined address.
In this case the IP address couldn't be connected to any known client.

The reason being that there was no entry in the neighbor cache for that address. How there was no entry after almost 3000 queries is beyond me.

I noticed this happening every now and again.

Even more perplexing, after simply pinging the offending address, and not getting a response, an entry was generated.

Now the IP address was correctly filed under an existing entry in the network table.

But, the dashboard is still showing the IP address, not one of the associated hostnames.

---

A somewhat related question:
When there are multiple different hostnames known for a device, as mentioned in the first case, but none for the specific address in question, which hostname is chosen to be displayed on the dashboard?

Additional Info:

Hardware: Raspberry Pi Zero W
OS: Raspbian GNU/Linux 10 (buster)

$ pihole version
  Pi-hole version is v5.3.1 (Latest: v5.3.1)
  AdminLTE version is v5.5 (Latest: v5.5)
  FTL version is v5.8.1 (Latest: v5.8.1)

Contents of pihole-FTL.conf:

$ cat /etc/pihole/pihole-FTL.conf
MAXDBDAYS=31
PRIVACYLEVEL=0
REFRESH_HOSTNAMES=IPV4
NAMES_FROM_NETDB=true

Debug Token:

https://tricorder.pi-hole.net/p55iho6q1o

[Bucking_Horn]

@Daxtorim

REFRESH_HOSTNAMES=IPV4

Your examples seem to be IPv6 exclusively.
You may want to switch to ALL, see also Configuration - Pi-hole documentation.

[Daxtorim]

REFRESH_HOSTNAMES only controls the hourly reverse lookups, which is irrelevant here. Reverse lookups are always done for all addresses when they first appear regardless of that setting.
There are no known hostnames for these addresses, the hourly lookups return NXDOMAIN. That's the whole point of this topic, the replacement of these addresses with known hostnames for other addresses of the same device doesn't work. Getting NXDOMAIN for the reverse lookup a few more times throughout the day won't change this.

Whether this is exclusive to IPv6 I don't know yet. I have one device that I know for certain will not give back any hostname. I manually added an entry to /etc/hosts for the IPv6 addresses. Unfortunately I don't have a way to trigger a DNS query from that device, but if it ever makes a query from its IPv4 address it should be clear whether it gets replace with a known hostname from the IPv6 addresses or NAMES_FROM_NETDB just doesn't work at all.

[Bucking_Horn]

@Daxtorim

REFRESH_HOSTNAMES only controls the hourly reverse lookups, which is irrelevant here.

It is relevant, as those reverse lookups are the prime source for hostnames. For an IPv6 only client, that may be the only way to acquire its name. Using NAMES_FROM_NETDB is a fallback option.
Note that ip neigh does only associate an IP address with a MAC address, not with a hostname. A hostname has to be sourced through other means.

Being a fallback option may also mean that NAMES_FROM_NETDB isn't used once you disable the primary method.
I don't know this - I'll check with development whether that is actually the case.

@Daxtorim

There are no known hostnames for these addresses, the hourly lookups return NXDOMAIN .

How do you know that for IPv6 addresses if you are using REFRESH_HOSTNAMES=IPV4?

If indeed NXDOMAIN would be returned, you would of course be right in stating that additional reverse lookups would be of little help.
However, your original post didn't include that information.

[Daxtorim]

@Bucking_Horn

How do you know that for IPv6 addresses if you are using REFRESH_HOSTNAMES=IPV4 ?

As I said already reverse lookups are always done for new clients regardless of that setting. I also checked manually via dig -x IPv6-ADDRESS.

@Bucking_Horn

Being a fallback option may also mean that NAMES_FROM_NETDB isn't used once you disable the primary method.

I had some time to look at the code and you are actually right about that, but again, that's only true when the client is already known. However, I think here lies exactly my issue. For whatever reason, these addresses aren't registered in the neighbor cache and thus Pi-hole cannot associate the address with a known client. When the reverse lookup does not return a hostname, the fallback option won't either because there is no way to know which client the address blongs to.

And now that the address is a separate known client there won't be any more reverse lookups because of REFRESH_HOSTNAMES=IPV4 (it should be noted here that this is the default value). Even if the address is eventually entered into the neighbor cache, it doesn't matter anymore.

I'm sorry, you were absolutely right. The solution is REFRESH_HOSTNAMES=ALL.

The underlying problem here is that the address is not entered into the neighbor cache, but that is obviously not Pi-hole's job.

[DL6ER]

Each client seen for the first time will trigger a name lookup. This is true for both IPv4 and IPv6 clients but can be altered using the config options RESOLVE_IPv4 and RESOLVE_IPv6 (both defaulting to true). Another condition for whether we should resolve a hostname is that said client has to have made at least one query within the last two hours. This to avoid making reverse lookups for clients that are already gone when restarting FTL and reloading the last 24 hours of queries from the database.

The REFRESH_HOSTNAMES option has nothing to do with this first lookup. It only controls said refreshing of the host names which happens on every full hour (compile-time option RERESOLVE_INTERVAL).

Note that you can set DEBUG_RESOLVER=true in pihole-FTL.conf to get detailed explanations in your FTL log about how/where a host name was found.

The procedure of resolving a host name is now the following (function resolveAndAddHostname()):

1. We query the nameserver at 127.0.0.1 (this will typically be your Pi-hole) for the name using a PTR query. A successful lookup will be logged with the debug message ---> "abc.de" (found internally).

2. If this returned no result, we try the nameserver(s) configured in resolv.conf. This is necessary to get hotnames associated with docker containers and can be the correct choice if conditional forwarding is not configured and the Pi-hole still using the DNS server installed before Pi-hole (typically the router in at-home installations).

3. If both above returned NXDOMAIN, we check the option NAMES_FROM_NETDB (defaulting to true) and ask the network table using:

   SELECT name FROM network_addresses
          WHERE name IS NOT NULL AND
                ip = 'ip-addr';
   

4. If nothing was found here, there is another fallback checking for a hostname associated with the same device but another IP address (if ambiguous, we choose the most recent one):

   SELECT name FROM network_addresses 
          WHERE name IS NOT NULL AND
                network_id = (SELECT network_id FROM network_addresses
                                     WHERE ip = 'ip-addr')
          ORDER BY lastSeen DESC LIMIT 1;
   

5. If all these lookups fail, we return with "hostname not known".

---

Point 4 above answers

@Daxtorim

A somewhat related question:
When there are multiple different hostnames known for a device, as mentioned in the first case, but none for the specific address in question, which hostname is chosen to be displayed on the dashboard?

---

@Daxtorim

Even if the address is eventually entered into the neighbor cache, it doesn't matter anymore.

A restart of FTL should pick the address up, too. But, yes, in the present scenario the issue comes from that IPv6 hostnames once found (or not) stay there forever. This default was chosen as more and more operating systems (I think Apple started the discussion, Windows entered and some Linux flavors followed) started to make IPv6 privacy extensions the default and quickly filled each small home network with several dozens of new addresses every few hours, resulting in a lot of lookups that have to be done on the full hour.

The option REFRESH_HOSTNAMES was added here:

Fix for numerous hourly PTR requests seen in network with many (dead) IPv6 addresses

It is open for discussion if we should add maybe a fourth option REFRESH_HOSTNAMES=IPV6-RECENT reducing the activity requirement to maybe 30 minutes for IPv6 addresses. The related previous discussion is here:

Pi-hole Reverse DNS queries every hour

I recently updated to Pi-Hole v5.2 and FTL v5.3.1. After that update, I am seeing in my logs that every hour at the top of the hour, the localhost, my Raspberry Pi4, is making about ~1,290 Reverse DNS queries. This starts literally at the top of each hour and lasts for about the first 30 seconds. I did notice that Pi-Hole is showing that it is listening on IPV6 although I turned off IPV6 on Pi-Hole as well as on my router. When I run pihole -d I see that it shows two IPV6 addresses with "does …