The lease request from the client is a broadcast packet and that could be looked at however, the response with the actual lease offer is unicast from the DHCP server to the client and the acknowledgement from the client to the server accepting the lease offer is also unicast.
What about monitoring the DHCP lease request for the host name and logging that into the database with the mac address to later be matched to the ip on DNS request?
Are you sure for your router?
Check with below if your router creates DNS records (A, AAAA and PTR) for hostnames that are advertised by your clients during the DHCP lease transaction:
If your router doesn't support disabling DHCP, you could still shrink its DHCP range to accommodate just your Pi-hole host's IP, configure a DHCP lease reservation for it, and have Pi-hole's DHCP use a range that wouldn't overlap with your router.
Some rare routers may force a certain minimum DHCP range - mostly, for two addresses (e.g. 192.168.1.2 to 192.168.1.3).
To avoid any DHCP clients to register with your router in that case, configure one fixed IP for your Pi-hole host's MAC, and fill the other slot with a bogus MAC.
I found this ("listen to DHCp broadcasts") an interesting idea and started investigating this in my small network. For this, I used a TP-Link WR841 which I had sitting in the cabinet. However, bad news: The "broadcasts" seem a lot less broad than thought: At least this router sends the broadcast only on the port it received the request. This is obviously a good idea to reduce unnecessary network traffic on all the other ports and shows that such a feature could easily fail while this wouldn't be simple to fix.
If you still want to use this router, one might be able to circumvent it by setting the pool to something like 10-20 (assuming 10 IPs will be enough for your guests) and block these IPs manually in the main network with IP reservations for devices that do not even exist. Ten is still a low-enough number for manual records.
However, I do agree on the idea of getting another router. If the setting are so limited ... who known how well the firewall, etc. works. Does this router even receive security updates?...
That is odd, the initial DHCP request from the client without a source IP 0.0.0.0 should be to the broadcast destination address 255.255.255.255 meaning it should be broadcasted on all switch ports for that network segment/broadcast domain ... to my knowledge.
And the reply from the DHCP service should switch to unicast instead of broadcast as depicted below:
pi@ph5a:~ $ sudo tcpdump -nvi any udp port 67 and udp port 68
[..]
0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 0c:2f:b0:XX:XX:XX, length 308, xid 0x32d760d0, Flags [none]
[..]
Hostname Option 12, length 11: "hak-phone"
[..]
10.0.0.2.67 > 10.0.0.143.68: BOOTP/DHCP, Reply, length 312, xid 0x32d760d0, Flags [none]
[..]
Below is a Windows client of mine renewing its lease (which it does very often):
pi@ph5a:~ $ sudo tcpdump -nvi any udp port 67 and udp port 68
[..]
10.0.0.11.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 00:1e:0b:XX:XX:XX, length 300, xid 0x7a45a3b6, Flags [Broadcast]
[..]
Hostname Option 12, length 6: "hak-pc"
[..]
10.0.0.2.67 > 10.0.0.11.68: BOOTP/DHCP, Reply, length 300, xid 0x7a45a3b6, Flags [Broadcast]
[..]
At least thats what I can make of it
Nice from you for looking into it!
EDIT: Ow ps. above ph5a host also does DHCP service for my network.