I've just edited my Wireguard configuration both on the OpenWrt router and mostly on Android Wireguard client - now I can indeed access internal devices and I would say that the configuration is satisfying.
Specifically, on the Android Wireguard client I've configured as Allowed IPs my PiHole residing at 192.168.1.10 - for DNS resolution - and the reverse proxy on the Linux Machine, residing at 192.168.1.15.
On the PiHole I've configured Local DNS Records so that music.myserver.com points to the reverse proxy at 192.168.1.15.
Now, when I am connected to my own WiFi and thus I have access to my LAN and the Wireguard tunnel is either active or inactive, when I access music.myserver.com with my Android device I am indeed directed to the reverse proxy at 192.168.1.15. When I am not at home (so either connected through mobile data or some other WiFi), when the Wireguard tunnel is active I can still reach my reverse proxy thanks to this configuration. So Bucking_Horn solution works!
I must add that, at first, when connected to my own WiFi, my Android device wouldn't want to reach my proxy unless the Wireguard tunnel was active. After checking some additional posts (such as this: Android device bipasses pi-hole ) I happened to understand that the IPv6 DNS configuration (or mis-configuration) on my APs was somehow "interfering" with the DNS resolution on Android devices, which were basically bypassing PiHole DNS server. So I have completely disabled IPv6 on my Openwrt APs (I have several around the house).
I also took an additional step in order to force all DNS queries from all devices (even those with hard-coded DNSes) to go through PiHole - this is an additional OpenWrt configuration listed here: Force All DNS Queries Through PiHole with OpenWRT
What still puzzles me is that when I am connected to my own WiFi (with Wireguard tunnel active or inactive) and, using Termux on Android, I run dig music.myserver.com, I get Cloudflare IPs in the ANSWER SECTION and 8.8.8.8#53 as DNS server.
On the other hand, if I run dig music.myserver.com @192.168.1.10, I do get my reverse proxy IP in the ANSWER SECTION and 192.168.1.10#53 as DNS server, which is the PiHole IP.
So I have the feeling the Android device is still somehow bypassing the PiHole and using its own hard-coded DNS, as the same dig music.myserver.com command used in several Linux machines in my network resolves the proxy IP and uses the PiHole as DNS server.
I've read something about DoH and DoT related to Android on OpenWrt forum, but this is getting beyond my abilities now...