Pihole DNS-Server stops working (Azure, Ubuntu)

[gdw4fpj4tayxug8]

Expected Behaviour:

We're running a pihole instance on Azure (Ubuntu). There's nothing special on this installation - i followed the official pihole installation guide.
We use pihole to block Cryptominers, Ads, & Co.

Actual Behaviour:

After some days (most of the time 2-4 Weeks) the Webinterface of pihole says, that "DNS service not running". I can still connect via HTTP or SSH to the machine. DNS-Lookups from the console fail.
If i reboot the VM, pihole is up again and working as expected. Also, if i restart "pihole-FTL", the local (and remote) DNS-resolution will also work again.

After the problem occures 2-3 times, i tought it would be a good idea to add a second pihole in another Azure Datacenter. So i created a second instance. I'm using GravitySync to keep the pihole-Database in sync between the two VMs; but i don't think that this is the cause of my problem (because the cause occured before i installed GravitySync and the second VM).

The problem occures on BOTH VMs. The pihole's are used as #1 and #2 DNS-Servers for 4 Offsite-Departments in their Firewall's. The access to the pihole's is restricted by the source ip address.

The VMs are using 1 vCPU and 512 MB RAM. It seems to be enought (Load < 1, RAM 262/403 used). I'm using the same Setup on my Homelab, with also 1 CPU an 512MB without problems. I've also tried more ressources (2 CPUs, 1 GB RAM), but the problem still occured, so i went back.

I'm not sure, what the exact problem is - i can't find hints for stopped services or something similar in the Logs. Also, pihole status seems to be OK:

/var/log/pihole# pihole status
  [✓] FTL is listening on port
     [✓] UDP (IPv4)
     [✓] TCP (IPv4)
     [✓] UDP (IPv6)
     [✓] TCP (IPv6)

  [✓] Pi-hole blocking is enabled

There's also enough disk space:

/var/log/pihole# df -h
Filesystem      Size  Used Avail Use% Mounted on
/dev/root        29G  7.4G   22G  26% /
devtmpfs        198M     0  198M   0% /dev
tmpfs           202M   32M  171M  16% /dev/shm
tmpfs            41M  988K   40M   3% /run
tmpfs           5.0M     0  5.0M   0% /run/lock
tmpfs           202M     0  202M   0% /sys/fs/cgroup
/dev/loop1       64M   64M     0 100% /snap/core20/1778
/dev/loop0       64M   64M     0 100% /snap/core20/1738
/dev/loop2       92M   92M     0 100% /snap/lxd/23991
/dev/loop3       92M   92M     0 100% /snap/lxd/24061
/dev/loop4       50M   50M     0 100% /snap/snapd/17883
/dev/sda15      105M  5.2M  100M   5% /boot/efi
/dev/sdb1       3.9G   28K  3.7G   1% /mnt
/dev/loop5       50M   50M     0 100% /snap/snapd/17950
tmpfs            41M     0   41M   0% /run/user/1000

Do you have any ideas, how i can troubleshoot the problem, if it occures the next time?

Debug Token:

I can't upload the Debug-Token, because the DNS-Resolving is not functional. So i paste it here:
(Moderator edit: explicit debug log removed)

[Bucking_Horn]

(For your own privacy and security, do not post the full, unsanitised debug log output here.)

Your debug log suggests that pihole-FTL has been offline, at least at the time of debug log creation:

*** [ DIAGNOSING ]: Ports in use
    udp:127.0.0.1:323 is in use by chronyd
    udp:10.3.0.4%eth0:68 is in use by systemd-network
    udp:[::1]:323 is in use by chronyd
[✓] tcp:0.0.0.0:80 is in use by lighttpd
    tcp:0.0.0.0:22 is in use by sshd
[✓] tcp:[::]:80 is in use by lighttpd
    tcp:[::]:22 is in use by sshd

*** [ DIAGNOSING ]: Name resolution (IPv4) using a random blocked domain and a known ad-serving domain
[✗] Failed to resolve rkinvestpl.info on lo (127.0.0.1)
[✗] Failed to resolve rkinvestpl.info on eth0 (10.3.0.4)
[✓] doubleclick.com is 142.250.74.238 via a remote, public DNS server (8.8.8.8)

Your debug log also shows your Pi-hole to receive traffic from public IPs, suggesting that you may be running an open resolver, which poses a potential threat for all Internet users, e.g. by serving as a multiplier in a DNS Amplification attack .

The Pi-hole team strongly discourages Pi-hole’s usage as an open resolver, and we won't provide support in that case.

If this is not your intended use, I'd strongly recommend to install a VPN server on your cloud-based Pi-hole instance, allowing secure access by properly authenticated clients only.
Filtering by IP address is not good enough (MITM, IP spoofing,...) - especially if this is a company setup, as your description seems to indicate.

[gdw4fpj4tayxug8]

Hello Bucking_Horn,

thanks for your Feedback. We will secure the traffic by a VPN-connection to the pihole-instances in the future.

Okay, so pihole-FTL has been offline/crashed. I can't find anything related in the logs (in which logs i should see further informations?).

Where i should troubleshoot, why pihole-FTL crashed?

Thanks for your Feedback!