Some sites are not loading with Unbound upstream

sudo rgrep -v '^ *#\|^$' /etc/unbound/unbound.conf*

sudo unbound-checkconf

sudo journalctl --full --no-pager --lines 30 -u unbound.service

?

EDIT: Sorry I editted that last one!

pi@raspberrypi:/etc $ sudo rgrep -v '^ *#\|^$' /etc/unbound/unbound.conf*
/etc/unbound/unbound.conf.d/pi-hole.conf:server:
/etc/unbound/unbound.conf.d/pi-hole.conf:    verbosity: 0
/etc/unbound/unbound.conf.d/pi-hole.conf:    interface: 127.0.0.1
/etc/unbound/unbound.conf.d/pi-hole.conf:    port: 5335
/etc/unbound/unbound.conf.d/pi-hole.conf:    do-ip4: yes
/etc/unbound/unbound.conf.d/pi-hole.conf:    do-udp: yes
/etc/unbound/unbound.conf.d/pi-hole.conf:    do-tcp: yes
/etc/unbound/unbound.conf.d/pi-hole.conf:    do-ip6: no
/etc/unbound/unbound.conf.d/pi-hole.conf:    prefer-ip6: no
/etc/unbound/unbound.conf.d/pi-hole.conf:    harden-glue: yes
/etc/unbound/unbound.conf.d/pi-hole.conf:    harden-dnssec-stripped: yes
/etc/unbound/unbound.conf.d/pi-hole.conf:    use-caps-for-id: no
/etc/unbound/unbound.conf.d/pi-hole.conf:    edns-buffer-size: 1232
/etc/unbound/unbound.conf.d/pi-hole.conf:    prefetch: yes
/etc/unbound/unbound.conf.d/pi-hole.conf:    num-threads: 1
/etc/unbound/unbound.conf.d/pi-hole.conf:    so-rcvbuf: 1m
/etc/unbound/unbound.conf.d/pi-hole.conf:    private-address: 192.168.0.0/16
/etc/unbound/unbound.conf.d/pi-hole.conf:    private-address: 169.254.0.0/16
/etc/unbound/unbound.conf.d/pi-hole.conf:    private-address: 172.16.0.0/12
/etc/unbound/unbound.conf.d/pi-hole.conf:    private-address: 10.0.0.0/8
/etc/unbound/unbound.conf.d/pi-hole.conf:    private-address: fd00::/8
/etc/unbound/unbound.conf.d/pi-hole.conf:    private-address: fe80::/10
pi@raspberrypi:/etc $ sudo unbound-checkconf
[1700248734] unbound-checkconf[13442:0] error: Could not open /etc/unbound/unbound.conf: No such file or directory

This is granted, because the pihole doco has this config in /etc/unbound/unbound.conf.d/pi-hole.conf

pi@raspberrypi:/etc $ sudo journalctl --full --no-pager --lines 30 -u unbound.service
-- Logs begin at Fri 2023-11-17 20:11:21 IST, end at Fri 2023-11-17 21:19:57 IST. --
Nov 17 21:18:04 raspberrypi systemd[1]: Stopped Unbound DNS server.
Nov 17 21:18:04 raspberrypi systemd[1]: Starting Unbound DNS server...
Nov 17 21:18:04 raspberrypi package-helper[13425]: [1700248684] unbound-checkconf[13427:0] error: Could not open /etc/unbound/unbound.conf: No such file or directory
Nov 17 21:18:04 raspberrypi package-helper[13428]: [1700248684] unbound-checkconf[13430:0] error: Could not open /etc/unbound/unbound.conf: No such file or directory
Nov 17 21:18:04 raspberrypi unbound[13431]: [1700248684] unbound[13431:0] error: Could not open /etc/unbound/unbound.conf: No such file or directory
Nov 17 21:18:04 raspberrypi unbound[13431]: [1700248684] unbound[13431:0] warning: Continuing with default config settings
Nov 17 21:18:04 raspberrypi unbound[13431]: [1700248684] unbound[13431:0] error: can't bind socket: Address already in use for ::1 port 53
Nov 17 21:18:04 raspberrypi unbound[13431]: [1700248684] unbound[13431:0] fatal error: could not open ports
Nov 17 21:18:04 raspberrypi systemd[1]: unbound.service: Main process exited, code=exited, status=1/FAILURE
Nov 17 21:18:04 raspberrypi systemd[1]: unbound.service: Failed with result 'exit-code'.
Nov 17 21:18:04 raspberrypi systemd[1]: Failed to start Unbound DNS server.
Nov 17 21:18:04 raspberrypi systemd[1]: unbound.service: Service RestartSec=100ms expired, scheduling restart.
Nov 17 21:18:04 raspberrypi systemd[1]: unbound.service: Scheduled restart job, restart counter is at 4.
Nov 17 21:18:04 raspberrypi systemd[1]: Stopped Unbound DNS server.
Nov 17 21:18:04 raspberrypi systemd[1]: Starting Unbound DNS server...
Nov 17 21:18:04 raspberrypi package-helper[13432]: [1700248684] unbound-checkconf[13434:0] error: Could not open /etc/unbound/unbound.conf: No such file or directory
Nov 17 21:18:04 raspberrypi package-helper[13435]: [1700248684] unbound-checkconf[13437:0] error: Could not open /etc/unbound/unbound.conf: No such file or directory
Nov 17 21:18:04 raspberrypi unbound[13438]: [1700248684] unbound[13438:0] error: Could not open /etc/unbound/unbound.conf: No such file or directory
Nov 17 21:18:04 raspberrypi unbound[13438]: [1700248684] unbound[13438:0] warning: Continuing with default config settings
Nov 17 21:18:04 raspberrypi unbound[13438]: [1700248684] unbound[13438:0] error: can't bind socket: Address already in use for ::1 port 53
Nov 17 21:18:04 raspberrypi unbound[13438]: [1700248684] unbound[13438:0] fatal error: could not open ports
Nov 17 21:18:04 raspberrypi systemd[1]: unbound.service: Main process exited, code=exited, status=1/FAILURE
Nov 17 21:18:04 raspberrypi systemd[1]: unbound.service: Failed with result 'exit-code'.
Nov 17 21:18:04 raspberrypi systemd[1]: Failed to start Unbound DNS server.
Nov 17 21:18:05 raspberrypi systemd[1]: unbound.service: Service RestartSec=100ms expired, scheduling restart.
Nov 17 21:18:05 raspberrypi systemd[1]: unbound.service: Scheduled restart job, restart counter is at 5.
Nov 17 21:18:05 raspberrypi systemd[1]: Stopped Unbound DNS server.
Nov 17 21:18:05 raspberrypi systemd[1]: unbound.service: Start request repeated too quickly.
Nov 17 21:18:05 raspberrypi systemd[1]: unbound.service: Failed with result 'exit-code'.
Nov 17 21:18:05 raspberrypi systemd[1]: Failed to start Unbound DNS server.

I used to have unbound-13.0 that I compiled myself based on a guide I had.
Is there a way to totally nuke anything that is unbound related? Maybe somethings needs to be reversed?

You're missing below file with a very important include directive:

pi@ph5a:~ $ sudo rgrep -v '^ *#\|^$' /etc/unbound/unbound.conf*
/etc/unbound/unbound.conf:include: "/etc/unbound/unbound.conf.d/*.conf"
[..]

It's included in the unbound package:

pi@ph5a:~ $ dpkg -S unbound.conf
unbound: /etc/unbound/unbound.conf
[..]

Can try reinstall it with below:

sudo apt -o Dpkg::Options::="--force-confmiss" install --reinstall unbound

Whats your OS release?

hostnamectl | grep Operating

I believe you're missing more.

pi@ph5a:~ $ sudo rgrep -v '^ *#\|^$' /etc/unbound/unbound.conf*
/etc/unbound/unbound.conf:include: "/etc/unbound/unbound.conf.d/*.conf"
[..]
/etc/unbound/unbound.conf.d/qname-minimisation.conf:server:
/etc/unbound/unbound.conf.d/qname-minimisation.conf:    qname-minimisation: yes
/etc/unbound/unbound.conf.d/root-auto-trust-anchor-file.conf:server:
/etc/unbound/unbound.conf.d/root-auto-trust-anchor-file.conf:    auto-trust-anchor-file: "/var/lib/unbound/root.key"
[..]
pi@raspberrypi:/etc $ sudo apt -o Dpkg::Options::="--force-confmiss" install --reinstall unbound
Reading package lists... Done
Building dependency tree
Reading state information... Done
Suggested packages:
  apparmor
The following NEW packages will be installed:
  unbound
0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
Need to get 674 kB of archives.
After this operation, 3,646 kB of additional disk space will be used.
Get:1 http://mirror.de.leaseweb.net/raspbian/raspbian buster/main armhf unbound armhf 1.9.0-2+deb10u3 [674 kB]
Fetched 674 kB in 2s (370 kB/s)
Selecting previously unselected package unbound.
(Reading database ... 157368 files and directories currently installed.)
Preparing to unpack .../unbound_1.9.0-2+deb10u3_armhf.deb ...
Unpacking unbound (1.9.0-2+deb10u3) ...
Setting up unbound (1.9.0-2+deb10u3) ...
Job for unbound.service failed because the control process exited with error code.
See "systemctl status unbound.service" and "journalctl -xe" for details.
Job for unbound.service failed because the control process exited with error code.
See "systemctl status unbound.service" and "journalctl -xe" for details.
invoke-rc.d: initscript unbound, action "restart" failed.
● unbound.service - Unbound DNS server
   Loaded: loaded (/lib/systemd/system/unbound.service; enabled; vendor preset: enabled)
   Active: activating (auto-restart) (Result: exit-code) since Fri 2023-11-17 22:08:48 IST; 28ms ago
     Docs: man:unbound(8)
  Process: 14242 ExecStartPre=/usr/lib/unbound/package-helper chroot_setup (code=exited, status=0/SUCCESS)
  Process: 14245 ExecStartPre=/usr/lib/unbound/package-helper root_trust_anchor_update (code=exited, status=0/SUCCESS)
  Process: 14248 ExecStart=/usr/sbin/unbound -d $DAEMON_OPTS (code=exited, status=1/FAILURE)
 Main PID: 14248 (code=exited, status=1/FAILURE)
Processing triggers for man-db (2.8.5-2) ...
Processing triggers for systemd (241-7~deb10u9+rpi1) ...
pi@raspberrypi:/etc $
Operating System: Raspbian GNU/Linux 10 (buster)

Below would help if still not working?

Oh I also have a buster node:

pi@ph5a:~ $ hostnamectl | grep Operating
  Operating System: Raspbian GNU/Linux 10 (buster)

Looks like this:

pi@ph5a:~ $ sudo rgrep -v '^ *#\|^$' /etc/unbound/unbound.conf*
/etc/unbound/unbound.conf:include: "/etc/unbound/unbound.conf.d/*.conf"
/etc/unbound/unbound.conf.d/qname-minimisation.conf:server:
/etc/unbound/unbound.conf.d/qname-minimisation.conf:    qname-minimisation: yes
/etc/unbound/unbound.conf.d/root-auto-trust-anchor-file.conf:server:
/etc/unbound/unbound.conf.d/root-auto-trust-anchor-file.conf:    auto-trust-anchor-file: "/var/lib/unbound/root.key"
/etc/unbound/unbound.conf.d/pi-hole.conf:server:
/etc/unbound/unbound.conf.d/pi-hole.conf:    verbosity: 0
/etc/unbound/unbound.conf.d/pi-hole.conf:    interface: 127.0.0.1
/etc/unbound/unbound.conf.d/pi-hole.conf:    port: 5335
/etc/unbound/unbound.conf.d/pi-hole.conf:    do-ip4: yes
/etc/unbound/unbound.conf.d/pi-hole.conf:    do-udp: yes
/etc/unbound/unbound.conf.d/pi-hole.conf:    do-tcp: yes
/etc/unbound/unbound.conf.d/pi-hole.conf:    do-ip6: no
/etc/unbound/unbound.conf.d/pi-hole.conf:    prefer-ip6: no
/etc/unbound/unbound.conf.d/pi-hole.conf:    harden-glue: yes
/etc/unbound/unbound.conf.d/pi-hole.conf:    harden-dnssec-stripped: yes
/etc/unbound/unbound.conf.d/pi-hole.conf:    use-caps-for-id: no
/etc/unbound/unbound.conf.d/pi-hole.conf:    edns-buffer-size: 1232
/etc/unbound/unbound.conf.d/pi-hole.conf:    prefetch: yes
/etc/unbound/unbound.conf.d/pi-hole.conf:    num-threads: 1
/etc/unbound/unbound.conf.d/pi-hole.conf:    so-rcvbuf: 1m
/etc/unbound/unbound.conf.d/pi-hole.conf:    private-address: 192.168.0.0/16
/etc/unbound/unbound.conf.d/pi-hole.conf:    private-address: 169.254.0.0/16
/etc/unbound/unbound.conf.d/pi-hole.conf:    private-address: 172.16.0.0/12
/etc/unbound/unbound.conf.d/pi-hole.conf:    private-address: 10.0.0.0/8
/etc/unbound/unbound.conf.d/pi-hole.conf:    private-address: fd00::/8
/etc/unbound/unbound.conf.d/pi-hole.conf:    private-address: fe80::/10
pi@ph5a:~ $ dpkg -L unbound
[..]
/etc/unbound/unbound.conf
/etc/unbound/unbound.conf.d
/etc/unbound/unbound.conf.d/qname-minimisation.conf
/etc/unbound/unbound.conf.d/root-auto-trust-anchor-file.conf
[..]
/etc/unbound/unbound.conf:include: "/etc/unbound/unbound.conf.d/*.conf"
/etc/unbound/unbound.conf.d/pi-hole.conf:server:
/etc/unbound/unbound.conf.d/pi-hole.conf:    verbosity: 0
/etc/unbound/unbound.conf.d/pi-hole.conf:    interface: 127.0.0.1
/etc/unbound/unbound.conf.d/pi-hole.conf:    port: 5335
/etc/unbound/unbound.conf.d/pi-hole.conf:    do-ip4: yes
/etc/unbound/unbound.conf.d/pi-hole.conf:    do-udp: yes
/etc/unbound/unbound.conf.d/pi-hole.conf:    do-tcp: yes
/etc/unbound/unbound.conf.d/pi-hole.conf:    do-ip6: no
/etc/unbound/unbound.conf.d/pi-hole.conf:    prefer-ip6: no
/etc/unbound/unbound.conf.d/pi-hole.conf:    harden-glue: yes
/etc/unbound/unbound.conf.d/pi-hole.conf:    harden-dnssec-stripped: yes
/etc/unbound/unbound.conf.d/pi-hole.conf:    use-caps-for-id: no
/etc/unbound/unbound.conf.d/pi-hole.conf:    edns-buffer-size: 1232
/etc/unbound/unbound.conf.d/pi-hole.conf:    prefetch: yes
/etc/unbound/unbound.conf.d/pi-hole.conf:    num-threads: 1
/etc/unbound/unbound.conf.d/pi-hole.conf:    so-rcvbuf: 1m
/etc/unbound/unbound.conf.d/pi-hole.conf:    private-address: 192.168.0.0/16
/etc/unbound/unbound.conf.d/pi-hole.conf:    private-address: 169.254.0.0/16
/etc/unbound/unbound.conf.d/pi-hole.conf:    private-address: 172.16.0.0/12
/etc/unbound/unbound.conf.d/pi-hole.conf:    private-address: 10.0.0.0/8
/etc/unbound/unbound.conf.d/pi-hole.conf:    private-address: fd00::/8
/etc/unbound/unbound.conf.d/pi-hole.conf:    private-address: fe80::/10
/etc/unbound/unbound.conf.d/root-auto-trust-anchor-file.conf:server:
/etc/unbound/unbound.conf.d/root-auto-trust-anchor-file.conf:    auto-trust-anchor-file: "/var/lib/unbound/root.key"
/etc/unbound/unbound.conf.d/qname-minimisation.conf:server:
/etc/unbound/unbound.conf.d/qname-minimisation.conf:    qname-minimisation: yes
/var/lib/unbound/root.key: No such file or directory
[1700468836] unbound-checkconf[5564:0] fatal error: auto-trust-anchor-file: "/var/lib/unbound/root.key" does not exist
-- Logs begin at Fri 2023-11-17 20:11:21 IST, end at Mon 2023-11-20 10:27:25 IST. --
Nov 17 22:08:50 raspberrypi systemd[1]: Stopped Unbound DNS server.
Nov 17 22:08:50 raspberrypi systemd[1]: Starting Unbound DNS server...
Nov 17 22:08:50 raspberrypi unbound[14317]: [14317:0] error: unable to open /var/lib/unbound/root.key for reading: No such file or directory
Nov 17 22:08:50 raspberrypi unbound[14317]: [14317:0] error: error reading auto-trust-anchor-file: /var/lib/unbound/root.key
Nov 17 22:08:50 raspberrypi unbound[14317]: [14317:0] error: validator: error in trustanchors config
Nov 17 22:08:50 raspberrypi unbound[14317]: [14317:0] error: validator: could not apply configuration settings.
Nov 17 22:08:50 raspberrypi unbound[14317]: [14317:0] error: module init for module validator failed
Nov 17 22:08:50 raspberrypi unbound[14317]: [14317:0] fatal error: failed to setup modules
Nov 17 22:08:50 raspberrypi systemd[1]: unbound.service: Main process exited, code=exited, status=1/FAILURE
Nov 17 22:08:50 raspberrypi systemd[1]: unbound.service: Failed with result 'exit-code'.
Nov 17 22:08:50 raspberrypi systemd[1]: Failed to start Unbound DNS server.
Nov 17 22:08:50 raspberrypi systemd[1]: unbound.service: Service RestartSec=100ms expired, scheduling restart.
Nov 17 22:08:50 raspberrypi systemd[1]: unbound.service: Scheduled restart job, restart counter is at 8.
Nov 17 22:08:50 raspberrypi systemd[1]: Stopped Unbound DNS server.
Nov 17 22:08:50 raspberrypi systemd[1]: Starting Unbound DNS server...
Nov 17 22:08:50 raspberrypi unbound[14328]: [14328:0] error: unable to open /var/lib/unbound/root.key for reading: No such file or directory
Nov 17 22:08:50 raspberrypi unbound[14328]: [14328:0] error: error reading auto-trust-anchor-file: /var/lib/unbound/root.key
Nov 17 22:08:50 raspberrypi unbound[14328]: [14328:0] error: validator: error in trustanchors config
Nov 17 22:08:50 raspberrypi unbound[14328]: [14328:0] error: validator: could not apply configuration settings.
Nov 17 22:08:50 raspberrypi unbound[14328]: [14328:0] error: module init for module validator failed
Nov 17 22:08:50 raspberrypi unbound[14328]: [14328:0] fatal error: failed to setup modules
Nov 17 22:08:50 raspberrypi systemd[1]: unbound.service: Main process exited, code=exited, status=1/FAILURE
Nov 17 22:08:50 raspberrypi systemd[1]: unbound.service: Failed with result 'exit-code'.
Nov 17 22:08:50 raspberrypi systemd[1]: Failed to start Unbound DNS server.
Nov 17 22:08:50 raspberrypi systemd[1]: unbound.service: Service RestartSec=100ms expired, scheduling restart.
Nov 17 22:08:50 raspberrypi systemd[1]: unbound.service: Scheduled restart job, restart counter is at 9.
Nov 17 22:08:50 raspberrypi systemd[1]: Stopped Unbound DNS server.
Nov 17 22:08:50 raspberrypi systemd[1]: unbound.service: Start request repeated too quickly.
Nov 17 22:08:50 raspberrypi systemd[1]: unbound.service: Failed with result 'exit-code'.
Nov 17 22:08:50 raspberrypi systemd[1]: Failed to start Unbound DNS server.
/.
/etc
/etc/apparmor.d
/etc/apparmor.d/usr.sbin.unbound
/etc/default
/etc/init.d
/etc/init.d/unbound
/etc/insserv.conf.d
/etc/insserv.conf.d/unbound
/etc/resolvconf
/etc/resolvconf/update.d
/etc/resolvconf/update.d/unbound
/etc/unbound
/etc/unbound/unbound.conf
/etc/unbound/unbound.conf.d
/etc/unbound/unbound.conf.d/qname-minimisation.conf
/etc/unbound/unbound.conf.d/root-auto-trust-anchor-file.conf
/lib
/lib/systemd
/lib/systemd/system
/lib/systemd/system/unbound-resolvconf.service
/lib/systemd/system/unbound.service
/usr
/usr/lib
/usr/lib/resolvconf
/usr/lib/resolvconf/dpkg-event.d
/usr/lib/resolvconf/dpkg-event.d/unbound
/usr/lib/unbound
/usr/lib/unbound/package-helper
/usr/sbin
/usr/sbin/unbound
/usr/sbin/unbound-checkconf
/usr/sbin/unbound-control
/usr/sbin/unbound-control-setup
/usr/share
/usr/share/doc
/usr/share/doc/unbound
/usr/share/doc/unbound/CREDITS
/usr/share/doc/unbound/FEATURES
/usr/share/doc/unbound/NEWS.Debian.gz
/usr/share/doc/unbound/README.DNS64
/usr/share/doc/unbound/README.gz
/usr/share/doc/unbound/TODO.gz
/usr/share/doc/unbound/changelog.Debian.gz
/usr/share/doc/unbound/changelog.gz
/usr/share/doc/unbound/contrib
/usr/share/doc/unbound/contrib/update-anchor.sh.gz
/usr/share/doc/unbound/copyright
/usr/share/doc/unbound/examples
/usr/share/doc/unbound/examples/unbound.conf
/usr/share/man
/usr/share/man/man5
/usr/share/man/man5/unbound.conf.5.gz
/usr/share/man/man8
/usr/share/man/man8/unbound-checkconf.8.gz
/usr/share/man/man8/unbound-control.8.gz
/usr/share/man/man8/unbound.8.gz
/usr/share/munin
/usr/share/munin/plugins
/usr/share/munin/plugins/unbound_munin_
/var
/var/lib
/var/lib/unbound

I removed Unbound and deleted the /etc/unbound folder.
Then I reinstalled it using the Pihole documentation, and you can see the results. Something there is not working as intended.

pi@raspberrypi:~ $ journalctl -u unbound.service

-- Logs begin at Fri 2023-11-17 20:11:21 IST, end at Mon 2023-11-20 10:48:33 IST. --
Nov 17 20:11:27 raspberrypi systemd[1]: Starting Unbound DNS server...
Nov 17 20:11:28 raspberrypi package-helper[477]: [1700244688] unbound-checkconf[481:0] error: Could not open /etc/unbounNov 17 20:11:28 raspberrypi package-helper[520]: [1700244688] unbound-checkconf[524:0] error: Could not open /etc/unbounNov 17 20:11:28 raspberrypi unbound[527]: [1700244688] unbound[527:0] error: Could not open /etc/unbound/unbound.conf: NNov 17 20:11:28 raspberrypi unbound[527]: [1700244688] unbound[527:0] warning: Continuing with default config settings
Nov 17 20:11:28 raspberrypi unbound[527]: [527:0] notice: init module 0: subnet
Nov 17 20:11:28 raspberrypi unbound[527]: [527:0] notice: init module 1: validator
Nov 17 20:11:28 raspberrypi unbound[527]: [527:0] notice: init module 2: iterator
Nov 17 20:11:28 raspberrypi unbound[527]: [527:0] info: start of service (unbound 1.9.0).
Nov 17 20:11:28 raspberrypi systemd[1]: Started Unbound DNS server.
Nov 17 20:12:46 raspberrypi unbound[527]: [527:0] info: service stopped (unbound 1.9.0).
Nov 17 20:12:46 raspberrypi systemd[1]: Stopping Unbound DNS server...
Nov 17 20:12:46 raspberrypi unbound[527]: [527:0] info: server stats for thread 0: 22 queries, 13 answers from cache, 9
Nov 17 20:12:46 raspberrypi unbound[527]: [527:0] info: server stats for thread 0: requestlist max 6 avg 1.22222 exceedeNov 17 20:12:46 raspberrypi unbound[527]: [527:0] info: average recursion processing time 0.305827 sec
Nov 17 20:12:46 raspberrypi unbound[527]: [527:0] info: histogram of recursion processing times
Nov 17 20:12:46 raspberrypi unbound[527]: [527:0] info: [25%]=0.16384 median[50%]=0.340787 [75%]=0.458752
Nov 17 20:12:46 raspberrypi unbound[527]: [527:0] info: lower(secs) upper(secs) recursions
Nov 17 20:12:46 raspberrypi unbound[527]: [527:0] info:    0.000000    0.000001 2
Nov 17 20:12:46 raspberrypi unbound[527]: [527:0] info:    0.131072    0.262144 1
Nov 17 20:12:46 raspberrypi unbound[527]: [527:0] info:    0.262144    0.524288 5
Nov 17 20:12:46 raspberrypi unbound[527]: [527:0] info:    0.524288    1.000000 1
Nov 17 20:12:46 raspberrypi systemd[1]: unbound.service: Succeeded.
Nov 17 20:12:46 raspberrypi systemd[1]: Stopped Unbound DNS server.
Nov 17 20:13:49 raspberrypi systemd[1]: Starting Unbound DNS server...
Nov 17 20:13:49 raspberrypi package-helper[1446]: [1700244829] unbound-checkconf[1448:0] error: Could not open /etc/unboNov 17 20:13:49 raspberrypi package-helper[1449]: [1700244829] unbound-checkconf[1451:0] error: Could not open /etc/unboNov 17 20:13:49 raspberrypi unbound[1452]: [1700244829] unbound[1452:0] error: Could not open /etc/unbound/unbound.conf:Nov 17 20:13:49 raspberrypi unbound[1452]: [1700244829] unbound[1452:0] warning: Continuing with default config settingsNov 17 20:13:49 raspberrypi unbound[1452]: [1700244829] unbound[1452:0] error: can't bind socket: Address already in useNov 17 20:13:49 raspberrypi unbound[1452]: [1700244829] unbound[1452:0] fatal error: could not open ports
Nov 17 20:13:49 raspberrypi systemd[1]: unbound.service: Main process exited, code=exited, status=1/FAILURE
Nov 17 20:13:49 raspberrypi systemd[1]: unbound.service: Failed with result 'exit-code'.
Nov 17 20:13:49 raspberrypi systemd[1]: Failed to start Unbound DNS server.
Nov 17 20:13:50 raspberrypi systemd[1]: unbound.service: Service RestartSec=100ms expired, scheduling restart.
Nov 17 20:13:50 raspberrypi systemd[1]: unbound.service: Scheduled restart job, restart counter is at 1.
Nov 17 20:13:50 raspberrypi systemd[1]: Stopped Unbound DNS server.
Nov 17 20:13:50 raspberrypi systemd[1]: Starting Unbound DNS server...
Nov 17 20:13:50 raspberrypi package-helper[1474]: [1700244830] unbound-checkconf[1477:0] error: Could not open /etc/unboNov 17 20:13:50 raspberrypi package-helper[1478]: [1700244830] unbound-checkconf[1481:0] error: Could not open /etc/unboNov 17 20:13:51 raspberrypi unbound[1495]: [1700244831] unbound[1495:0] error: Could not open /etc/unbound/unbound.conf:Nov 17 20:13:51 raspberrypi unbound[1495]: [1700244831] unbound[1495:0] warning: Continuing with default config settingsNov 17 20:13:51 raspberrypi unbound[1495]: [1700244831] unbound[1495:0] error: can't bind socket: Address already in useNov 17 20:13:51 raspberrypi unbound[1495]: [1700244831] unbound[1495:0] fatal error: could not open ports
Nov 17 20:13:51 raspberrypi systemd[1]: unbound.service: Main process exited, code=exited, status=1/FAILURE
Nov 17 20:13:51 raspberrypi systemd[1]: unbound.service: Failed with result 'exit-code'.
Nov 17 20:13:51 raspberrypi systemd[1]: Failed to start Unbound DNS server.
Nov 17 20:13:51 raspberrypi systemd[1]: Stopped Unbound DNS server.
Nov 17 20:13:51 raspberrypi systemd[1]: Starting Unbound DNS server...
Nov 17 20:13:51 raspberrypi package-helper[1500]: [1700244831] unbound-checkconf[1502:0] error: Could not open /etc/unboNov 17 20:13:51 raspberrypi package-helper[1503]: [1700244831] unbound-checkconf[1505:0] error: Could not open /etc/unboNov 17 20:13:51 raspberrypi unbound[1506]: [1700244831] unbound[1506:0] error: Could not open /etc/unbound/unbound.conf:Nov 17 20:13:51 raspberrypi unbound[1506]: [1700244831] unbound[1506:0] warning: Continuing with default config settingsNov 17 20:13:51 raspberrypi unbound[1506]: [1700244831] unbound[1506:0] error: can't bind socket: Address already in useNov 17 20:13:51 raspberrypi unbound[1506]: [1700244831] unbound[1506:0] fatal error: could not open ports
Nov 17 20:13:51 raspberrypi systemd[1]: unbound.service: Main process exited, code=exited, status=1/FAILURE

After a system reboot:

pi@raspberrypi:~ $ journalctl -u unbound.service

[527:0] error: unable to open /var/lib/unbound/root.key for reading: No such f
Nov 20 11:03:10 raspberrypi unbound[527]: [527:0] error: error reading auto-trust-anchor-file: /var/lib/unbound/root.key
Nov 20 11:03:10 raspberrypi unbound[527]: [527:0] error: validator: error in trustanchors config
Nov 20 11:03:10 raspberrypi unbound[527]: [527:0] error: validator: could not apply configuration settings.
Nov 20 11:03:10 raspberrypi unbound[527]: [527:0] error: module init for module validator failed
Nov 20 11:03:10 raspberrypi unbound[527]: [527:0] fatal error: failed to setup modules

What do below five output?

sudo systemctl cat unbound.service

sudo systemctl stop unbound.service

sudo mv /var/lib/unbound/root.key ~

sudo sh -x /usr/lib/unbound/package-helper root_trust_anchor_update

stat /var/lib/unbound/root.key

[Unit]
Description=Unbound DNS server
Documentation=man:unbound(8)
After=network.target
Before=nss-lookup.target
Wants=nss-lookup.target

[Service]
Type=notify
Restart=on-failure
EnvironmentFile=-/etc/default/unbound
ExecStartPre=-/usr/lib/unbound/package-helper chroot_setup
ExecStartPre=-/usr/lib/unbound/package-helper root_trust_anchor_update
ExecStart=/usr/sbin/unbound -d $DAEMON_OPTS
ExecReload=/usr/sbin/unbound-control reload
PIDFile=/run/unbound.pid

[Install]
WantedBy=multi-user.target
+ UNBOUND_CONF=/etc/unbound/unbound.conf
+ dirname /etc/unbound/unbound.conf
+ UNBOUND_BASE_DIR=/etc/unbound
+ unbound-checkconf -o chroot
+ CHROOT_DIR=
+ DNS_ROOT_KEY_FILE=/usr/share/dns/root.key
+ ROOT_TRUST_ANCHOR_FILE=/var/lib/unbound/root.key
+ RESOLVCONF=true
+ ROOT_TRUST_ANCHOR_UPDATE=true
+ [ -f /etc/default/unbound ]
+ . /etc/default/unbound
+ ROOT_TRUST_ANCHOR_UPDATE=false
+ ROOT_TRUST_ANCHOR_UPDATE=false
+ do_root_trust_anchor_update
+ false
stat: cannot stat '/var/lib/unbound/root.key': No such file or directory

It doesnt want to update/create that missing file /var/lib/unbound/root.key.

What does below show?

cat /etc/default/unbound

FYI, I dont have above file on my buster release:

pi@ph5a:~ $ cat /etc/default/unbound
cat: /etc/default/unbound: No such file or directory

ROOT_TRUST_ANCHOR_UPDATE="false"

Any thoughts and how to start a fresh install (reset) unbound, without resetting my entire pihole installation?

Remove that file with:

sudo rm /etc/default/unbound

Run below one again:

sudo sh -x /usr/lib/unbound/package-helper root_trust_anchor_update

Check if that missing file exists now:

stat /var/lib/unbound/root.key

If exists, restart:

sudo systemctl restart unbound.service

And check status and journals again:

sudo systemctl status unbound.service

sudo journalctl --full --no-pager --lines 30 -u unbound.service

EDIT: Oh and run below one too just to be sure:

sudo unbound-checkconf

Most likey wont need to if above steps work.

 File: /var/lib/unbound/root.key
  Size: 758             Blocks: 8          IO Block: 4096   regular file
Device: b302h/45826d    Inode: 1838074     Links: 1
Access: (0644/-rw-r--r--)  Uid: (  111/ unbound)   Gid: (  118/ unbound)
Access: 2023-11-20 22:02:15.551226929 +0200
Modify: 2023-11-20 22:02:15.551226929 +0200
Change: 2023-11-20 22:02:15.561226920 +0200
 Birth: -
● unbound.service - Unbound DNS server
   Loaded: loaded (/lib/systemd/system/unbound.service; enabled; vendor preset: enabled)
   Active: active (running) since Mon 2023-11-20 22:02:37 IST; 4s ago
     Docs: man:unbound(8)
  Process: 5464 ExecStartPre=/usr/lib/unbound/package-helper chroot_setup (code=exited, status=0/SUCCESS)
  Process: 5467 ExecStartPre=/usr/lib/unbound/package-helper root_trust_anchor_update (code=exited, status=0/SUCCESS)
 Main PID: 5471 (unbound)
    Tasks: 1 (limit: 2059)
   CGroup: /system.slice/unbound.service
           └─5471 /usr/sbin/unbound -d

Nov 20 22:02:37 raspberrypi systemd[1]: Starting Unbound DNS server...
Nov 20 22:02:37 raspberrypi package-helper[5467]: /var/lib/unbound/root.key has content
Nov 20 22:02:37 raspberrypi package-helper[5467]: success: the anchor is ok
Nov 20 22:02:37 raspberrypi unbound[5471]: [5471:0] info: start of service (unbound 1.9.0).
Nov 20 22:02:37 raspberrypi systemd[1]: Started Unbound DNS server.
-- Logs begin at Mon 2023-11-20 11:03:03 IST, end at Mon 2023-11-20 22:02:52 IST. --
Nov 20 11:12:43 raspberrypi unbound[1183]: [1700471563] unbound[1183:0] error: unable to open root.key for reading: No such file or directory
Nov 20 11:12:43 raspberrypi unbound[1183]: [1700471563] unbound[1183:0] error: error reading auto-trust-anchor-file: root.key
Nov 20 11:12:43 raspberrypi unbound[1183]: [1700471563] unbound[1183:0] error: validator: error in trustanchors config
Nov 20 11:12:43 raspberrypi unbound[1183]: [1700471563] unbound[1183:0] error: validator: could not apply configuration settings.
Nov 20 11:12:43 raspberrypi unbound[1183]: [1700471563] unbound[1183:0] error: module init for module validator failed
Nov 20 11:12:43 raspberrypi unbound[1183]: [1700471563] unbound[1183:0] fatal error: failed to setup modules
Nov 20 11:12:43 raspberrypi systemd[1]: unbound.service: Main process exited, code=exited, status=1/FAILURE
Nov 20 11:12:43 raspberrypi systemd[1]: unbound.service: Failed with result 'exit-code'.
Nov 20 11:12:43 raspberrypi systemd[1]: Failed to start Unbound DNS server.
Nov 20 11:12:43 raspberrypi systemd[1]: unbound.service: Service RestartSec=100ms expired, scheduling restart.
Nov 20 11:12:43 raspberrypi systemd[1]: unbound.service: Scheduled restart job, restart counter is at 5.
Nov 20 11:12:43 raspberrypi systemd[1]: Stopped Unbound DNS server.
Nov 20 11:12:43 raspberrypi systemd[1]: unbound.service: Start request repeated too quickly.
Nov 20 11:12:43 raspberrypi systemd[1]: unbound.service: Failed with result 'exit-code'.
Nov 20 11:12:43 raspberrypi systemd[1]: Failed to start Unbound DNS server.
Nov 20 11:12:45 raspberrypi systemd[1]: unbound.service: Start request repeated too quickly.
Nov 20 11:12:45 raspberrypi systemd[1]: unbound.service: Failed with result 'exit-code'.
Nov 20 11:12:45 raspberrypi systemd[1]: Failed to start Unbound DNS server.
Nov 20 11:19:03 raspberrypi systemd[1]: Starting Unbound DNS server...
Nov 20 11:19:03 raspberrypi unbound[1315]: [1315:0] info: start of service (unbound 1.9.0).
Nov 20 11:19:03 raspberrypi systemd[1]: Started Unbound DNS server.
Nov 20 20:23:14 raspberrypi unbound[1315]: [1315:0] info: service stopped (unbound 1.9.0).
Nov 20 20:23:14 raspberrypi systemd[1]: Stopping Unbound DNS server...
Nov 20 20:23:14 raspberrypi systemd[1]: unbound.service: Succeeded.
Nov 20 20:23:14 raspberrypi systemd[1]: Stopped Unbound DNS server.
Nov 20 22:02:37 raspberrypi systemd[1]: Starting Unbound DNS server...
Nov 20 22:02:37 raspberrypi package-helper[5467]: /var/lib/unbound/root.key has content
Nov 20 22:02:37 raspberrypi package-helper[5467]: success: the anchor is ok
Nov 20 22:02:37 raspberrypi unbound[5471]: [5471:0] info: start of service (unbound 1.9.0).
Nov 20 22:02:37 raspberrypi systemd[1]: Started Unbound DNS server.

unbound-checkconf: no errors in /etc/unbound/unbound.conf

Ok, so now I have a new Unbound installation and the official pihole config, yet the issue persists.

dig sigok.verteiltesysteme.net @127.0.0.1 -p 5335

; <<>> DiG 9.11.5-P4-5.1+deb10u9-Raspbian <<>> sigok.verteiltesysteme.net @127.0.0.1 -p 5335
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 44795
;; flags: qr rd ra ad; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
;; QUESTION SECTION:
;sigok.verteiltesysteme.net.    IN      A

;; ANSWER SECTION:
sigok.verteiltesysteme.net. 1799 IN     CNAME   sigok.rsa2048-sha256.ippacket.stream.
sigok.rsa2048-sha256.ippacket.stream. 60 IN A   195.201.14.36

;; Query time: 135 msec
;; SERVER: 127.0.0.1#5335(127.0.0.1)
;; WHEN: Mon Nov 20 22:05:23 IST 2023
;; MSG SIZE  rcvd: 121

dig e-services.clalit.co.il @127.0.0.1 -p 5335


; <<>> DiG 9.11.5-P4-5.1+deb10u9-Raspbian <<>> e-services.clalit.co.il @127.0.0.1 -p 5335
;; global options: +cmd
;; connection timed out; no servers could be reached

Up verbosity to 3, reload unbound and tail the journals live when running above dig.
Post results here.

verbosity is up to 3, and restarted unbound using sudo systemctl restart unbound.service

then I ran sudo journalctl --full --follow --priority 4 -u unbound.service
which gives:

-- Logs begin at Mon 2023-11-20 11:03:03 IST. --
Nov 20 11:12:42 raspberrypi systemd[1]: unbound.service: Failed with result 'exit-code'.
Nov 20 11:12:42 raspberrypi systemd[1]: Failed to start Unbound DNS server.
Nov 20 11:12:43 raspberrypi systemd[1]: unbound.service: Failed with result 'exit-code'.
Nov 20 11:12:43 raspberrypi systemd[1]: Failed to start Unbound DNS server.
Nov 20 11:12:43 raspberrypi systemd[1]: unbound.service: Start request repeated too quickly.
Nov 20 11:12:43 raspberrypi systemd[1]: unbound.service: Failed with result 'exit-code'.
Nov 20 11:12:43 raspberrypi systemd[1]: Failed to start Unbound DNS server.
Nov 20 11:12:45 raspberrypi systemd[1]: unbound.service: Start request repeated too quickly.
Nov 20 11:12:45 raspberrypi systemd[1]: unbound.service: Failed with result 'exit-code'.
Nov 20 11:12:45 raspberrypi systemd[1]: Failed to start Unbound DNS server.

when running dig e-services.clalit.co.il @127.0.0.1 -p 5335 in a new terminal window, the log doesn't update at all. It still shows the same thing.

Maybe by upping verbosity, you did something wrong .. typo etc.
Check below now:

sudo unbound-checkconf

Dont run that one now, tail live with below:

sudo journalctl --full --follow -u unbound.service

Below are all the equivalent dig queries my unbound instance makes (from the journals) when trying to resolve that e-services.clalit.co.il domain.
I suspect one or more failing to connect for your unbound instance.

pi@ph5a:~ $ ./unbound_check.sh unbound.good.journals | column -t
dig  +norecurse  @202.12.27.33     .                         NS      IN
dig  +norecurse  @192.36.148.17    il.                       A       IN
dig  +norecurse  @192.115.7.53     co.il.                    A       IN
dig  +norecurse  @192.115.4.235    clalit.co.il.             A       IN
dig  +norecurse  @202.12.27.33     net.                      A       IN
dig  +norecurse  @202.12.27.33     net.                      A       IN
dig  +norecurse  @192.112.36.4     net.                      A       IN
dig  +norecurse  @192.55.83.30     bezeqint.net.             A       IN
dig  +norecurse  @192.41.162.30    bezeqint.net.             A       IN
dig  +norecurse  @212.179.7.7      ns3.bezeqint.net.         A       IN
dig  +norecurse  @212.179.7.7      ns2.bezeqint.net.         A       IN
dig  +norecurse  @192.35.51.30     bezeqint.net.             A       IN
dig  +norecurse  @192.115.132.132  ns1.bezeqint.net.         A       IN
dig  +norecurse  @212.179.7.7      e-services.clalit.co.il.  A       IN
dig  +norecurse  @192.43.172.30    impervadns.net.           A       IN
dig  +norecurse  @198.143.61.165   ucetyes.impervadns.net.   A       IN
dig  +norecurse  @192.112.36.4     .                         DNSKEY  IN
dig  +norecurse  @198.41.0.4       _ta-4f66.                 A       IN
dig  +norecurse  @204.61.216.134   il.                       DNSKEY  IN
dig  +norecurse  @194.146.106.122  co.il.                    DS      IN
dig  +norecurse  @194.146.106.122  co.il.                    DNSKEY  IN
dig  +norecurse  @192.26.92.30     net.                      DNSKEY  IN
dig  +norecurse  @198.143.63.165   impervadns.net.           DNSKEY  IN

They all reply on my Raspi (no connection timed out).
Can check the same on your node (copy/paste ENTER).
I know they are allot but thats just how recursive resolvers like unbound do it :wink: