LXC and FTL permissions

fred.c · February 10, 2023, 8:58pm

I have the exact same issue reported by MightySlaytanic. Also using an LXC container for my pihole, running debian buster.

fred.c · February 10, 2023, 9:02pm

I've just tried updating but the issue persists.
Here's my debug log: https://tricorder.pi-hole.net/F30JHmfo/
Thank you for your help.

DanSchaper · February 10, 2023, 9:04pm

Please provide more details. What exactly are you seeing, what are the status outputs, anything that can be reviewed to see.

fred.c · February 10, 2023, 9:11pm

Basically the same as described by MightySlaytanic above.
Running debian buster on a LXC container in proxmox.
When running pihole -up I get the same error:

[✗] Checking apt-get for upgraded packages
      Kernel update detected. If the install fails, please reboot and try again

and it ends at:

[i] Restarting pihole-FTL service...
  Unable to complete update, please contact Pi-hole Support

I've just run it again after 5.15.5, but am getting the same result

fred.c · February 10, 2023, 9:21pm

For me lighttpd is working fine:

root@pihole-2:~# systemctl status lighttpd
* lighttpd.service - Lighttpd Daemon
   Loaded: loaded (/lib/systemd/system/lighttpd.service; enabled; vendor preset: enabled)
   Active: active (running) since Fri 2023-02-10 21:15:28 UTC; 2min 13s ago
  Process: 139 ExecStartPre=/usr/sbin/lighttpd -tt -f /etc/lighttpd/lighttpd.conf (code=exited, status=0/SUCCESS)
 Main PID: 164 (lighttpd)
    Tasks: 11 (limit: 38192)
   Memory: 42.0M
      CPU: 157ms
   CGroup: /system.slice/lighttpd.service
           |-164 /usr/sbin/lighttpd -D -f /etc/lighttpd/lighttpd.conf
           |-169 /usr/bin/php-cgi
           |-171 /usr/bin/php-cgi
           |-175 /usr/bin/php-cgi
           |-176 /usr/bin/php-cgi
           |-177 /usr/bin/php-cgi
           |-178 /usr/bin/php-cgi
           |-183 /usr/bin/php-cgi
           |-184 /usr/bin/php-cgi
           |-185 /usr/bin/php-cgi
           `-186 /usr/bin/php-cgi

Feb 10 21:15:28 pihole-2 systemd[1]: Starting Lighttpd Daemon...
Feb 10 21:15:28 pihole-2 systemd[1]: Started Lighttpd Daemon.
Feb 10 21:17:09 pihole-2 sudo[2613]: www-data : TTY=unknown ; PWD=/var/www/html/admin ; USER=root ; COMMAND=/usr/local/bin/pihole -a -c
Feb 10 21:17:09 pihole-2 sudo[2613]: pam_unix(sudo:session): session opened for user root by (uid=0)
Feb 10 21:17:09 pihole-2 sudo[2613]: pam_unix(sudo:session): session closed for user root

My problem seems to be with pihole-FTL

root@pihole-2:~# pihole status
  [✗] DNS service is NOT running
root@pihole-2:~# pihole restartdns
  [✗] Job for pihole-FTL.service failed because the control process exited with error code.
See "systemctl status pihole-FTL.service" and "journalctl -xe" for details.
root@pihole-2:~# systemctl status pihole-FTL.service
* pihole-FTL.service - Pi-hole FTL
   Loaded: loaded (/etc/systemd/system/pihole-FTL.service; enabled; vendor preset: enabled)
   Active: activating (auto-restart) (Result: exit-code) since Fri 2023-02-10 21:20:58 UTC; 3s ago
  Process: 7608 ExecStartPre=/opt/pihole/pihole-FTL-prestart.sh (code=exited, status=226/NAMESPACE)
  Process: 7609 ExecStopPost=/opt/pihole/pihole-FTL-poststop.sh (code=exited, status=226/NAMESPACE)
      CPU: 3ms

Feb 10 21:20:58 pihole-2 systemd[1]: pihole-FTL.service: Control process exited, code=exited, status=226/NAMESPACE
Feb 10 21:20:58 pihole-2 systemd[1]: pihole-FTL.service: Failed with result 'exit-code'.
Feb 10 21:20:58 pihole-2 systemd[1]: Failed to start Pi-hole FTL.
Feb 10 21:20:58 pihole-2 systemd[1]: pihole-FTL.service: Consumed 3ms CPU time.

Thank you for your help

DanSchaper · February 10, 2023, 9:29pm

That looks like a permission issue with your LXC environment. A quick google of code=exited status=226/namespace leads to some discussions of /var/tmp being mapped. Might want to search around for that.

DanSchaper · February 10, 2023, 9:32pm

This is the systemd file that we use, it includes some protection and isolation that may cause issues if you do not have things configured on the LXC side with this in mind.

github.com/pi-hole/pi-hole

advanced/Templates/pihole-FTL.systemd

d86b325df

[Unit]
Description=Pi-hole FTL
# This unit is supposed to indicate when network functionality is available, but it is only
# very weakly defined what that is supposed to mean, with one exception: at shutdown, a unit
# that is ordered after network-online.target will be stopped before the network
Wants=network-online.target
After=network-online.target
# A target that should be used as synchronization point for all host/network name service lookups.
# All services for which the availability of full host/network name resolution is essential should
# be ordered after this target, but not pull it in.
Wants=nss-lookup.target
Before=nss-lookup.target

# Limit (re)start loop to 5 within 1 minute
StartLimitBurst=5
StartLimitIntervalSec=60s

[Service]
User=pihole
PermissionsStartOnly=true

This file has been truncated. show original

fred.c · February 10, 2023, 9:55pm

Thank you so much for your help.
This is strange - pihole has been working fine in this container for over two years. Updated it several times without any error. It's a privileged LXC.
When I restore the backup it works fine, but when I try to update it gives me the error. I think it was working fine up until v. 5.19.
When i try to restore as unprivileged container the restore fails.
I'm still learning my way around proxmox and LXC - any suggestion on how to solve the issue? Been googling it but is a bit confused. Should I enable "nesting"?

MightySlaytanic · February 10, 2023, 10:39pm

Hi, I’m running it in an LXC unprivileged container without issues. There was a problem that has been solved few minutes ago and now the update works fine. The message about kernel is fine since pi-hole doesn’t find some kernel folders but this is due to the fact that the container relies on the PVE host kernel.

Btw this is my LXC container config:

arch: amd64
cores: 6
features: nesting=1
hostname: DNS-Stack
memory: 2048
nameserver: 10.0.0.254
net0: name=eth0,bridge=vmbr0,firewall=1,gw=10.0.0.254,hwaddr=B6:D6:0D:2A:FC:FC,ip=10.0.0.6/24,type=veth
onboot: 1
ostype: debian
rootfs: local-lvm:vm-105-disk-1,size=8G
searchdomain: homenetwork
snaptime: 1676056693
startup: order=2
swap: 512
unprivileged: 1

DanSchaper · February 10, 2023, 11:35pm

Some of the topics I've seen mention mount points and mapping. Can you show your mount output?

fred.c · February 13, 2023, 9:58pm

Thank you! This was most helpful in building a new LXC container. Working fine now!

fred.c · February 13, 2023, 10:05pm

Thank you for your help and apologies for the delay in replying.

I´ve actually decided to rebuild the LXC container as unprivileged, and in the process upgraded to Debian 11 (and also increased the disk so I don´t have to worry about erasing the FTL database in the future). All is working fine now.

I did took the output of mount before rebuilding, though:

root@pihole-2:~# mount
/dev/mapper/pve-vm--101--disk--0 on / type ext4 (rw,relatime,stripe=16)
none on /dev type tmpfs (rw,relatime,size=492k,mode=755,inode64)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
proc on /proc/sys/net type proc (rw,nosuid,nodev,noexec,relatime)
proc on /proc/sys type proc (ro,relatime)
proc on /proc/sysrq-trigger type proc (ro,relatime)
sysfs on /sys type sysfs (ro,nosuid,nodev,noexec,relatime)
sysfs on /sys/devices/virtual/net type sysfs (rw,nosuid,nodev,noexec,relatime)
fusectl on /sys/fs/fuse/connections type fusectl (rw,nosuid,nodev,noexec,relatime)
none on /sys/fs/cgroup type cgroup2 (rw,nosuid,nodev,noexec,relatime)
lxcfs on /proc/cpuinfo type fuse.lxcfs (rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other)
lxcfs on /proc/diskstats type fuse.lxcfs (rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other)
lxcfs on /proc/loadavg type fuse.lxcfs (rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other)
lxcfs on /proc/meminfo type fuse.lxcfs (rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other)
lxcfs on /proc/slabinfo type fuse.lxcfs (rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other)
lxcfs on /proc/stat type fuse.lxcfs (rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other)
lxcfs on /proc/swaps type fuse.lxcfs (rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other)
lxcfs on /proc/uptime type fuse.lxcfs (rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other)
lxcfs on /sys/devices/system/cpu type fuse.lxcfs (rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=666,max=1026)
devpts on /dev/ptmx type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=666,max=1026)
devpts on /dev/console type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=666,max=1026)
devpts on /dev/tty1 type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=666,max=1026)
devpts on /dev/tty2 type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=666,max=1026)
none on /proc/sys/kernel/random/boot_id type tmpfs (ro,nosuid,nodev,noexec,relatime,size=492k,mode=755,inode64)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev,inode64)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755,inode64)
tmpfs on /run/lock type tmpfs (rw,nosuid,nodev,noexec,relatime,size=5120k,inode64)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime,pagesize=2M)
tmpfs on /run/user/998 type tmpfs (rw,nosuid,nodev,relatime,size=6563428k,mode=700,uid=998,gid=998,inode64)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=6563428k,mode=700,inode64)

Something must have changed in the last version as my instance of pihole was working fine for a couple of years up until the last update. But an update was due anyway.

Thanks once again for your help.

JGC · February 17, 2023, 3:56pm

I was hit by the same problem, running pihole inside an unprivileged LXC container with Proxmox as host OS.

The reason why this breaks is the new systemd unit file for FTL which has these options:
ProtectSystem=full
ReadWriteDirectories=/etc/pihole

To fix the problem, the container should have nesting enabled in Options. This also applies for regular LXC containers, lxc config set {container-name} security.nesting true.

Trimbabimba · February 23, 2023, 8:20pm

I had the exact same issue and the change provided by JGC solved it.
For people who prefer the webinterface, nesting is under the options tab, the 'features' row in the table.

Enable nesting, boot the container back up and all is well again!

system · March 16, 2023, 8:20pm

This topic was automatically closed 21 days after the last reply. New replies are no longer allowed.