Conflict with other webservices running on Pi (Ports 80 and 443)

[bcutter]

Alright, after a few weeks there´s only one thing freaking me out: from time to time the router mixes up the MAC-to-IP-binding for the system running pi-hole and other web services. I need to edit the static IP in the router (FritzBox) config and need to restart the pi (only restarting networking service or anything like that doesn´t work). I think it´s a FritzBox bug, unable to manage two IPs for one MAC address. It´s like a "race condition", and sometimes the wrong IP address wins...

[deHakkelaar]

Try the "arp" command.
My router allows telnet so I can check ARP on Pi as well as router.

Ohw Ps, I managed to have the relevant Pi-Hole daemons running on the aliased interface instead of the physical:

Run pi-hole on an additional ip

I was not able to assign a static IP to an aliased interface using the dhcpcd5 way. So i devised below making use of the default "source-directory /etc/network/interfaces.d" directive in the interfaces file. This is just prove of concept/an example as I dont have ipv6 configured at home. This test Pi only got WiFi so thats why the wlan0:0 (with ip 10.0.0.200). Create new file: sudo nano /etc/pihole/pihole-if Containing below aliased interface with second IP: auto wlan0:0 iface wlan0:0 in…

[bcutter]

My router doesn´t, so ARP is a one-side tool

I installed pi-hole exactly like in the link described, only on eth0:0 instead of wifi interface.

[deHakkelaar]

Thats bummer.
But is it this simple to enable the Fritzbox telnet daemon ?

https://www.fritzmod.net/modification/telnet/

It says you have to connect phone and dial the numbers

#96*7* Telnet einschalten
#96*8* Telnet ausschalten

[deHakkelaar]

Was thinking, if get in, and things go wrong again, below one will clear ARP so router and/or Pi need to rediscover MAC (try instead of reboot etc):

ip -s -s neigh flush all

Might need sudo.

[bcutter]

UPDATE: Don´t read until the "UPDATE" section at the end, just ignore it...

It´s really strange. This time, the problem is slightly different:

After adding "sunmaker.com" and "www.sunmaker.com" to the blocking list (at least that´s the only trigger imaginable for me, because there´s no router involved this time), on a Windows machine a "nslookup piholeandwebservername" gave me:

Server:  sunmaker.com
Address:  192.168.0.9"

*** piholeandwebservername hasn´t been found by sunmaker.com: Non-existent domain.

So i did a "ipconfig /flushdns" and again nslookuped to the piholeandwebservername while tailing the pihole.log. That log told me:

dnsmasq[10323]: /etc/pihole/local.list 192.168.0.9 is piholeandwebservername

local.list content:
192.168.0.9 piholeandwebservername
192.168.0.9 pi.hole

So, QUIZ QUESTIONS:
1) Why the hell is local.list using 192.168.0.9 (and not 192.168.0.15 as it should) for the piholeandwebservername?!?
2) What triggers the creation of local.list content?

Instant workaround (editing local.list manually) didn´t help.

UPDATE:
Forget everything above. A closer look to tail pihole.log pointed out that pi.hole forwarded the DNS request to the router... which - again - told pi.hole that piholeandwebservername is 192.168.0.9 instead of 192.168.0.15...

So once again: no telnet or ssh or ANY other baseline access to the router. AVM killed the enabling/disabling using phone code in it´s firmware around 2015.

[bcutter]

Update:

More debugging pointed out:

1. Using FQDN is very important. "nslookup piholeandwebservername" is NOT the same like "nslookup piholeandwebservername.fritz.box"
2. Specifing the DNS server helped a bit:

nslookup piholeandwebservername pi.hole

gave 192.168.0.9 --> WRONG (because pi.hole forwards the request to fritz.box)

nslookup piholeandwebservername fritz.box

gave 192.168.0.9 --> WRONG

nslookup piholeandwebservername.fritz.box pi.hole

gave 192.168.0.9 --> WRONG, BUT there´s a workaround:

a) Pi: nano /etc/pihole/local.list
b) Pi: Adding line "192.168.0.15 piholeandwebservername.fritz.box"
c) Pi: pihole restartdns
d) Windows endpoint: "ipconfig /flushdns" and now...

nslookup piholeandwebservername fritz.box fritz.box

still gives 192.168.0.9 --> WRONG, but:

nslookup piholeandwebservername.fritz.box pi.hole

now gives 192.168.0.15 --> RIGHT, because pi.hole uses the new line of local.list.

So finally: Not a nice workaround (especially because i still don´t know the root cause (what´s going on in the router/fritz.box?!?)), but at least it avoids rebooting the pi.

[deHakkelaar]

@bcutter

1. Why the hell is local.list using 192.168.0.9 (and not 192.168.0.15 as it should) for the piholeandwebservername?!?
2. What triggers the creation of local.list content?

pihole -r

As described in the concept howto:

Run pi-hole on an additional ip

I was not able to assign a static IP to an aliased interface using the dhcpcd5 way. So i devised below making use of the default "source-directory /etc/network/interfaces.d" directive in the interfaces file. This is just prove of concept/an example as I dont have ipv6 configured at home. This test Pi only got WiFi so thats why the wlan0:0 (with ip 10.0.0.200). Create new file: sudo nano /etc/pihole/pihole-if Containing below aliased interface with second IP: auto wlan0:0 iface wlan0:0 in…

[bcutter]

Basically that´s what I did (weeks ago already). The wrong IP for the webserver name comes from FritzBox als already mentioned.

Meanwhile I made my workaround (editing /etc/pihole/local.list) permanent with a "sudo chattr +i /etc/pihole/local.list" so it can´t be updated with wrong values, not even by root.

Will be excited to see what happens on the next pihole (core) update - just in case the setup/update routine tries to write the local.list.

But at the moment I´m quite happy with this "solution" and it seems I need to live with that because the router doesn´t properly handle the two IPs for my webserver/pihole machine.

[Mcat12]

The local.list file is written every time you run gravity, so it probably will work fine with the update.

[bcutter]

This seems to not work anymore:

Conflict with other webservices running on Pi (Ports 80 and 443)

Okay, got it. www.econda-monitor.de is showing the blocking page. Just to summarize (e. g. for other users facing similar issues / having similar setups with other web services running on the Pi-hole Pi): Using a second IP address is the basis for a clean service separation (not only but especially when your other web service is running on ports 80 oder 443). Don't forget to set the same IP address as lighttpd uses for Pi-hole/dnsmasq. Easiest way is to reconfigure Pi-hole (pihole -r) and say…

I'm currently in the progress of migrating a Pi-Hole v5 instance to another host and keep the v5 version and all settings, history etc.

Meanwhile on the new host, as /etc/network/interfaces is not being used anymore (Pi OS 12) I used

• /etc/systemd/network/eth0.network to set the static primary IP for the host (which should be used by the other service) --> 192.168.0.55 / eth0
• /etc/network/if-up.d/eth0-alias to set the static secondary (virtual) IP which should be used by Pi-Hole --> 192.168.0.99 / eth0:0

Now after installing v5 Pi-Hole on the new host and using

• /etc/lighttpd/external.conf with server.bind = "192.168.0.99" and

• /etc/pihole/pihole-FTL.conf with REPLY_ADDR4=192.168.0.99 and

• /etc/pihole/setupVars.conf with IPV4_ADDRESS=192.168.0.99/24
  as well as restarting lighttpd and pihole-FTL services, the problem is:

• a) Pi-Hole Admin Interface is reachable via http://192.168.0.99/admin (good) as well as via http://192.168.0.55 (bad, as this IP + port combination is designated for another web service to be migrated and set up later, using nginx)

• b) sudo netstat -nltp | grep 80 gives

  • tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN 254240/lighttpd on the new host while on the old host it gives
  • tcp 0 0 192.168.0.9:80 0.0.0.0:* LISTEN 1656/lighttpd tcp6 0 0 :::80 :::* LISTEN 1656/lighttpd

• c) http://192.168.0.55/admin/messages.php shows
  

  
  
  which according to dnsmasq warnings - Pi-hole documentation translates as
  
  
  and I'm not sure I ever saw this on the old host with the other method of setting up the virtual eth0:0 interface (not sure if related)
  --> fixed (not shown anymore) after changing DNSMASQ_LISTENING from locale to single in /etc/pihole/setupVars.conf and doing a pihole -r

So in the end it seems lighttpd is still not serving Pi-Hole exclusively via the configured 192.168.0.99.

The only things different to the old host is

1. Another OS release (12 instead of 11)
2. Another way of setting the virtual eth0:0 interface
3. The web service grabbing the physical eth0 interface / primary IP and port (192.168.0.55:80) is not running yet.
4. ifconfig on the new host has an additional line device interrupt 112 for both eth0 and eth0:0 while the old host does not.
5. /etc/dnsmasq.d/01-pihole.conf on the new host does not contain a few lines from the old host, e. g. it has local-service (new host) but not interface=eth0 (old host). Trying to edit this file makes pihole-FTL service to not start again (and those files shouldn't be changed at all as stated in their header sections). --> fixed after changing DNSMASQ_LISTENING from locale to single in /etc/pihole/setupVars.conf and doing a pihole -r

I meanwhile also already ran pihole -r (with custom changes to remain at the latest v5 release), unfortunately both by choosing the "repair" as well as the "reconfigure" path, it (still) defaults to the physical eth0 interface IP address:

I ran completely out of ideas for what reason and on how to convince Pi-Hole to not use the primary/physical eth0 IP (192.168.0.55) but only the virtual eth0:0 IP (192.168.0.99).

• Using PIHOLE_INTERFACE=eth0:0 in /etc/pihole/setupVars.conf instead of PIHOLE_INTERFACE=eth0 like on the old host did not change anything.

Any help is highly appreciated.

---

EDIT: Seems to be fixed by using a well-working /etc/lighttpd/lighttpd.conf (copied from the old host):

default /etc/lighttpd/lighttpd.conf after pi-hole (v5) installation

server.modules = (
        "mod_indexfile",
        "mod_access",
        "mod_alias",
        "mod_redirect",
)

server.document-root        = "/var/www/html"
server.upload-dirs          = ( "/var/cache/lighttpd/uploads" )
server.errorlog             = "/var/log/lighttpd/error.log"
server.pid-file             = "/run/lighttpd.pid"
server.username             = "www-data"
server.groupname            = "www-data"
server.port                 = 80

# features
#https://redmine.lighttpd.net/projects/lighttpd/wiki/Server_feature-flagsDetails
server.feature-flags       += ("server.h2proto" => "enable")
server.feature-flags       += ("server.h2c"     => "enable")
server.feature-flags       += ("server.graceful-shutdown-timeout" => 5)
#server.feature-flags       += ("server.graceful-restart-bg" => "enable")

# strict parsing and normalization of URL for consistency and security
# https://redmine.lighttpd.net/projects/lighttpd/wiki/Server_http-parseoptsDetails
# (might need to explicitly set "url-path-2f-decode" = "disable"
#  if a specific application is encoding URLs inside url-path)
server.http-parseopts = (
  "header-strict"           => "enable",# default
  "host-strict"             => "enable",# default
  "host-normalize"          => "enable",# default
  "url-normalize-unreserved"=> "enable",# recommended highly
  "url-normalize-required"  => "enable",# recommended
  "url-ctrls-reject"        => "enable",# recommended
  "url-path-2f-decode"      => "enable",# recommended highly (unless breaks app)
 #"url-path-2f-reject"      => "enable",
  "url-path-dotseg-remove"  => "enable",# recommended highly (unless breaks app)
 #"url-path-dotseg-reject"  => "enable",
 #"url-query-20-plus"       => "enable",# consistency in query string
)

index-file.names            = ( "index.php", "index.html" )
url.access-deny             = ( "~", ".inc" )
static-file.exclude-extensions = ( ".php", ".pl", ".fcgi" )

# default listening port for IPv6 falls back to the IPv4 port
include_shell "/usr/share/lighttpd/use-ipv6.pl " + server.port
include_shell "/usr/share/lighttpd/create-mime.conf.pl"
include "/etc/lighttpd/conf-enabled/*.conf"

#server.compat-module-load   = "disable"
server.modules += (
        "mod_dirlisting",
        "mod_staticfile",
)

old /etc/lighttpd/lighttpd.conf copied from old host with well-working pi-hole installation

# Pi-hole: A black hole for Internet advertisements
# (c) 2017 Pi-hole, LLC (https://pi-hole.net)
# Network-wide ad blocking via your own hardware.
#
# Lighttpd config for Pi-hole
#
# This file is copyright under the latest version of the EUPL.
# Please see LICENSE file for your rights under this license.

###################################################################################################
#   IF THIS HEADER EXISTS, THE FILE WILL BE OVERWRITTEN BY PI-HOLE'S UPDATE PROCEDURE.            #
#   ANY CHANGES MADE TO THIS FILE WILL BE LOST ON THE NEXT UPDATE UNLESS YOU REMOVE THIS HEADER   #
#                                                                                                 #
#   ENSURE THAT YOU DO NOT REMOVE THE REQUIRED LINE:                                              #
#                                                                                                 #
#   include "/etc/lighttpd/conf-enabled/*.conf"                                                   #
#                                                                                                 #
###################################################################################################

server.modules = (
    "mod_access",
    "mod_auth",
    "mod_expire",
    "mod_redirect",
    "mod_setenv",
    "mod_rewrite"
)

server.document-root        = "/var/www/html"
server.upload-dirs          = ( "/var/cache/lighttpd/uploads" )
server.errorlog             = "/var/log/lighttpd/error-pihole.log"
server.pid-file             = "/run/lighttpd.pid"
server.username             = "www-data"
server.groupname            = "www-data"
# For lighttpd version 1.4.46 or above, the port can be overwritten in `/etc/lighttpd/external.conf` using the := operator
# e.g. server.port := 8000
server.port                 = 80

# Allow streaming response
# reference: https://redmine.lighttpd.net/projects/lighttpd/wiki/Server_stream-response-bodyDetails
server.stream-response-body = 1
#ssl.read-ahead              = "disable"

index-file.names            = ( "index.php", "index.html", "index.lighttpd.html" )
url.access-deny             = ( "~", ".inc", ".md", ".yml", ".ini" )
static-file.exclude-extensions = ( ".php", ".pl", ".fcgi" )

mimetype.assign = (
    ".ico"   => "image/x-icon",
    ".jpeg"  => "image/jpeg",
    ".jpg"   => "image/jpeg",
    ".png"   => "image/png",
    ".svg"   => "image/svg+xml",
    ".css"   => "text/css; charset=utf-8",
    ".html"  => "text/html; charset=utf-8",
    ".js"    => "text/javascript; charset=utf-8",
    ".json"  => "application/json; charset=utf-8",
    ".map"   => "application/json; charset=utf-8",
    ".txt"   => "text/plain; charset=utf-8",
    ".eot"   => "application/vnd.ms-fontobject",
    ".otf"   => "font/otf",
    ".ttc"   => "font/collection",
    ".ttf"   => "font/ttf",
    ".woff"  => "font/woff",
    ".woff2" => "font/woff2"
)

# Add user chosen options held in (optional) external file
include "external*.conf"

# default listening port for IPv6 falls back to the IPv4 port
include_shell "/usr/share/lighttpd/use-ipv6.pl " + server.port
include "/etc/lighttpd/conf-enabled/*.conf"

Immediately after using that lighttpd.conf and restarting lighttpd service, the web interface on the physical eth0 IP (192.168.0.55) did not load anymore, while the virtual eth0:0 IP (192.168.0.99) still does - exactly the same behaviour as on the old host ==>

Really strange. That default lighttpd.conf looks very different, also misses the header section. At the same time the timestamp of the lighttpd.conf from the old host is exactly the date/time when Pi-Hole got updated to latest v5 release last year. What specific change in that file made lighttpd finally respect the /etc/lighttpd/external.conf content is kind of a miracle to me.