Thanks for the very quick reply. (just knocked off for a cuppa). There are not 2 active servers running there is only Pi-hole.
But when NetworkManager creates a new connection (or someone tries to connect to the Hotspot), NetworkManager doesn't know about the PiHole so it attempts to start its own internal DHCP server but it cannot complete because port 53 is already in use.
Network manager goes through its config files looking for a DHCP server (there can be multiple) and when it doesn't find one it tries DHCPClient, it that is not specified it will then use its internal lightweight DHCP server which it will attempt to start. I guess it is setup like this to cover all situations. If there is no servers, as is the case in the default configuration it can still function.
The problem is that NetworkManager does not know about Pi-hole.
I think there is a misunderstanding: NetMan doesn't run on the client but on the Pi-hole system. The problem is not that it assigns a wrong /etc/resolv.conf entry but that it starts a conflicting DNS server on the Pi-hole system when using its WiFi hotspot feature.
Do not touch /etc/resolv.conf on the Pi-hole server, especially do not change it to use Pi-hole itself if there is no specific reason, like your using a desktop and browser on the same machine so that you want to have ads blocked on it as will. Else keep a valid upstream DNS like Quad9, Cloudflare or Google DNS.
That makes it much clearer to me.
NM attempts to start "dnsmasq" but fails, so i assume the Hotspot does not bind to anything.
The trouble i see is that dns-FTL is using dnsmasq as its template and uses the same directories/configuration files. Makes sense I guess from a developers point of view, who wants to reinvent the wheel.
I am not sure where you mean for me to set the dns none.
If I set a dns to point to localhost, will it be resolved by pi-hole?
Which actually brings up another unrelated question. The raspberry pi has a static IP address. Can I point its name server to either its own IP or to localhost?
Thanks for that Michalng. That is exactly what is happening.
I checked the dependencies and NetMan does not depend on dnsmasq. So to prevent conflicts in the first place it probably makes sense to simply purge it:
apt purge dnsmasq
Probably NetMan then gracefully skips it or it throws and error when trying to start the WiFi hotspot (or it reinstalls dnsmasq automatically). If so, then maybe setting DNS to none solves it, but I'm not sure whether this really toggles the dnsmasq start or only prevents NetMan from applying an /etc/resolv.conf entry, which is irrelevant here.
When you apply either 127.0.0.1 or localhost or its own IP to /etc/resolv.conf, it should in all cases be then resolved via Pi-hole. Usually you do not use hostnames for DNS nameserver entries, but in this case localhost is resolved via /etc/hosts to 127.0.0.1 if I'm not mistaken. However, use 127.0.0.1 to have least overhead and be independant of the LAN IP when you really need the Pi-hole server to resolve its own DNS requests.
Well I think I should be taken out and publicly flogged and have my clothes burned.
When set up a Hotspot the interface itself needs an IP(the actual hardware) address and that is what is failing, not the subsequent connections.
I can set the Hotspot to have a static IP and then I can connect to it, even though I cannot get to the internet, that i think is another problem, probably with the name servers.
The really strange thing is that the Pihole DHCP shows the the Wlan0 interface (Hotspot) has now been given a different IP to what I manually set. There is something I am missing.
I don't think i should purge the dnsmasq. It was installed by the pi-hole installer and I think it was to retain configuration directories in case pi-hole was installed over the top of dnsmasq.
No, Pi-hole comes with its very own pihole-FTL - it does not install dnsmasq.
That was most likely installed either by NetworkManager itself or by your OS.
Note that NetworkManager also would manage its dnsmasq configuration files in a separate location, at /etc/NetworkManager/dnsmasq.d/ or /etc/NetworkManager/dnsmasq-shared.d/.
It is NetworkManager that claims to manage your network settings, including DNS.
The critical information to consider has been provided in the first reply:
The remaining problem is how to configure NetworkManager to avoid claiming port 53 and to configure it to make use of Pi-hole for DNS for those connections where you want it.
Thanks for the reply
I am sure that there is no dnsmasq installed before pihole is installed.
Without pihole the router is providing the dhcp & dns services and all the network interfaces work without alteration, including the Hotspot.
Here is part of the install log for pihole
[i] Installing scripts from /etc/.pihole...
e[K [✓] Installing scripts from /etc/.pihole
This sums up the problem succinctly. Surely it is the responsibility of new software to play nice with the existing operating system.
OK a Hotspot alongside pihole on a RaspberryPi could be considered an edge case but it still should behave.
Then there should still be none now.
However, you mentioned your system had some conflicts over port 53 from the very start:
If it's not NetworkManager, then it's whatever software is associated with enabling your WiFi Hotspot.
I suspect that dnsmasq as a package would have been present all along, but the dnsmasq process would have been launched on demand only.
The absence of /etc/dnsmasq.conf would just suggest the absence of a standard dnsmasq configuration at the time of Pi-hole's installation.
It would be in agreement with NetworkManager handling its own dnsmasq instance via /etc/NetworkManager/dnsmasq.d/ as mentioned before, or with any other DNS resolver claiming port 53.
There is no configuration files for dnsmasq in the etc/NetworkManager/dnsmasq.d/ directory without pihole installed.
But that may mean that the "lightweight dns/dhcp " referred to in the documentation of network manager is in fact some version of dnsmasq running as defaults.
The "standard" or "default" way to invoke Hotspot is to use NetworkManager which invokes this cut down version of dnsmasq. This uses port 53.
This is part of the ubuntu release.
Pi-hole is a release for this operating system version so it needs to work for this system.
I am hoping that someone (or me?) can come up with a config change that can be eventually implemented in the pihole installer to fix it.
Changing the operating system (other than the config) just so a program can work seems silly to me.
I have a couple of things to try as yet but they are long winded (like me!) and will take time because of my setup.
OK I am pretty sure I have nailed what is actually happening. NetworkManager in creating a Hotspot uses its internal DHCP server.
pihole-FTL uses the dnsmasq code as its base. BUT it also uses the dnsmasq configuration files and directories for its configuration. This allows to use updates to the dnsmasq to seamlessly upgade/update pihole-FTL code base.
In setting up a hotspot, it does a dhcp to get an ip address for the interface itself. That uses the internal DHCP server (which should be fine....
It then uses dnsmasq (an "internal copy") to set the various DNS/gateway/routes etc for this interface.
BUT pihole actually uses the dnsmasq configuration files and settings (since its based on the dnsmasq code) and there is no way to reconfigure the internal dnsmasq without affecting pihole, and visa versa, since they effectively share the configuration.
It appears NetworkManager will not recognize pihole-FTL as either a DHCP or DNS server.
That means when it tries to start its internal DHCP/DNS (dnsmasq) it cannot because port 53 is already in use.
I cannot change port 53 on the dnsmasq because it changes on pihole. (they share the same config files)
I will leave it for now but the next step is to try unbound & dhcpd and set pihole up as the authoritive server for unbound and another unbound as the authoritative server for pihole. (very messy) But I will have to leave it for now and move on to other things.
Thankyou for your reply.
So is NetworkManager compiled with a change to the directory structure of its DNSMASQ?
According to the man dnsmasq
At startup, dnsmasq reads /etc/dnsmasq.conf, if it exists. (On FreeBSD, the file is
/usr/local/etc/dnsmasq.conf ) (but see the -C and -7 options.)'*
That can be changed with a command line option when it is invoked, but that will be part of NetworkManager and does not seem available/changeable in its configuration. That file points to dnsmasq.d where the various pihole configurations are present.
Is there a way to find out what the NetworkManager dnsmasq configuration directory/file is set to or is it just a "suck it and see?
Such port conflicts is not something Pi-hole can or should ever try to handle, the logic may be what user A wants but may break the system for user B. It's like you have Nginx installed and then wonder why installing Apache fails as it tries to bind to the same port by default. No such package or installer will every try to disable Nginx or change its port, but the install fails to inform you that there is a port/service conflict that only you can solve as only you know what your intention actually is.
In your case you obviously want Pi-hole as DNS resolver for the network so you need to configure NetworkManager to not start its own DNS server along with the WiFi hotspot. A DNS server is no integral component of a WiFi hotspot, strictly not even a DHCP server is. So while it may be convenient that NetworkManager spins up those when it does not find any, there should be a setting to prevent it from doing so.
Something that could be done is sending a request to NetworkManager maintainers to detect Pi-hole FTL as local DNS server, as it does with dnsmasq. Pi-hole should be known and used enough to hope for an approval.
Thanks for your reply.
You have previously made this same point. My suggestion is still the same.
This is a pihole installer version specifically built for the standard operating of Ubuntu 21.04.
The standard installed network configurator/operator for this version is NetworkManager.
This installer breaks an aspect of the standard operating system operation.
By your logic I should be able to produce a program for any operating system and expect that the enduser should substantially alter that operating system to make it work.
Pihole acts as a DNS (& DHCP) interceptor. It is a caching DNS server with filters. It provides DHCP services as well so that it can propagate it's address rather than have to set every device manually. That is convenient where Routers/modems are unable to have their DHCP altered to do it instead.
If it did not use the dnsmasq directories for it's own configuration then the problem would not occur. I understand the pihole reasoning behind this.
Perhaps there should be "pihole_only_config_section=" or a "pihole_ignore_section=" option to allow the parameters in the dnsmasq config directories to serve both. Even something as simple as an "extra_config_file=xxxxxx-config-pihole" (without the .conf so dnsmasq ignores it)
But there isn't.
I am seeking a configuration work around, preferably one that will work for this ( and future) version of Ubuntu.
You mean because Pi-hole's and NetMan's dnsmasq are using the same config files? Did you actually verify that e.g. by adding some invalid setting to /etc/dnsmasq.conf and see whether this breaks NetMan start or leads to a related warning or such? As it would be logic if it used "only" its very own at /etc/NetworkManager/dnsmasq.d. As a new config path was added, it would have been no issue to remove the other one(s) to avoid conflicts with other dnsmasq instances and keep all NetMan configs below a single directory.
But regardless, you do not want it to start with different configs, you don't want to have it started at all, and that must be configured on NetMan core configs (not the dnsmasq ones). It seems /etc/NetworkManager/NetworkManager.conf contains a dns=dnsmasq setting which you can comment disable it, or change it to dns=none.
No it doesn't break anything, it might if it tried to resolve this somehow by manipulating NetMan or NetMan's dnsmasq instance. Do apt install unbound dnsamsq and you will run into an error as well, even that both are native Ubuntu packages. There is and will be always a minimum of self-responsibility on every (server) system administrator to configure multiple components a way that they work together as wanted. However, looks like we won't agree on that, which is fine .
What I agree with, is that it may be better if Pi-hole's dnsmasq didn't use the default dnsmasq config directory but (only) an own one e.g. in /etc/pihole/dnsmasq.d, same as NetMan seems to do it. I think this has historical reasons as Pi-hole v3 did not include dnsmasq yet but installed it separately. Probably it is still kept as a design choice so that admins can find and configure Pi-hole's dnsmasq just like the dedicated one. Or it is kept to keep maintenance efforts lower. But that way Pi-hole FTL cannot run besides a dedicated dnsmasq, for whatever reason someone might want to do that .
If you recently acquired a BMW X2 (Pi-hole) to accompany your Ford Falcon ute (dnsmasq), and that BMW doesn't fit in your single-port garage, you'd have a hard time to get BMW to build you a garage extension, or Ford, or your garage builder (NetworkManager - unless that builder had asserted that both any Ford and any Beamer would fit your garage).
DNS/port 53 is such a single-port garage.
Pi-hole does not make any claims to avoid or even know how to avoid arbitrary port conflicts - on the contrary. Pi-hole's documentation clearly states that it requires exclusive access to port 53, and that any other resolvers have to be disabled
You have been advised of that port conflict in the very first reply.
You've finally acknowledged there is a port conflict in your post 36.
We are a friendly folks here, and so yubiuser has also offered some advice for allowing coexistence of your AP's dnsmasq with Pi-hole, also in the very first reply.
That is a question for NetworkManager support.
As far as directory locations as quoted by me are concerned: Those have been taken from the official NetworkManager documentation. yubiuser has already provided links to that documentation for you.
Just to repeat this for the fourth time:
NetworkManager and Pi-hole do not use the same file locations for configuration.
In practice they do.
dnsmasq looks at dnsmasq.conf in /etc/. So does pihole.
This is the default behavior for dnsmasq. It can be changed with a command line option but if no command line option is given then the default is used. (Based on your previous answer it is why I asked if you knew it was done on a command line within NetworkManager )
Like all the parameters in dnsmasq they have inbuilt defaults so it is not actually necessary to have a config file.
There is a certainly a /networkmanager/ directory to put dnsmasq configs in but it must be pointed to by the dnsmasq.conf in /etc.
This file contains a pointer to the /etc/dnsmasq/directory and files installed by pihole.
Pihole operates this way (according to its documentation) so that that upgrades to the dnsmasq codebase (which is the basis for pihole-FTL) can flow easily to pihole-FTL with little or no engineering changes.
If I put a port=0 in the pihole configuration file it kills pihole. But it also kills dnsmasq.
You are absolutely right. The normal or expected way to disable it is to add a config file in one of its standard default locations with a port=0.
But its standard locations are already populated with pihole configurations and the standard pointer file (/etc/dnsmasq.conf) is already used to point to those files.
You are kidding right? Why would you say something like that?
I appreciate all advice but also if that advise doesn't work or has problems I will point it out and try to give as much explanation as possible, because down the track someone else just like me will go through the same process.
He pointed to the documentation which I used to find out the circumstances of where port 53 was conflicting. I took that advice. I had to learn what was going on and why following that advice it killed pihole as well.
There is no "standard" or "standalone" installation of dnsmasq. My best guess is that in the development of NetworkManager they did the same as what the pihole people did, they just used the dnsmasq code base in a default configuration and compiled NetworkManager with it, putting the executable under a NetworkManager location.
I have been trying to find a way to prevent it and hoping that there might be a trick someone knows to change either NetworkManager or pihole-FTL configuration. Even though the NetworkManager documentation suggests I should be able to specify the DHCP server' it is limited to dhclient, unbound or internal but that does not effect its DNS source.
I can make it stop DNS service port=0 in the dnsmasq configuration, but that is not in the Networkmanager config.
I may have missed something in my testing (I hope not) and tend to be long winded so more experienced people can point it out.
There is indeed a NetworkManager config entry dns = . BUT it (according to the man pages) it is the dns processing mode. It has to do with how the dns is resolved not whether it runs at all. I have tried to set it to none but dnsmasq still starts (& fails).
Most settings in NetworkManager have sensible defaults that are used if not changed in a config file.
From what I can see most programes are moving that way, so config files become optional.
There is always the possibility I missed a vital step though because NetworkManager does not reload all its config files on a restart only at boot restart.
There is a .config section of NetworkManager config, but the documentation is a little sparse. It may mean setting a conditional setting like port = 0 for the particular version of dnsmasq I am a little unclear of its use.
I will ask on one of the NetworkManager forums.
They don't. NetworkManager even hardcodes the conf-file option to /dev/null by default, in order avoid loading any external dnsmasq configuration. Actual configuration is done by individual conf-file options when NM is launching an NM administered dnsmasq instance.
You should be able to verify that, e.g. by running ps auxgww | grep dnsmasq.
The trick is to find out:
and if that would be dnsmasq:
As you are using NetworkManager, those dnsmasq config files should be located at either
You could consider trying out yubiuser's advice, or inquire with NetworkManager's support for the proper way to prevent its Hotspot from claiming port 53 and/or avoiding conflicts with Pi-hole altogether.