Check for <newline> before adding to config files

After the update to 5.22/5.22/533 and also when adding CNAME I noticed that the added text was not starting on a new line but was added behind the last already present line.

To avoid this happening any write to a config file by Pi-hole could use an sed command that checks if there is a newline (\n) present and if not it will add it. Then Pi-hole can add line and it will be always on a new line despite if a user has changed the config file and forgot to add a newline at the end of the file.

As far as I could test it it will not add an newline if the file is empty.

This sed command adds \n at the end of the file only if it doesn’t already end with a newline. So if you run it twice, it will not add another newline:

sed -i -e '$a\' filename

Can you provide an example or two? I am not seeing this behavior in a current build.

This has been always the case and not only the case in the the latest version. I thought is was tackled because I saw some chatter about that on Pi-hole's Github about this more than a year ago.

Example: edit /etc/dnsmasq.d/05-pihole-custom-cname.conf and enter for example cname=duckduckgo.com,www.duckduckgo.com,safe.duckduckgo.com , then do not put an enter after the line and safe the file.

Add now in Pi-hole web-interface a CNAME and it will be added directly behind the already present line instead of on a new line. This you can see in the web-interface and the file.

This can happens to all config files that are changed/appended by Pi-hole and it happened also during the upgrade to 5.22/5.22/5.33 I did.

@msatter Do you want to make a PR for this when you already tested it and verified at which place in what file it works reliably?

Thanks and someone else have to make a PR if that is something you have to get a account with with Github (Microsoft).

I tested it with text files and those are the ones having the problems not having a newline on the end which Pi-hole expect when adding to them.

Okay, no problem, I wanted to preserve authorship for you, but I can do it as well. The question: Do we really need this only for CNAME editing? I just want to trigger some brainstorming here to not having to do this again in two weeks with the next list where newlines are missing.

I this problem in the past and this time also after the update to current Master. Then Pi-hole just did not start.

Now it started with the message in the webinterface. It is all text files users can change and the Pi-hole add lines to. Files in directories /etc/pihole and /etc/dnsmasq.d .

Could I ask you to test

pihole checkout core tweak/ensure_newline

if you think I got them all?

You'll note that I used a bashism here instead of the sed because the sed has the drawback of always rewriting the file entirely. This does not only change the timestamp (we change something in here anyway), however, it just seems like an unnecessary write operation. The bash way I'm using right now is very fast as it directly jumps to the end of the file (instead of sed reading the entire file) and just checks the very last byte for \n

1 Like

I noticed also sed always changing the timestamp so that was no indicator as check.
Your way is better.

On update of Pi-hole you could just do a blanket (bombing) check of the files so that they are good to be changed or added to.

Good catch of disable and enable Pi-hole and I would have missed that.

I will test it in a few moments.

1 Like

I checked with creating a CNAME to a file with no newline at the end and it worked perfectly.

How about the Strickler error on install/update? Can you exclude these changes and run a separate check instead of that.

Many thanks for implementing this and I only want to have less problems with Pi-hole. The ones cropping up when installing or upgrading, are the ones that make users loose confidence very quickly.

The problem can only arise when we add something to a file where no newline was before. Hence, checking every time before writing seems better than checking once during an update.

I thought about both.

Example I change a config line and all works great. A few months later I do a update and then I have the problem of missing that newline at the end. Then I have to remember what I did a few months ago.

No, the only error can be when Pi-hole re-processes the config during an update. However, this is covered as newlines are always ensured every time before writing.

1 Like

Had this on trying to recover from a out of sequence dev. updates:


  [â] man pages installed and database updated
/etc/.pihole/advanced/Scripts/webpage.sh: line 152: ensure_newline: command not found
  [i] Testing if systemd-resolved is enabled

Maybe it is nothing.