Support writing to pihole.toml directly

I manage my Pi-Holes with ansible (i.e. configuration management) so that I can run more than one node. This works with v6, but because Pi-Hole writes to pihole.toml, ansible always detects it as "changed" which means it writes a new copy (which Pi-Hole promptly loads and overwrites).

It would be great if this use case could be supported, because it's made bootstrapping Pi-Hole a charm for me. If this is something you'd be interested in, then ideally I could define a setting that disables re-writing the config on startup.

A second question that then comes up is what to do with edits via the UI (or API) that bypass config mgmt. One way could to make that the user's problem (this is how it is today with v5) which works well enough (for example, I take backups in case I change something in the UI and then config mgmt overwrites it). Another way could be a setting that disables any edits via the UI, hiding the buttons and all. It could even be implicit in the first setting, meaning Pi-Hole won't write back to pihole.toml under any circumstance.

Let me know what you think about this idea. Cheers!

It only writes when it has to update the file. This can be because of changed setting or because it has to update outdated comments, remove deprecated and not removed options, ...

When you touch the file without modifying it, there will be no write attempt. Same if you replace one config by another which has no errors (while "error" means anything FTL needs/wants to change).

We already have something like this for the docker context: Any variable set through the environment pihole-FTL runs in will force the setting - it cannot be changed manually.

If I get this request right, you are now asking for a new option like misc.readOnlyConfig (name is totally open for discussion). I just wonder what special handling we need here. I guess it has to be limited to be set only via the file. Because if you'd set it either via the CLI or API, it couldn't update the file subsequently to actually store this change. Also, it wouldn't be possible to undo this later without having to access the file on the server/inside the container directly.

We just need to make such an option useful for more than one user and at the same time sufficiently easy to understand including all its consequences down the line. What do you say?

It only writes when it has to update the file. This can be because of changed setting or because it has to update outdated comments, remove deprecated and not removed options, ...
When you touch the file without modifying it, there will be no write attempt. Same if you replace one config by another which has no errors (while "error" means anything FTL needs/wants to change).

There are a few things Pi-Hole writes back that make it hard to produce the same file it would produce itself:

  • A line at the top with a date & time of when the config was last written
  • It removes trailing commas from a list element (which I could workaround, but TOML can handle it and it's easier to write a loop without the special "last" case)
  • Removes custom comments that I might add to remind myself in the future why a value is the way it is
  • Adds trailing comment "### CHANGED, default = X" which I'd have to conditionally write depending on settings

We already have something like this for the docker context: Any variable set through the environment pihole-FTL runs in will force the setting - it cannot be changed manually.

Interesting! I've followed this approach for my local installation of Gitea which uses the same approach. This unfortunately turns a well-structured, typed configuration file into a mess of environment variables, esp. when lists and booleans get involved.

If I get this request right, you are now asking for a new option like misc.readOnlyConfig (name is totally open for discussion). I just wonder what special handling we need here. I guess it has to be limited to be set only via the file. Because if you'd set it either via the CLI or API, it couldn't update the file subsequently to actually store this change. Also, it wouldn't be possible to undo this later without having to access the file on the server/inside the container directly.

We just need to make such an option useful for more than one user and at the same time sufficiently easy to understand including all its consequences down the line. What do you say?

Yeah I think there would a "minimal" use case where it doesn't rewrite itself unless the user made explicit changes via the web UI (and then it would maybe only update that value, not comments etc.?). Or a broader option pretty much like you describe. Unfortunately, based on what I've seen in the wider IT industry, nobody as figured out a super elegant solution here: If you allow users to write config via the UI then you create an issue with config mgmt / Infra as Code. Alternatively, you just disable changing the config via the UI and the only things that can be changed are those things that go in a database, not config file.

As for personal preference: I'd be happy if config changes via the UI were disabled, but I also realise that's a serious change since every bit would have to account for it and there's multiple places to change settings. I'd be quite happy if FTL & ansible stopped fighting each other over who gets to write the config file :sweat_smile:

There's a clear winner: FTL. It's the "owner" of the file :slight_smile:

This line is ignored to not needlessly rewrite just because FTL was updated (version changes) or restarted (the date changes).


I guess we will go with the misc.readOnly kind-of-option. Please try if my first attempt does what you expect when you use FTL from branch new/misc_readOnly instead of development-v6

This is how the option looks like on the web interface (all settings) but you obviously cannot enable it from here:

grafik

grafik

Wow you're quick! I tried it out and it works like a charm, if I run my ansible playbook it doesn't report any config changes b/c FTL didn't overwrite the config. So it definitely works!

There's a clear winner: FTL. It's the "owner" of the file :slight_smile:

Based on this, I'm not sure the "readOnly" option is the right way to go as the config could be considered an "implementation detail". If we are supposed to interact with the Pi-Hole in a different way (e.g. API/UI) then that's doable, too, esp. if all config options can be set via the API.

This adds a fair bit of complexity for things like ansible as they essentially need a custom module to handle talking to the API. I use this approach with things like Proxmox & Uptime Kuma, both of which have modules available and I don't write directly to config as they don't expect that.

My worry is that the readOnly option will add pain in the future, complexity in the code, edge cases that are rare and lead to unexpected behaviour and so on...

What do you think?

Even when I'll probably never use it myself (never say never), I don't think this will add to much of complexity down the line. The implementation is extremely simple:

  • false (default)
    • nothing changes
  • true:
    • writing to the config file is prevented
    • changing settings via API is forbidden
    • changing settings via CLI is forbidden

This indeed makes the configuration read-only, hence, the chosen name.

What about if forcing a setting via environment variable?

Environment variables always win - the corresponding values are not read from the config file at all. Note that the environment cannot be changed for a running process (unless you are doing some really nasty putenv stuff) so read-only is still ensured even under the vicinity of env vars.

I guess what I was thinking is where "SET BY ENV" is written to the toml file on FTL startup. Maybe this isn't a problem..

readOnly makes all things a bit special. Changes are not written to the config file even when they are forced through environment, i.e., they also don't need to be reset to default - when the env var goes away, we reload whatever was in the config file before