pihole-FTL.log full of "Notice: Increasing wildcards struct size"

Help
1

I noticed that the FTL log is littered with these entries. I have 28000 of them since the last 5 days and it's like >2 MB now

[2017-05-05 01:58:47.916] Notice: Increasing wildcards struct size from 15 to 16 (744.94 KB)

I did add pgl.yoyo list in the wildcard section, which is about 2K entries. So once in a while it outputs "Increasing wildcards struct size" 2K times. Anyway to silence these verbose messages?

Also when I tried to upload a debug log, it said "error uploading your debug log".

2

Why did you add a list to the wildcard section? That is for manual wildcards only. Try adding the list to the ad lists via the settings page instead. FTL is not reserving enough space before it has to reallocate, which is why it is showing that message so many times. @DL6ER might want to comment on this.

1 Like
3

Why did I add a list to wildcard? I think the correct question is "why not?" :smile_cat:
Also Blocking with ad server and tracking server hostnames talks about blocking subdomains using this list.

I only add top level ad server domains. So, please don't submit hostnames like "ad.se.doubleclick.net", as "doubleclick.net" is already listed

4

I concur. Since I didn't expect users to have a humongous number of wildcard lists (I know ... how dare I ...) I have not thought about that. On the other hand, it is required to allocate the memory for each and every entry separately since we only allocate what we need (we know that only when we know the length of the domain name to be wildcard blocked).

Also, I don't want to stop logging this altogether because we would have no chance to identify OOM issues as the cuase for a crash otherwise.

5

Fixed it. Renamed /etc/dnsmasq.d/03-pihole-wildcard.conf

6

That still has to be clarified. Due to your fix, FTL won't analyze the file anymore. But that also means that the detection if a domain has been blocked due to a wildcard entry won't work any longer.

7

Yes, the wildcard hits on the custom conf file shows as cached.
Does FTL process on demand or pre-process them in memory?

8

Everything is computed on the fly when analyzing the logs and then stored in memory for ultra-fast retrieval. Even huge amounts of data can be requested withing few milliseconds and currently our PHP API is the only remaining bottleneck (although much less noticeable than with Pi-hole v2.x).