Great, will get it setup and see how it goes.
Thanks
Dang, can't post more than 3 replies to same topic.
Ok, new stack up and running with latest versions. I've completed your steps for Logstash and PiHole. Now need to do the Kibana steps:
I've tried importing the json files but keep getting the error below:
File could not be read
[illegal_argument_exception] Could not find a timestamp in the sample provided
I'm trying to get elk-hole working on version 7.3. I imported all of the visualizations and dashboards but I'm getting "Could not locate that index-pattern-field ".
I pasted the json into the dev tools and clicked the play button. It resoponded with:
{
"acknowledged" : true
}
But when I look for the index pattern it's not there. Did I miss something?
Thanks!
(Editing to add that my ELK instance is receiving the logs - I can filter them in the discover app no problem. But they're all going into filebeats-* and not the index I think I'm supposed to be creating from dev tools that doesn't appear to be created. I'm new to ELK so maybe I'm missing something?)
@ezrae are there any other configuration files except the elk-hole.conf in /etc/logstash/conf.d ?
If lot it could also be possible that the filebeat „agent“ on the pihole is forcing the index name.
by the way its not index itself getting created via the dev tools you mentioned. Its only the index template(!), the index gets created during startup of the stack. Do the logs have the correct tag (pihole)?
Did the index pattern in kibana successfully create? ("management -> index patterns") and search for e.g. "logstash-dns"
Did you try to create the index via kibana manually? Does it pop up there?
Please also note, new Indexes are only created while starting the whole stack and not during runtime
@tep1997 Sorry didn’t get a notification
Is the index pattern for elk-hole already created? you can find it via "management -> index patterns" and search for e.g. “logstash-dns”
Could you add an issue at the GitHub repo please?
Thanks! I have two other files in /etc/logstash/conf.d. I don't have elk-hole.conf, but I do have 20-dns-syslog.conf. Should I have renamed it?
I'm super new to ELK - where/how do I check the logs for the correct tag?
I have two total index patterns under management -> index patterns:
filebeat-*
winbeat-*
I'm not sure how to create an index manually. If I click on create index pattern from the management -> index patterns screen I get an error:
"The index pattern you've entered doesn't match any indices. "
I'm not sure the best way to start/restart the whole stack so when I want to do more than just restart kibana or logstash I'm just rebooting. Nothing appears after a reboot either though..
Thanks for your help.
My guess is that your other 2 files in this dir catch the pihole logs because they are most likely not very strict in terms of what type of logs they process.
You can try to rename your files to say "30-first.conf" and "40-second.conf" as logstash is processing the files by looking at the numbers in the filenames at the beginning 
20-dns-syslog.conf is absolutely ok, I was just to lazy to type the complete name via mobile.
logstash-syslog-dns* is the index pattern I use in kibana. Try create it please. If the index already exists it will show up (but it has to be this exact phrase)
You cant really create the actual index manually, it gets created via the very last line in the output section of 20-dns-syslog-conf
output {
if "pihole" in [tags]{
elasticsearch {
hosts => [""<ELASTICSEARCHHOST:PORT>""]
# manage_template => false
------------> index => "logstash-syslog-dns-%{+YYYY.MM}"
}
}
}
You said you can see the logs in another index via discover. There is "tag" field in every log entry. What tags do your entries have?
It is sufficient to restart the service (here most likely only logstash) to restart the process. What distribution do you use? try "systemctl restart logstash" or "/etc/init.d/logstash restart" with sudo or as root
Oh! I found my problem! Thanks for helping point me in the right direction. I had:
hosts => [""<localhost:9200>""]
in my outputs instead of:
hosts => ["localhost:9200"]
So logstash was crashing at startup. Once I fixed it, I was able to create the index and re-import the .json for the visualizations and it all works fine now.
Thank you for your help!!!
Got past the import problem and I'm so close I think, but after completing all the steps, when I click on the pihole dashboard in Kibana, I get this:
@tep1997 try this please
elk-hole/elk-hole - vis_enhanced_fix.json at master · nin9s/elk-hole · GitHub
retry please, typo ...
the error sources from the missing index pattern "logstash-syslog-dns*" so we have to figure out why it wasnt created. Can you paste the content of the output section of 20-dns-syslog.conf please
output {
if "pihole" in [tags]{
elasticsearch {
hosts => ["192.168.1.66:9200"]
index => "logstash-syslog-dns-%{+YYYY.MM}"
}
}
}
192.168.1.66 is the ELK stack box.
output {
if "pihole" in [tags]{
elasticsearch {
hosts => ["192.168.1.66:9200"]
# manage_template => false
index => "logstash-syslog-dns-%{+YYYY.MM}"
}
}
}
sent you pm
Hi, strange issue wanted to get your input on. I had an issue on my ELK stack VM and restored it from a backup from a couple days before the issue occurred. Now, when I launch the Pihole dashboard, all I get is a blank page.If I use the Visualize tab, I can see the data, but nothing is showing up on the dashboard.
hm strange. did you try different browser and/or clear the cache?
Good catch! That was the issue. I was so focused on the no visualizations issue that I never considered that. Thanks!
Hello, I have been trying to get this setup using the instructions but for some reason after I install filebeat I am getting this error message when I try to start up the service:
Failed to start filebeat.service: Unit filebeat.service not found.
Any help is much appreciated!
