Sneaky Query... how did this happen?


#20

I’ve found another chunk, but looking at the pihole.log seems normal until you see the corresponding query log that doesn’t match at all. Here is another instance (I’m not sure how to grep a range of time and what good it would do without the query log):

and here is the rest of the query log for 00:54:08

2017-08-11 00:54:08 IPv4 otf.msn.com 10.139.12.100 Pi-holed Whitelist
2017-08-11 00:54:08 IPv4 otf.msn.com 10.139.12.100 Pi-holed Whitelist
2017-08-11 00:54:08 IPv4 www.myhomemsn.com 10.139.12.100 OK (forwarded) Blacklist
2017-08-11 00:54:08 IPv4 rad.msn.com 10.139.12.100 Pi-holed Whitelist
2017-08-11 00:54:08 IPv4 rad.msn.com 10.139.12.100 Pi-holed Whitelist
2017-08-11 00:54:08 IPv4 support.microsoft.com 10.139.12.100 OK (forwarded) Blacklist
2017-08-11 00:54:08 IPv4 outlook.com 10.139.12.100 OK (forwarded) Blacklist
2017-08-11 00:54:08 IPv4 e3843.g.akamaiedge.net 10.139.12.100 OK (cached) Blacklist
2017-08-11 00:54:08 IPv4 clk.tradedoubler.com 10.139.12.100 Pi-holed Whitelist
2017-08-11 00:54:08 IPv4 clk.tradedoubler.com 10.139.12.100 Pi-holed Whitelist
2017-08-11 00:54:08 IPv4 www.skype.com 10.139.12.100 OK (forwarded) Blacklist
2017-08-11 00:54:08 IPv4 redirect.viglink.com 10.139.12.100 Pi-holed Whitelist
2017-08-11 00:54:08 IPv4 redirect.viglink.com 10.139.12.100 Pi-holed Whitelist
2017-08-11 00:54:08 IPv4 outlook.com 10.139.12.100 OK (cached) Blacklist
2017-08-11 00:54:08 IPv4 onedrive.live.com 10.139.12.100 OK (forwarded) Blacklist
2017-08-11 00:54:08 IPv4 livecmseastus.cloudapp.net 10.139.12.100 OK (cached) Blacklist
2017-08-11 00:54:08 IPv4 www.facebook.com 10.139.12.100 OK (forwarded) Blacklist
2017-08-11 00:54:08 IPv4 a-0014.a-msedge.net 10.139.12.100 OK (cached) Blacklist
2017-08-11 00:54:08 IPv4 www.onenote.com 10.139.12.100 OK (forwarded) Blacklist
2017-08-11 00:54:08 IPv4 star-mini.c10r.facebook.com 10.139.12.100 OK (cached) Blacklist
2017-08-11 00:54:08 IPv4 go.microsoft.com 10.139.12.100 OK (forwarded) Blacklist
2017-08-11 00:54:08 IPv4 e11290.dspg.akamaiedge.net 10.139.12.100 OK (cached) Blacklist
2017-08-11 00:54:08 IPv4 www.fool.com 10.139.12.100 OK (forwarded) Blacklist
2017-08-11 00:54:08 IPv4 www.autotrader.com 10.139.12.100 OK (forwarded) Blacklist
2017-08-11 00:54:08 IPv4 static-entertainment-eus-s-msn-com.akamaized.net 10.139.12.100 OK (forwarded)
2017-08-11 00:54:08 IPv4 static-entertainment-eus-s-msn-com.akamaized.net 10.139.12.100 OK (forwarded)
2017-08-11 00:54:08 IPv4 prod-na.reverseproxy-onenote.com.akadns.net 10.139.12.100 OK (cached) Blacklist
2017-08-11 00:54:08 IPv4 zone.msn.com 10.139.12.100 OK (forwarded) Blacklist
2017-08-11 00:54:08 IPv4 e8175.a.akamaiedge.net 10.139.12.100 OK (cached) Blacklist
2017-08-11 00:54:08 IPv4 www.match.com 10.139.12.100 OK (forwarded) Blacklist
2017-08-11 00:54:08 IPv4 evcert.motleyfool.map.fastly.net 10.139.12.100 OK (cached) Blacklist
2017-08-11 00:54:08 IPv4 flights.msn.com 10.139.12.100 OK (forwarded) Blacklist

edit: the queries in the query log are extremely similar to the previous instance. I’ve ran MalwareBytes, CCCleaner, and our AV scanner on 10.139.12.100 with nothing out of the ordinary popping up.


#21

Try uploading your pihole.log, pihole.log.1, and the output of http://pi.hole/admin/api.php?getAllQueries to Tricorder:


#22

/var/log/pihole.log -> 2ulh347mu9
/var/log/pihole.log.1 -> 708i49i1k1

what command should I use for http://pi.hole/admin/api.php?getAllQueries before piping it?

edit: I used curl and the entire output was: []
I double checked with my browser:


#23
[]

is the response if there is (a) no data at all (unlikely) or (b) if you are not authorized to get this data. You have to be logged into your Pi-hole dashboard and you will get the data only in the very same browser. Even on the same computer, curl is not authorized to get the data and hence will still see [].


#24

You are correct. I logged into the PiHole dashboard, opened another tab with the api.php URL and it did indeed pull the information. So, how can I pass credentials for the api.php with curl on the PiHole?

I tried curl -u pi:[password] but that still got me []. Isn’t that the username that is passed on the dashboard?


#25

Use something like

curl http://pi.hole/admin/api.php?getAllQueries\&auth=183c1b634da0078fcf5b0af84bdcbb3e817708c3f22b329be84165f4bad1ae58

as described here

You can get the auth key from your /etc/pihole/setupVars.conf file (see WEBPASSWORD=).


#26

Another success brings another failure:

not sure if you got that tricorder upload or not, I’m not familiar with the curl error that occurred.


#27

Yes, it uploaded partially. We apply a certain (not too high) limit on the amount of data users can put to our Tricorder server to protect ourselves from a few possible attack vectors. Hence, your data was cut off at some point and this is how curl reacted to this.

I’ll now look at your data. Can you please also send a screenshot of your Query Log (possibly the last or second to last page you can see)?


#28

I hope this is what you’re looking for. I went to query log, enabled Show All, then went to the last page available.

Let me know if you need anything else, I’m standing by.


#29

Thanks. What I can see here matches your uploaded API data perfectly. Any chance that you see somewhere the problem you have?


#30

If the api.php?getAllQueries is showing exactly what the Query Log shows, then I have an example above (00:54:08)


#31

Okay, sorry, I should have been clearer on this. The uploaded data is, unfortunately, not sufficient for seeing data from today.

One other odd thing is, though, that your screenshot shows a domain e3843.g.akamaiedge.net which does not seem to exist in either of your uploaded files (pihole.log / pihole.log.1). It is possible that it was cut off during upload.

Could you please run

grep 'e3843' /var/log/pihole.log
grep 'e3843' /var/log/pihole.log.1

#32


#33

Okay, the problem so far was just that you grepped for the wrong time in the log. The time zone of your Pi-hole device is not correctly set up (or at least it is not the same as your local time zone). Hence, it logged the queries at 05:54:08 of its time which the Pi-hole dashboard (correctly) translated into 00:54:08 in your local time.

We can now delete the last 30 posts in the thread and return to the initial problem :slight_smile:

Has that ever happend since then? Do you have, by any chance, the log data of this day (August 10) still available?
(it may be /var/log/pihole.log.2.gz or some other compressed archive file by now)


#34

Wow, that made me feel pretty stupid.

Back to the matter at hand… I tried to sudo cat /var/log/pihole.log.2.gz | grep “11:40” and got no results. I tried without grep: sudo cat /var/log/pihole.log.2.gz and the output was absolutely haywire. The bash went psycho and then ended up printing “PuTTyPuTTyPuTTy” over and over again in the prompt, so I closed out. Can you help me out with a command to handle this .gz file?


#35

No worries :slight_smile:

I really enjoyed a laugh when reading the rest of your post. The file has been compressed, so we cannot expect readable output. Try using zgrep for compressed files.


#36

Also see this site for more info on decompressing .gz files:
https://www.cyberciti.biz/faq/howto-compress-expand-gz-files/


#37

I’m learning so much today.

So to be clear, this is relevant to the very first post on the thread. Remember, the first picture is the Query Log with Show All, searching for the phrase “ad” and sorted by URL. This screenshot does seem to show forwarding the query, though.


#38

Are there any other ad domains being forwarded around that time?


#39

Repeat the zgrep for adserver.pandora.com and hope for the best (that there are not too many results).