Might I suggest you download all your lists locally then change them to load locally eg off the PiHole web server. (You may already do that.)
I run ~4 000 000 on a RaspberryPi 3 B and there are large delays when Whitelisting Blacklisting and updating.
Memory usage: 37.3 % of 1 G
Here is the post on what I have done you are welcome to try them. The reason the second list is split because of the manual upload size to Github over my connection has about a 8-9 mb's per file.
https://discourse.pi-hole.net/t/i-concatenated-every-blocklist-i-could-find/5184
