Super helpful CallMeCurious - this is what I was hoping for to find everything so I can make sure I miss something
In case someone else finds this thread in my instance the command was slightly different
sudo grep -oP "Unpacking \K[^: ]+" /var/log/syslog | sort -u | comm -13 /dev/stdin <(apt-mark showmanual | sort) >installed.txt
The log was not in a subdirectory in case the formatting does not work in my post