Yes, please do this. Note that IPv6 is actually faster in practice and you limit yourself these days if you decide to abandon IPv6 by intention. Instead, go into your Router's settings and enable ULA. A detailed HowTo is available here, scrol down to see how to setup the Speedport 724V:
Note that even if devices have no IPv6 upstream connectivity, they can still send AAAA queries to the DNS server, because nothing stops them for doing this over an IPv4 connection. Remember that the IPv4/6 protocol is only the protocol which transports your data over the network it has no direct influence on the content of the packages it transports. Your devices may stop doing AAAA queries, since they think that they are meaningless without an upstream IPv6 connection, but that is a very hack workaround.