Nginx has nothing to do with resolving DNS. Does 192.168.3.10 point to a device with a webserver? What is the result of this when using the automatic DNS?
It sounds like it's doing exactly what you told it to do . Do you want mail.domain.com to go to 192.168.3.10 or 100.12.35.141? You should make sure that there's a webserver listening on 192.168.3.10.
Encouraging to know that. one point i missed-> I m trying out resolving mail.domain.com within the network(192.168.3) and its here that the browser could not resolves it although digging resolves to correct address. When i access mail.domain.com outside the network, i don't have any issues. The address resolves to correct IP.
You should make sure that there’s a webserver listening on 192.168.3.10
So i need to have apache or nginx running in 192.168.3.10 ?