"with like an authentication keyfile or something on their computer "
If you have outside people that you want to limit to access your game.. Simple enough to limit your port forward to their source IPs - if they know them and they do not change all the time. Another option would be to just let them vpn in.. And then through the vpn access the game.. This way you know for a FACT that its them, since they will be the only ones that can auth to your vpn via the cert you give them.