I am using multiple Asterisk servers to connect to multiple providers on the internet. I also have enpoints from outside connecting to these Asterisk servers.
Endpoints connecting from outside to one of the Asterisk servers I have work just find as I have NAT forward port 5060 and RTP ports to one Asterisk server.
As you have pointed out, you are using the firewall and NAT. The problem with NAT is that an inbound port can be assigned to an internal address, but not multiple addresses. There are a couple of ways you can work around this issue with multiple Asterisk systems. One, you could assign each box to listen to a specific port such as one being on 5060, another on 5068, and another on 5046. Notice the span between port numbers? That is because in some cases sequential ports are used by one machine and you don't want them overlapping one another.
Another solution and generally the best solution is to put a pfSense firewall in bridged mode in front of your Asterisk servers and then all ports and functions can remain the same on all boxes. I prefer running the firewall in bridged mode as it gives me the most flexibility and standard network device installations. You will no longer have complications with ports and your rules can be very well defined for access.
For Asterisk VOIP systems it is extremely important to protect your ports from malicious intent. When you setup your rules make sure they only allow your endpoints access. I can't tell you how many times our clients have been compromised and systems rebuilt because the client insisted on public access. You should also make sure you have a very complicated / complex registration password for each account.
Bottom line, the pfSense in bridged mode will eliminate the complications NAT presents in a VOIP environment and make it much more flexible to manage access to multiple servers.