@Antibiotic most internet providers block those ports.. Those would not run over the internet no.. Your vpn client you have setup on pfsense would have zero to do with you talking to some client on another one of your networks..
Your not policy routing out your vpn are you.. That would prevent you from talking to other devices on other networks/vlans.. ping shouldn't work either..
What are you rules on the client side interface on pfsense? The rules on the server side, where your file shares are mean nothing in accessing something on that network from the client. Do you have rules in floating forcing traffic out a vpn connection?
When you policy route, and you want to access stuff on your other networks you need to make sure you setup a bypass to allow what you want.
https://docs.netgate.com/pfsense/en/latest/multiwan/policy-route.html#bypassing-policy-routing