[multi-WAN load balancing]

Just to be sure: we are talking about multi-WAN load balancing (for traffic going from the LAN to the Internet)?

[there is one howto that i followed in 2.0.1 that works]

in my experience, YES. no simple steps, no shortcut methods and no standard way to do it. Standard in a sense that all the HOWTO posted here MIGHT work in your system. But most of the time, they don't. For example, there is one howto that i followed in 2.0.1 that works but in 2.0.3 it doesn't.

Would, by any luck, still have it somewhere?

I heard some users are using a separate proxy server. that is also my plan, but i am a window baby, so i have a hard time of installing and configuring squid in unix or different OS.

I'll give it a try and check if pfSense allows that (either via pfSense's Squid + "upsteam proxy" setting or without pfSense's Squid and two simples Firewall/NAT rules to allow direct TCP 80 access from OtherSquid and translate any TCP 80 traffic from LAN to OtherSquid:3128)

squid uses the default wan (in you case, ISP1) for http 80.
the rest can be switched as per your testing in mtr.

the link you posted sometimes work, sometimes not.
it's the reason why i don't want to use proxy because of it's inability to failover/loadbalance without tweaking or adding numerous changes to pfsense.
and you need a lot of LUCK to run the proxy in failover. 

You're saying there is no simple way to make Squid works with a multi-WAN load-balanced/fail-overed pfSense setup?
What about using a separate server running Squid and specified in the pfSense's Squid configuration as an upstream proxy server (of course, this server won't be configured to use pfSense's Squid)?

[is it safe (and not too cumbersome) to switch to a manual (Manual Outbound NAT rule generation / AON - Advanced Outbound NAT) setting ?]

I'm having concerns with DimitriS's tutorial regarding the NAT Outbound part: is it safe (and not too cumbersome) to switch to a manual (Manual Outbound NAT rule generation / AON - Advanced Outbound NAT) setting ?

I'm actually using Automatic outbound NAT rule generation on my setup which is the following:

                                                        ,-----{WAN0 interface}--[ ISP0's Modem ]
                                                       /    ,--{WAN1 interface}--[ ISP1's Modem ]
[ LAN switch ]---{LAN interface}---[ pfSense ]
                                                       \    `--{WAN2 interface}--[ ISP2's Modem ]
                                                        `-----{WAN3 interface}--[ ISP3's Modem ]

I have a load balacing over each WAN0-3 with a Gateway group named GW_LoadBalancing that is used in the following Firewall rule:

ID	Proto	Source	Port	Destination	Port	Gateway	Queue	Schedule
	*	LAN net	*	*	*	GW_LoadBalancing	none

No servers are housed behind pfSense, appart the IPSec and PPTP VPNs (served by pfSense itself).

Edit: I've tried the "Manual Outbound NAT" setting with advised floating firewall rule and NAT rules: no luck (any proxied request always goes through the same WAN connection) :-/

Should I just configure a cron to periodically restart the DNS Forwarder?

Sticky keeps an association between a client IP and a gateway so long as there are active states for that client.

This is how I understood the sticky connections works, but as the "Show States" display no active states I think there is somewhere something that "inadvertently" prevents the sticky connections option to achieve it's goal by ending connections (thus becoming in FIN_WAIT_2): it might simply be HTTP "Connection: close" header.

In https://github.com/bsdperimeter/pfsense/commit/4573641589d50718b544b778cea864cfd725078a I added a GUI field to control the state tracking timeout so that sticky association can be held longer.

I'll give it a try...

What some people do is direct HTTPS into a failover group instead of load balancing.

In this case, the websites aren't served via HTTPS protocol.

One funny thing I found while testing: some site's backoffice where I was always "kicked" of in 3 or 4 seconds were now working just fine (or at least 10 minutes) after I added the following firewall rule just before my "load balancing" rule:

Code:

TCP 192.168.0.0/24 * * 80 (HTTP) GW_LoadBalancing none

So my firewall rules are:

Code:

TCP LAN net * * 443 (HTTPS) GW_WanA_FO_WanB none
TCP 192.168.0.0/24 * * 80 (HTTP) GW_LoadBalancing none
* LAN net * * * GW_LoadBalancing none

I can't really explain how this could be of any impact as the rule #3 does the same job as the newly added rule #2.

[but only for a few seconds]

I'm having issues with pfSense outgoing connections load balancing (through my 3 WAN Internet connections) and many websites that requires some sort of authentication (via a simple web/HTML form):
When you try to access the protected page you get redirected to the login form, once credentials sent you get access to the protected pages but only for a few seconds: about 20-25 second (sometimes even less but never more than a minute) after login-in the the next asked page redirects you to the login form.

I've tried using the "Use sticky connections" option (located in "System: Advanced: Miscellaneous > Load Balancing") but it doesn't work either.

I've checked connections states (located in "Diagnostics: Show States") and filtered it on one the website's IP (82.165.xx.yy) for which I have the issue and they are all in "FIN_WAIT_2":

Proto    Source -> Router -> Destination    State    
tcp    82.165.xx.yy:80 <- 192.168.0.55:55919    FIN_WAIT_2:FIN_WAIT_2    
tcp    192.168.0.55:55919 -> 88.160.aa.bb:53387 -> 82.165.xx.yy:80    FIN_WAIT_2:FIN_WAIT_2

• 192.168.0.55 is the LAN IP of my computer
• 88.160.aa.bb is the WAN IP of one of my WAN Internet connection
• 82.165.xx.yy is the IP of the webserver

So my guess (and it's seems to be what support@optimalnetworks.c guessed in it's topic: sticky sessions) is that, as states are "FIN_WAIT_2:FIN_WAIT_2", connections are dropped.

Side note: All three WAN connections are working just fine and, apart from that web-session issue all seems to be good. The websites aren't using HTTPS but simple clear HTTP so I can't create a rule to use my no-loadbalancing gateway group.

Edit: typo in first paragraph.

[Debian machine (192.168.0.5, with bind9 9.7.3)]

Hello,

I might have a configuration problem with DNS forwarder.

Setup:
I already have a DNS server running on a Debian machine (192.168.0.5, with bind9 9.7.3) which is responsible for resolving the internal local domain "mycompany.ext." to the LAN (192.168.0.0/24). This domain is automatically populated by Dynamic DNS, the DHCP service being on the same Debian server.

The pfSense (192.168.0.1) is responsible for resolving the other Internet domains with the "DNS servers" list located at "System: General Setup":

12.34.56.78    WAN_A

Computers located inside the LAN are told (via DHCP) to use 192.168.0.1 (pfSense) as DNS server (and as their gateway) so that Internet resolution can be asked to 12.34.56.78 by pfSense on behalf of local computers.
To resolve "mycompany.ext." domain I tried to use "Services: DNS forwarder" by adding the following infos ne into the "Domain Overrides" list:

Domain: mycompany.ext
IP: 192.168.0.5

Problem:
This Internet-resolution works fine but the local is buggy: at start it works fine, but after some time it won't find resolve local addresses:

$ nslookup xxxxx.mycompany.ext 192.168.0.1
server can't find xxxxx.mycompany.ext: NXDOMAIN

Whereas if asking to the Debian DNS:

$ nslookup xxxxx.mycompany.ext 192.168.0.5
Name:   xxxxx.mycompany.ext
Address: 192.168.0.102

For the pfSense to give an answer again, I have to disable and then enable the "DNS forwarder".

Possible cause:
I suspect the DNS forwarder service to have asked 192.168.0.5 for "xxxxx.mycompany.ext" once while this machine was out of the office for a while (thus it's A record were deleted/obsolated from 192.168.0.5 and it's returned that answer to pfSense) and to have cached this result. Thus, even when the A record has been recreated by DDNS on 192.168.0.5, pfSense keeps saying NXDOMAIN to any request.

I get the idea.
It's always a choice between fine tuning precision (providing some security) and ease of administration
I'll stick with security then.

[If]

Hello,

I'm still in the process of installing a pfSense router for:

• One local network (192.168.1.0/24)
• Three different Internet connections (say: WAN_InternetA, WAN_InternetB, WAN_InternetC)

I'm now testing NAT Port forwarding (PAT that is) and I was wondering if it was possible to create one NAT rules for multiple interfaces?
Example:
Say I have one web server on 192.168.1.105 that I want to be accessed from the outside (whatever WAN interface it is coming on) on port say 1086. It seems that I have to create 3 rules : one for each WAN interface (.../firewall_nat.php).

If	Proto	Src. addr	Src. ports	Dest. addr	Dest. ports	NAT IP	NAT Ports	Description
WAN_InternetA	TCP/UDP	*	*	*	1086	192.168.1.105	80 (HTTP)	Awesome webserver
WAN_InternetB	TCP/UDP	*	*	*	1086	192.168.1.105	80 (HTTP)	Awesome webserver
WAN_InternetC	TCP/UDP	*	*	*	1086	192.168.1.105	80 (HTTP)	Awesome webserver

I thought I could circumvent this by creating Interfaces Groups (.../interfaces_groups.php) but they don't populates the "Interface" drop-down field when creating a new NAT rule (.../firewall_nat_edit.php).
Is there any solution to avoid creating multiple rules?
Thanks

What is a possible source of problems in your setup is, that you assigned the parent interface on which VLAN's are assigned.
--> em1 is directly assigned.

OK, that's what I was afraid of.
I deleted one of my previous "WAN_x" and renamed/reconfigured "WAN" as the previously deleted.

[won't this bring problems?]

I'm in the process of installing a pfSense router for:

• One local network (192.168.1.0/24)
• Three different Internet connections (say: WAN_InternetA, WAN_InternetB, WAN_InternetC)

As the machine I'm running pfSense on (live from a USB stick for now) only has two network interfaces (em0 and em1) I've plugged the three Internet modems on a switch and created 3 VLANs (VLAN IDs : 2, 3 and 4), one for each modem. My server em1 interface is connected to a tagged port of this switch.
On pfSense webGUI I've configured the 3 corresponding VLANs and I now have the following Interface setup:


For now I'm only testing with WAN_InternetA so the other two interfaces are disabled.

What's bothering me is the "WAN" interface that I have to map to a network port (em1 in the example), however, it is disabled: won't this bring problems?
In theory I only need 4 interfaces in total : one "LAN" and three "WAN_".

Thanks for your help on that particular question.

Hello,

I'm also trying to put a pfSense VM into Xen (running on a Debian 6).

By following the "DevelopersBootStrapAndDevIso" guide (on a VirtualBox-ed FreeBSD VM), I'm running into the following issue when running ./build_iso.sh:

chmod: /usr/local/pfsense-fs/usr/local/lib/php/20060613/ioncube/ioncube_loader_fre_5.2_ts.so: No such file or directory
>>> Installation collected library information (usr/local), please wait...
>>> chroot'ing and running /etc/rc.php_ini_setup
>>> Copying config.xml from conf.default/ to cf/conf/
WARNING: attempt to domain_add(netgraph) after domainfinalize()
Loaded /boot/kernel/ng_socket.ko, id=2
>>> Testing PHP installation in /usr/local/pfsense-fs:
Fatal error: Call to undefined function pfSense_get_interface_addresses() in /etc/inc/globals.inc on line 49

An error occured while testing the php installation in /usr/local/pfsense-fs


####################################
Something went wront, check errors!
####################################

NOTE: a lot of time you can run ./clean_build.sh to resolve.


Press enter to continue.

Clearly, the builder is lacking some PHP components but why?