Netgate SG-1000 microFirewall

Show Posts

This section allows you to view all posts made by this member. Note that you can only see posts made in areas you currently have access to.

Messages - cpk

Pages: [1] 2 3
I have a pfSense firewall with 2 WAN ports.  Our main WAN is a slow but reliable bonded T1.  Our other WAN is a fast but unreliable cable service.  Our default route using our main WAN as does all of our inbound traffic (SMTP, HTTPS, DNS, etc.). I use pfSense 2.4.2-RELEASE-p1 (amd64) running on Intel(R) Core(TM) i5-2400 CPU @ 3.10GHz.  I'm using firewall rules with gateway groups to route standard user traffic (HTTP, HTTPS, and misc. others) to the faster WAN.

A recent cable update gives us 200Mbps (download) network.  Typical (Speakeasy) speed test over the firewall shows 50-75Mbit download speeds while the same test connected directly to the cable modem shows 200Mbit download speeds.

I found that if I have a firewall rule with "advanced options: gateway" set, the throughput speeds are significantly different.  The following tests were done with two machines and one firewall, and had only 1 rule changed.  The tests were run multiple times to be sure that the results were reproducible.

With a rule allowing traffic with no gateway specified, the test download/upload speeds were 960/820 Mbps.
With the same rule modified to specify a gateway (the same gateway that was used anyway, so this was redundant), the download/upload speeds changed to 390/40 Mbps.

Can anyone suggest why this happens or what I'm doing wrong that could be causing this discrepancy?

The problem happened again, so here's what I was able to test/determine:
  • Once the problem happens, no email goes out to the Internet from that computer (several different servers were attempted)
  • DNS lookups work
  • From that computer I cannot ping (which works typically)
  • From that computer, I can access the Internet using a web browser -- I suspect because I have ports 80 and 443 load balanced with a different Internet connection.
  • I did not see anything unusual in the mail server's mail.log
  • I did not see anything unusual in the mail server's system.log
  • I hadn't mentioned before that networking internally to that server works as normal.
It feels like pfSense receives the packet for SMTP connection and doesn't know what to do with it.  One thing I forgot to test was SMTP connection from another computer on the same network ( something like this: telnet 25 ).  I'll try that next time.

Is there any way to determine how pfSense is routing a connection?

Any other suggestions?

No, there's nothing of interest in the mail server system log.

Yes I have NAT rules.  I use 1:1 NAT for each public-facing machine.

The next time I have a problem, I'll try to access the Internet from the mail server (likely just use a browser to visit

I've disabled snort for now, so the system log goes back days instead of minutes.  That will allow me to view the system log when this problem occurs again.

I was working with the impression that some email was still working while other email was not.  Turns out that this is not the case (at least not on April 14).  All SMTP connections from our mail server to servers outside our network failed with "Network is unreachable".

I am also working with the impression that other Internet traffic is still working when this happens.  I have verified this by checking a web server log that shows we were receiving web traffic.  I can also confirm that inbound SMTP was working to our mail filter at that time.

If you can think of anything else I should check or test, please let me know.

I don't see any breaks in the graphs.

What I'm hoping to get here are some theories of what might be happening and some ideas of how to prove or disprove those theories (even if it's something to look at the next time the problem occurs).

I looked for similar problems online, and this was the closest I could find:
Unfortunately, it doesn't explain anything about why the problem happened or why the fix worked.

We have static IPs from our ISPs.

Not with this version.  I have a bonded T1 Internet connection which never goes down and a Cable Internet that fails from time to time (until you reset the cable modem).  In general, this has been working fairly well since 2.1.5 came out.  The log entries I posted were the last two in the Gateways section.  We had some Cable outages on April 10 but were good through most of March.

I'm SO sorry for forgetting the details.

pfSense: 2.1.5-RELEASE (i386)

I have snort enabled, so my system log only goes back a few minutes.  I don't believe this is snort-related because I don't block any outbound traffic.  However, I'd be happy to disable that if you think it would be a good diagnostic step.  Nothing listed in the gateways log:

Apr 10 08:25:37 -- apinger: alarm canceled: WAN_CABLEGW( *** down ***
Apr 14 14:40:02 -- apinger: Starting Alarm Pinger, apinger(19472)

The problem appeared today at 14:32:26 (just before I rebooted the router).

Routing and Multi WAN / Error Sending Email: Network is unreachable
« on: April 14, 2015, 02:54:13 pm »
Every so often, my email server complains that email can't be sent:
 ... server postfix/smtp ... dsn=4.4.1, status=deferred (connect to ... Network is unreachable)

When this happens, if I reboot my pfSense system and flush my email queue, the messages are delivered almost instantly.

I've searched this forum for similar problems, but I have not been able to find problems similar to mine.  Can someone give some advice on how to diagnose this problem?

Routing and Multi WAN / Re: Slow traffic when gateway rule is configured.
« on: October 01, 2014, 04:40:15 pm »
UPDATE:  For my situation, in our live environment I found a rule that was causing our traffic to go over a slower link.  Once I fixed that, I was less interested in why I had this problem in the test environment and quit researching.

I'm running 2.1.4.  I think this problem started for me when I upgraded to 2.1.3 (but I can't verify that).

I would like some advice on how to troubleshoot this problem further.

Currently, I'm setting up a test firewall in an attempt to duplicate the symptoms.  If I can do that, I might revert to an older pfSense to see if my theory that an upgrade introduced this.

Routing and Multi WAN / Slow traffic when gateway rule is configured.
« on: July 15, 2014, 02:56:56 pm »
I have the latest pfSense installed with two Internet connections (T1 & Cable).  Because our cable is unreliable, our default route is our T1.  To get better speed, I set up a Gateway Group named "Load_Balanced" that prefers the Cable gateway and includes the T1 gateway.  I added a Rule so that traffic destined to the Internet via ports 80 or 443 are routed through the Load_Balanced gateway.

This has worked well since January.  Recently (I can't tell when), the Internet connection slowed to a crawl, so I did some testing.  Here's what I found:

I'm testing network speed using a web browser and a utility from  I have the web server connected to a network switch on our Cable WAN network (along with our cable modem).  With no specific rules, I can connect to this server and get 80-90 Mbps download speed.  *BUT* when I add a rule that tells my traffic (by specific port destination or by source IP address) to use that network (by setting a gateway for that rule), my download speed drops to about 10 Mbps.

I'm looking for ideas on how to further troubleshoot this problem, and I'm looking for anyone else who's experienced significant slowdown of Internet speed after updating pfSense so we can compare configurations.

Routing and Multi WAN / Re: 4 public IP addresses , same modem
« on: July 15, 2014, 02:53:52 pm »
If you're using DHCP addresses, how would you set up a service?

I have a similar configuration, but my ISP gives me 5 static IP addresses.  I configured a VirtualIP for each IP I want to use a service with.

Poppy, did you ever get this to work?  I'm in a similar situation, and I'd like to know what you found.

Pages: [1] 2 3