pfSense Support Subscription

Author Topic: Can't DHCP from Cable modem  (Read 5496 times)

0 Members and 1 Guest are viewing this topic.

Offline !real

  • Newbie
  • *
  • Posts: 12
  • Karma: +0/-0
    • View Profile
Can't DHCP from Cable modem
« on: March 29, 2012, 05:38:08 pm »
Running pfSense 2.0.1 in ESXi 4.1 VMware, I have 3 WAN's on single interface on 3 VLAN's,  

WAN1 el0-VLAN900 > DHCP cable modem
WAN2 el0-VLAN100 > Static microwave
WAN3 el0-VLAN200 > DHCP microwave

Problem is pfSense will never get DHCP from the cable modem on WAN1.  It DHCP's perfectly from WAN3, and static WAN2 works flawless.  

I have powered on and off in every order I can think of.  Always power modem off when switching from device to device. All ignore bogon boxes are unchecked.  Log shows dhclient is asking for IP but fails with dhclient error "No DHCPOFFERS received."  I can do a packet capture on WAN1 and see all the DHCP and ARP requests for everyone else on the cable system.  The pfSense charts around 25kbs-35kbs data inbound on the WAN1 interface and appears to be all the broadcast traffic from the cable system.

I can power off modem, plug laptop to modem, power on modem, laptop gets public IP (NOT private IP) and works flawless.

After pfSense tries to get an IP, I can (without powering down the modem) plug laptop into the modem with static IP=192.168.100.5/24 GW=192.168.100.1/24, then access the modem web page 192.168.100.1 and see that it has captured the correct MAC address entered in the psSense WAN1 interface page.

Running SuddenCrap cable, or Suddenlink as they call themselves, cable modem is Cisco 2100 (which this one apparently only works in bridge mode).

Any idea what is going wrong?  Could the firewall somehow blocking the DHCP packets on the cable interface?  I feel sure it is something simple I'm looking over. ::)

  

Offline wallabybob

  • Hero Member
  • *****
  • Posts: 5262
  • Karma: +0/-0
    • View Profile
Re: Can't DHCP from Cable modem
« Reply #1 on: March 29, 2012, 06:13:53 pm »
Could the firewall somehow blocking the DHCP packets on the cable interface?
Have you checked the firewall log? What firewall rules do you have on WAN1?

Could VMware be filtering the response?

Offline !real

  • Newbie
  • *
  • Posts: 12
  • Karma: +0/-0
    • View Profile
Re: Can't DHCP from Cable modem
« Reply #2 on: March 29, 2012, 09:01:17 pm »
Could the firewall somehow blocking the DHCP packets on the cable interface?
Have you checked the firewall log? What firewall rules do you have on WAN1?


Could VMware be filtering the response?
I see no logs at all for WAN1, log shows WAN3 blocking lots of stuff.

I have no rules under any of the WAN interfaces.  WAN3 with no rules DHCP's fine, WAN1 with no rules does not. 

I don't think the VMware can be filtering anything.  All WAN's are on a single interface on separate VLAN's, same setup for WAN1 and WAN3 and WAN3 works fine.

Doing a packet capture in pfSense and downloading into Wireshark I see my pfSense do 12 DHCP Discover, but never see any DHCP Offers to my pfSense, I do see DHCP offers for others on the network.

Exactly where in the process is the pfSense packet capture taken?  Before or after the firewall rules?  If a packet is blocked by the firewall will it show up in the packet capture?

Offline cmb

  • Administrator
  • Hero Member
  • *****
  • Posts: 6333
  • Karma: +0/-0
    • LinkedIn
    • Twitter
    • View Profile
    • Chris Buechler
Re: Can't DHCP from Cable modem
« Reply #3 on: March 29, 2012, 10:16:06 pm »
Not firewall rule related (unless you're putting in some floating rules overriding the default rules). Cable ISPs commonly won't assign a different device an IP, for some just power cycling the modem suffices, for others you have to either spoof the MAC of whatever device had a lease on there previously, or call them. Seriously doubt it's anything other than that based on your description.

Offline cmb

  • Administrator
  • Hero Member
  • *****
  • Posts: 6333
  • Karma: +0/-0
    • LinkedIn
    • Twitter
    • View Profile
    • Chris Buechler
Re: Can't DHCP from Cable modem
« Reply #4 on: March 29, 2012, 10:36:18 pm »
One other thing came to mind - any MAC your modem sees may be considered a device connected. For instance I have two public IPs via DHCP with my cable provider, and that means they only allow the first two MAC addresses seen. I ran into troubles when I had a managed switch between the cable modem and my firewalls because something the switch was emitting (STP IIRC, it's been a while) was being picked up by the modem as one of my MACs. So the switch was counted as one of my devices even though it wasn't pulling an IP from them. At one point I got someone with a clue on the phone and they upped it to 3 for me so I could actually use two devices, and that got reset at some point and I couldn't get anyone with a clue so ended up just having to put an unmanaged switch in there. If your ESX host is talking on that network (shouldn't unless it has a management IP or similar on that vswitch) or you have a managed switch, you could be hitting the same thing.

Offline wallabybob

  • Hero Member
  • *****
  • Posts: 5262
  • Karma: +0/-0
    • View Profile
Re: Can't DHCP from Cable modem
« Reply #5 on: March 29, 2012, 10:40:33 pm »
Exactly where in the process is the pfSense packet capture taken?  Before or after the firewall rules?  If a packet is blocked by the firewall will it show up in the packet capture?
I believe packet capture is before firewall processing on input to pfSense and after firewall processing on output from pfSense.

I presume you have some sort of VLAN capable switch between the WAN1 cable modem and the pfSense VM. Is that correctly configured - for example, is the cable modem port a member of the correct VLAN? Does the switch have a monitor capability (e.g. direct traffic on a VLAN or a port to a 'monitor' port)? If so, you might be able to use your laptop to see traffic on the VLAN of which the cable modem is a member.

Maybe you could get get some help from the ISP tech support: are they seeing your DHCP requests? are they favourably responding to them?

Does the cable modem have some sort of counter you could look at to verify it is getting the DHCP requests from pfSense - e.g. broadcast packets received on the Ethernet interface?

Offline !real

  • Newbie
  • *
  • Posts: 12
  • Karma: +0/-0
    • View Profile
Re: Can't DHCP from Cable modem
« Reply #6 on: March 29, 2012, 11:25:24 pm »
One other thing came to mind - any MAC your modem sees may be considered a device connected. For instance I have two public IPs via DHCP with my cable provider, and that means they only allow the first two MAC addresses seen. I ran into troubles when I had a managed switch between the cable modem and my firewalls because something the switch was emitting (STP IIRC, it's been a while) was being picked up by the modem as one of my MACs. So the switch was counted as one of my devices even though it wasn't pulling an IP from them. At one point I got someone with a clue on the phone and they upped it to 3 for me so I could actually use two devices, and that got reset at some point and I couldn't get anyone with a clue so ended up just having to put an unmanaged switch in there. If your ESX host is talking on that network (shouldn't unless it has a management IP or similar on that vswitch) or you have a managed switch, you could be hitting the same thing.

No floating firewall rules.  My setup is currently pretty much factory install with the exception of the 2 extra WAN interfaces.

I do have a managed Cisco 2960 that all my WANs go through.  I too have had the managed switch blues where it grabbed the only MAC but have solved that a long time ago with other setups.  Plus I can view the modem's web page and verify the modem has locked onto my pfSense MAC address and not something else.  The modem seems to lock to anything else with just a power cycle, a laptop or WRT54GL router.

Odd thing is I had this same setup working flawlessly several months back on the exact same switches, same server, same modem, same ESXi VMware, until a power failure corrupted everything and pfSense came back up with all new different interface names, forcing a reinstall.  I remember having this problem when I built the first setup but can't remember what I did to solve it.


FYI: This config will keep a Cisco switch from taking a MAC from the modem.  This will keep the Cisco switch from speaking until spoken too, and has worked great for me in the past.

interface GigabitEthernet0/1
 description WAN from Cable Modem
 switchport access vlan 900
 switchport mode access
 no keepalive
 no cdp enable
 spanning-tree portfast
 spanning-tree bpdufilter enable
end



Offline !real

  • Newbie
  • *
  • Posts: 12
  • Karma: +0/-0
    • View Profile
Re: Can't DHCP from Cable modem
« Reply #7 on: March 29, 2012, 11:44:35 pm »
Exactly where in the process is the pfSense packet capture taken?  Before or after the firewall rules?  If a packet is blocked by the firewall will it show up in the packet capture?
I believe packet capture is before firewall processing on input to pfSense and after firewall processing on output from pfSense.

I presume you have some sort of VLAN capable switch between the WAN1 cable modem and the pfSense VM. Is that correctly configured - for example, is the cable modem port a member of the correct VLAN? Does the switch have a monitor capability (e.g. direct traffic on a VLAN or a port to a 'monitor' port)? If so, you might be able to use your laptop to see traffic on the VLAN of which the cable modem is a member.

Maybe you could get get some help from the ISP tech support: are they seeing your DHCP requests? are they favourably responding to them?

Does the cable modem have some sort of counter you could look at to verify it is getting the DHCP requests from pfSense - e.g. broadcast packets received on the Ethernet interface?


I can span a port on the switch and sniff the traffic.  I'm not physically at the switch, its at the other end of a piece of fiber so I have to go to the switch.  Not too hard, but not like its here in the basement.  That was why I was asking about the pfSense packet capture, trying to figure out if it may be missing something.  Probably best to span a port on the switch and watch it to be sure I'm seeing all the packets.

The web interface on this modem is super simple, not much there at all.

I hate the thought of even trying to find any intelligence at their support.  Every time I call all I get is "have you tried turning off and back on yet..."
 

Offline wallabybob

  • Hero Member
  • *****
  • Posts: 5262
  • Karma: +0/-0
    • View Profile
Re: Can't DHCP from Cable modem
« Reply #8 on: March 30, 2012, 12:20:39 am »
[Odd thing is I had this same setup working flawlessly several months back on the exact same switches, same server, same modem, same ESXi VMware, until a power failure corrupted everything and pfSense came back up with all new different interface names, forcing a reinstall.  I remember having this problem when I built the first setup but can't remember what I did to solve it.
Any chance you have a backup of previous pfSense configuration?

Presumably you also lost your VMware config. Is that backed up?

Offline !real

  • Newbie
  • *
  • Posts: 12
  • Karma: +0/-0
    • View Profile
Re: Can't DHCP from Cable modem
« Reply #9 on: March 30, 2012, 01:21:22 am »
[Odd thing is I had this same setup working flawlessly several months back on the exact same switches, same server, same modem, same ESXi VMware, until a power failure corrupted everything and pfSense came back up with all new different interface names, forcing a reinstall.  I remember having this problem when I built the first setup but can't remember what I did to solve it.
Any chance you have a backup of previous pfSense configuration?

Presumably you also lost your VMware config. Is that backed up?

Unfortunately I never backed up the original pfSense install.  The ESXi didn't crash, it is the same setup I had before, I just created a new VM.  I had first tried to reset my original install to factory defaults but it seemed the OS was hosed as the interface names were all screwy, so I just started over.

Offline wallabybob

  • Hero Member
  • *****
  • Posts: 5262
  • Karma: +0/-0
    • View Profile
Re: Can't DHCP from Cable modem
« Reply #10 on: March 30, 2012, 05:32:54 am »
I would try to verify the DHCP request is at least getting onto the cable plugged into the modem and look for DHCP responses there.

That you created a new VM for pfSense suggests you may not have precisely recreated the VM configuration used by the working pfSense This may not be significant unless the packet capture at the switch (or on the cable to the modem) shows both DHCP request and response.

Offline cmb

  • Administrator
  • Hero Member
  • *****
  • Posts: 6333
  • Karma: +0/-0
    • LinkedIn
    • Twitter
    • View Profile
    • Chris Buechler
Re: Can't DHCP from Cable modem
« Reply #11 on: March 30, 2012, 04:52:02 pm »
I believe packet capture is before firewall processing on input to pfSense and after firewall processing on output from pfSense.

yes, it's what is on the wire, with the only exception being checksums if you have hardware checksum offloading (in which case the checksums will be 0).

Offline !real

  • Newbie
  • *
  • Posts: 12
  • Karma: +0/-0
    • View Profile
Re: Can't DHCP from Cable modem
« Reply #12 on: March 31, 2012, 01:00:40 am »
OK, spanned a port on the switch and used Wireshark to sniffed the traffic in and out of the WAN interface on the VM host.

I see a DHCP discover leaving my pfSense with the correct MAC address, then immediately see the cable provider reply with a  DHCP offer containing my MAC address, but my pfSense never replies with a DHCP request.

Using the pfSense packet capture, I see the DHCP discover leave the server with my MAC address, but never see any reply back with my MAC address.  I see all kinds of traffic on the wire coming from the cable including APR's and DHCP offers for other users.  Its like others make it in but mine don't.

I can spoof my pfSense MAC address on a XP VM in the same ESXi host, in the ESXi host disconnect the WAN interface from my pfSense VM, connect  the WAN interface to the XP machine and it instantly acquires a public IP address from the cable company and the connection works fine to the Internet.  This at least verifies my cable modem is not locked to some other MAC.

Any ideas?  I will continue searching for a solution.


Offline biggsy

  • Sr. Member
  • ****
  • Posts: 380
  • Karma: +0/-0
    • View Profile
Re: Can't DHCP from Cable modem
« Reply #13 on: March 31, 2012, 03:30:10 am »
I can't see why it would be needed but have you enabled promiscuous mode on the vswitch? 




 

Offline !real

  • Newbie
  • *
  • Posts: 12
  • Karma: +0/-0
    • View Profile
Re: Can't DHCP from Cable modem
« Reply #14 on: March 31, 2012, 06:51:09 am »
I can't see why it would be needed but have you enabled promiscuous mode on the vswitch? 


I just tried setting promiscuous mode from "reject" to "accept", didn't make any difference. 

I don't understand how VMware could be the problem, WAN1 and WAN3 are on the same vinterface on the same vswitch, the interface VLAN is set "VLAN ID: All (4095)", WAN3 works flawless while WAN1 does not, and WAN2 works flawless though it is static IP.