VLAN Problem VIA based Nexgate NSA-1041 since 1.2.3 RC2

[mibo]

Hi pfSense Team,

i have a problem with my NSA-1041 Hardware. Since 1.2.3 RC2 VLANs are not working. (Also the release version). Normal networking without VLANs works.
Releases before 1.2.3 RC2 are working fine.

Was there a driver change or sth. else?

My Southbridge is a VT8235M.

Can anyone give me a hint to get the 1.2.3 relase working?

[Perry]

Things you can try.
Upgrade firmware/bios
Post ifconfig and pciconf -lv of the nic here
Install a stock freebsd 7.2 to confirm problem

[wallabybob]

It might also help to elaborate on "VLANs are not working."

[mibo]

So here some informations


if you configure a VLAN it fells like no network connection. Interface (physical and vlan) is up, but no traffic.

i have seen that between RC1 and RC3 freebsd was updated to 7.2. and now i have more options in the interfaces than before. can a new default interface option cause this?

here my testing plan:

- booting from LiveCD. No harddisk. working with defaults.
- setup vlan0 on fxp1
- setup lan on fxp0. connected to my notebook
- setup wan on vlan0. connected to DHCP Server (company LAN)

now you can not reach any host behind WAN(vlan). it doesnt matter if i test on my notebook or in the ssh shell.

with freebsd 7.1 (1.2.3 RC1) it works.



infos from 1.2.3 release (not working):

Code: [Select]

uname -a
FreeBSD pfSense.local 7.2-RELEASE-p5 FreeBSD 7.2-RELEASE-p5 #0: Sun Dec  6 23:20:31 EST 2009    
sullrich@FreeBSD_7.2_pfSense_1.2.3_snaps.pfsense.org:/usr/obj.pfSense/usr/pfSensesrc/src/sys/pfSense_SMP.7  i386

Code: [Select]

ifconfig
fxp0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=219b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,WOL_MAGIC>
        ether 00:10:f3:0e:19:ce
        inet 192.168.1.1 netmask 0xffffff00 broadcast 192.168.1.255
        inet6 fe80::210:f3ff:fe0e:19ce%fxp0 prefixlen 64 scopeid 0x1
        media: Ethernet autoselect (100baseTX <full-duplex>)
        status: active
fxp1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=219b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,WOL_MAGIC>
        ether 00:10:f3:0e:19:cf
        inet6 fe80::210:f3ff:fe0e:19cf%fxp1 prefixlen 64 scopeid 0x2
        media: Ethernet autoselect (100baseTX <full-duplex>)
        status: active
fxp2: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=219b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,WOL_MAGIC>
        ether 00:10:f3:0e:19:d0
        media: Ethernet autoselect (none)
        status: no carrier
fxp3: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=219b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,WOL_MAGIC>
        ether 00:10:f3:0e:19:d1
        media: Ethernet autoselect (none)
        status: no carrier
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
        inet 127.0.0.1 netmask 0xff000000
        inet6 ::1 prefixlen 128
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x5
enc0: flags=0<> metric 0 mtu 1536
pfsync0: flags=41<UP,RUNNING> metric 0 mtu 1460
        pfsync: syncdev: lo0 syncpeer: 224.0.0.240 maxupd: 128
pflog0: flags=100<PROMISC> metric 0 mtu 33204
vlan0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=3<RXCSUM,TXCSUM>
        ether 00:10:f3:0e:19:cf
        inet6 fe80::210:f3ff:fe0e:19ce%vlan0 prefixlen 64 scopeid 0x9
        inet 0.0.0.0 netmask 0xff000000 broadcast 255.255.255.255
        media: Ethernet autoselect (100baseTX <full-duplex>)
        status: active
        vlan: 10 parent interface: fxp1


infos from 1.2.3 RC1 (working):

Code: [Select]

uname -a
FreeBSD pfSense.local 7.1-RELEASE-p5 FreeBSD 7.1-RELEASE-p5 #0: Wed Apr 22 15:35:36 EDT 2009    
sullrich@RELENG_1_2-snapshots.pfsense.org:/usr/obj.pfSense/usr/pfSensesrc/src/sys/pfSense_SMP.7  i386

Code: [Select]

ifconfig
fxp0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=b<RXCSUM,TXCSUM,VLAN_MTU>
        ether 00:10:f3:0e:19:ce
        inet 192.168.1.1 netmask 0xffffff00 broadcast 192.168.1.255
        inet6 fe80::210:f3ff:fe0e:19ce%fxp0 prefixlen 64 scopeid 0x1
        media: Ethernet autoselect (100baseTX <full-duplex>)
        status: active
fxp1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=b<RXCSUM,TXCSUM,VLAN_MTU>
        ether 00:10:f3:0e:19:cf
        inet6 fe80::210:f3ff:fe0e:19cf%fxp1 prefixlen 64 scopeid 0x2
        media: Ethernet autoselect (100baseTX <full-duplex>)
        status: active
fxp2: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=b<RXCSUM,TXCSUM,VLAN_MTU>
        ether 00:10:f3:0e:19:d0
        media: Ethernet autoselect (none)
        status: no carrier
fxp3: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=b<RXCSUM,TXCSUM,VLAN_MTU>
        ether 00:10:f3:0e:19:d1
        media: Ethernet autoselect (none)
        status: no carrier
plip0: flags=108810<POINTOPOINT,SIMPLEX,MULTICAST,NEEDSGIANT> metric 0 mtu 1500
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
        inet 127.0.0.1 netmask 0xff000000
        inet6 ::1 prefixlen 128
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x6
enc0: flags=0<> metric 0 mtu 1536
pflog0: flags=100<PROMISC> metric 0 mtu 33204
pfsync0: flags=41<UP,RUNNING> metric 0 mtu 1460
        pfsync: syncdev: lo0 syncpeer: 224.0.0.240 maxupd: 128
vlan0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        ether 00:10:f3:0e:19:cf
        inet6 fe80::210:f3ff:fe0e:19ce%vlan0 prefixlen 64 scopeid 0xa
        inet 192.168.3.110 netmask 0xfffffe00 broadcast 192.168.3.255
        media: Ethernet autoselect (100baseTX <full-duplex>)
        status: active
        vlan: 10 parent interface: fxp1

[Perry]

Warning for those using Intel PRO/100 cards – there is a regression in the fxp driver in FreeBSD 7.2 that may require disabling hardware checksum offloading under System -> Advanced if you have connectivity problems.

[mibo]

i have tried disabling hardware checksum offloading but no luck...

on fxp0 the RXSUM and TXSUM option disappeared, but on vlan0 and fxp1 not.

see ifconfig

Code: [Select]

fxp0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=2198<VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,WOL_MAGIC>
        ether 00:10:f3:0e:19:ce
        inet 192.168.1.1 netmask 0xffffff00 broadcast 192.168.1.255
        inet6 fe80::210:f3ff:fe0e:19ce%fxp0 prefixlen 64 scopeid 0x1
        media: Ethernet autoselect (100baseTX <full-duplex>)
        status: active
fxp1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=219b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,WOL_MAGIC>
        ether 00:10:f3:0e:19:cf
        inet6 fe80::210:f3ff:fe0e:19cf%fxp1 prefixlen 64 scopeid 0x2
        media: Ethernet autoselect (100baseTX <full-duplex>)
        status: active
fxp2: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=219b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,WOL_MAGIC>
        ether 00:10:f3:0e:19:d0
        media: Ethernet autoselect (none)
        status: no carrier
fxp3: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=219b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,WOL_MAGIC>
        ether 00:10:f3:0e:19:d1
        media: Ethernet autoselect (none)
        status: no carrier
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
        inet 127.0.0.1 netmask 0xff000000
        inet6 ::1 prefixlen 128
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x5
enc0: flags=0<> metric 0 mtu 1536
pfsync0: flags=41<UP,RUNNING> metric 0 mtu 1460
        pfsync: syncdev: lo0 syncpeer: 224.0.0.240 maxupd: 128
pflog0: flags=100<PROMISC> metric 0 mtu 33204
vlan0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=3<RXCSUM,TXCSUM>
        ether 00:10:f3:0e:19:cf
        inet6 fe80::210:f3ff:fe0e:19ce%vlan0 prefixlen 64 scopeid 0x9
        inet 0.0.0.0 netmask 0xff000000 broadcast 255.255.255.255
        media: Ethernet autoselect (100baseTX <full-duplex>)
        status: active
        vlan: 10 parent interface: fxp1


[mibo]

Switching the interfaces will switch the RXCSUM Options.

see ifconfig

Code: [Select]

fxp0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=219b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,WOL_MAGIC>
        ether 00:10:f3:0e:19:ce
        inet6 fe80::210:f3ff:fe0e:19ce%fxp0 prefixlen 64 scopeid 0x1
        media: Ethernet autoselect (100baseTX <full-duplex>)
        status: active
fxp1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=2198<VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,WOL_MAGIC>
        ether 00:10:f3:0e:19:cf
        inet 192.168.1.1 netmask 0xffffff00 broadcast 192.168.1.255
        inet6 fe80::210:f3ff:fe0e:19cf%fxp1 prefixlen 64 scopeid 0x2
        media: Ethernet autoselect (100baseTX <full-duplex>)
        status: active
fxp2: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=219b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,WOL_MAGIC>
        ether 00:10:f3:0e:19:d0
        media: Ethernet autoselect (none)
        status: no carrier
fxp3: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=219b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,WOL_MAGIC>
        ether 00:10:f3:0e:19:d1
        media: Ethernet autoselect (none)
        status: no carrier
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
        inet 127.0.0.1 netmask 0xff000000
        inet6 ::1 prefixlen 128
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x5
enc0: flags=0<> metric 0 mtu 1536
pfsync0: flags=41<UP,RUNNING> metric 0 mtu 1460
        pfsync: syncdev: lo0 syncpeer: 224.0.0.240 maxupd: 128
pflog0: flags=100<PROMISC> metric 0 mtu 33204
vlan0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=3<RXCSUM,TXCSUM>
        ether 00:10:f3:0e:19:ce
        inet6 fe80::210:f3ff:fe0e:19ce%vlan0 prefixlen 64 scopeid 0x9
        inet 0.0.0.0 netmask 0xff000000 broadcast 255.255.255.255
        media: Ethernet autoselect (100baseTX <full-duplex>)
        status: active
        vlan: 10 parent interface: fxp0

[Perry]

You could try the manual way
ifconfig fxp0 -rxcsum -txcsum

[mibo]

sorry.. did not work. the option disappears but it still not work..

i had a look at a packet capture. At the DHCP discover on the WAN interface i can not see a VLAN tag.
other packet from my network come with a correct VLAN tag.


the vlan id is 10 and the following value appears in the ip packet:

Code: [Select]

Differentiated Services Field: 0x10 (DSCP 0x04: Unknown DSCP; ECN: 0x00)
0001 00.. = Differentiated Services Codepoint: Unknown (0x04)
.... ..0. = ECN-Capable Transport (ECT): 0
.... ...0 = ECN-CE: 0


i think the vlan tagging is not working correctly...

[jimp]

If it's not working properly, it must be something specific to your fxp chip revision.

I'm using VLANs heavily on fxp cards on 1.2.3-RELEASE and it works just fine for me, either with onboard NICs or add-in cards. It does look like yours have tso, you might try disabling that if it isn't already:

Code: [Select]

sysctl -w net.inet.tcp.tso=0
The output of these commands might also help:

Code: [Select]

# pciconf -lv | grep fxp0
fxp0@pci0:0:11:0:       class=0x020000 card=0x000c8086 chip=0x12298086 rev=0x08 hdr=0x00

# ifconfig -m fxp0
fxp0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=2009<RXCSUM,VLAN_MTU,WOL_MAGIC>
        capabilities=2049<RXCSUM,VLAN_MTU,POLLING,WOL_MAGIC>
        ether 00:02:b3:03:80:28
        inet6 fe80::202:b3ff:fe03:8028%fxp0 prefixlen 64 scopeid 0x1
        media: Ethernet autoselect (100baseTX <full-duplex>)
        status: active
        supported media:
                media autoselect
                media 100baseTX mediaopt full-duplex
                media 100baseTX
                media 10baseT/UTP mediaopt full-duplex
                media 10baseT/UTP
                media none
                media 100baseTX mediaopt hw-loopback

[mibo]

TSO is already disabled.

Code: [Select]

# sysctl -w net.inet.tcp.tso
net.inet.tcp.tso: 0

here the infos:

Code: [Select]

# pciconf -lv | grep fxp0
fxp0@pci0:0:9:0:        class=0x020000 card=0xffffffff chip=0x12098086 rev=0x10 hdr=0x00

Code: [Select]

# ifconfig -m fxp0
fxp0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=2198<VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,WOL_MAGIC>
        capabilities=21db<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,POLLING,VLAN_HWCSUM,TSO4,WOL_MAGIC>
        ether 00:10:f3:0e:19:ce
        inet6 fe80::210:f3ff:fe0e:19ce%fxp0 prefixlen 64 scopeid 0x1
        media: Ethernet autoselect (100baseTX <full-duplex>)
        status: active
        supported media:
                media autoselect
                media 100baseTX mediaopt full-duplex
                media 100baseTX
                media 10baseT/UTP mediaopt full-duplex
                media 10baseT/UTP
                media none
                media 100baseTX mediaopt hw-loopback

[jimp]

Looks like yours has a later chip revision and claims to do VLANs in hardware. Mine doesn't claim such support, not sure any of them do.

Perhaps it's a problem along those lines, but unfortunately it's a bit beyond my level of expertise. It might be interesting to try a stock FreeBSD 7.2 install on there to see if you have similar problems, or it may be easier to try a pfSense 2.0 beta snapshot to see if the problem persists on FreeBSD 8.

[mibo]

So i have tested a pfSense 2.0 snapshot.
The same tests as before were made but the problem also exists in this version with the same behaviour.


to complete the tests here the outputs:

Code: [Select]

# ifconfig -m fxp0
fxp0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=2198<VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,WOL_MAGIC>
        capabilities=21db<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,POLLING,VLAN_HWCSUM,TSO4,WOL_MAGIC>
        ether 00:10:f3:0e:19:ce
        inet 192.168.1.1 netmask 0xffffff00 broadcast 192.168.1.255
        inet6 fe80::210:f3ff:fe0e:19ce%fxp0 prefixlen 64 scopeid 0x1
        nd6 options=3<PERFORMNUD,ACCEPT_RTADV>
        media: Ethernet autoselect (100baseTX <full-duplex>)
        status: active
        supported media:
                media autoselect
                media 100baseTX mediaopt full-duplex
                media 100baseTX
                media 10baseT/UTP mediaopt full-duplex
                media 10baseT/UTP
                media none
                media 100baseTX mediaopt hw-loopback

Code: [Select]

# ifconfig
fxp0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=2198<VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,WOL_MAGIC>
        ether 00:10:f3:0e:19:ce
        inet 192.168.1.1 netmask 0xffffff00 broadcast 192.168.1.255
        inet6 fe80::210:f3ff:fe0e:19ce%fxp0 prefixlen 64 scopeid 0x1
        nd6 options=3<PERFORMNUD,ACCEPT_RTADV>
        media: Ethernet autoselect (100baseTX <full-duplex>)
        status: active
fxp1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=2198<VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,WOL_MAGIC>
        ether 00:10:f3:0e:19:cf
        inet6 fe80::210:f3ff:fe0e:19cf%fxp1 prefixlen 64 scopeid 0x2
        nd6 options=3<PERFORMNUD,ACCEPT_RTADV>
        media: Ethernet autoselect (100baseTX <full-duplex>)
        status: active
...

fxp1_vlan10: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        ether 00:10:f3:0e:19:cf
        inet6 fe80::210:f3ff:fe0e:19ce%fxp1_vlan10 prefixlen 64 scopeid 0xa
        inet 0.0.0.0 netmask 0xff000000 broadcast 255.255.255.255
        nd6 options=3<PERFORMNUD,ACCEPT_RTADV>
        media: Ethernet autoselect (100baseTX <full-duplex>)
        status: active
        vlan: 10 parent interface: fxp1


Any idea how to proceed?


I have found the following lines in the FreeBSD Release Notes:

The fxp(4) driver has been improved. Changes include:
- Rx checksum offload support for 82559 or later controllers has been added
- VLAN hardware tag insertion/stripping support and Tx/Rx checksum offload for VLAN frames support have been added. Note that the VLAN hardware assistance is available only on 82550 or 82551-based controllers.

pfSense recognize my card as a "Intel 82559ER". Can the VLAN hardware support be the issue? Like using VLAN hardware tag but my card cannot do this?

[jimp]

Try this:

Code: [Select]

ifconfig fxp0 -vlanhwtag -vlanhwfilter
Not sure about the hwfilter but try it anyhow, if it doesn't work, try just -vlanhwtag

That should disable the hardware tagging. If that works, install the shellcmd package and add a command in there for that. Might help for now.

[mibo]

Solved!
at first sight.

only disabling hwtag.

Code: [Select]

ifconfig fxp0 -vlanhwtag
I will do some more tests on the system till it goes back as my carp slave (with many VLANs ;-)).


Thanks a lot for your help!