The pfSense Store

Author Topic: IPv6 with HE Tunnel: ping works, but TCP fails to establish  (Read 2873 times)

0 Members and 1 Guest are viewing this topic.

Offline xurubin

  • Newbie
  • *
  • Posts: 5
  • Karma: +1/-0
    • View Profile
IPv6 with HE Tunnel: ping works, but TCP fails to establish
« on: August 14, 2016, 05:17:04 pm »
I have set up a HE IPv6 tunnel using the official guide (https://doc.pfsense.org/index.php/Using_IPv6_with_a_Tunnel_Broker). I'm able to ping any ipv6 addresses from pfsense, and gif0's IPv6 address is also pingable from another host. The problem is, I cannot initiate TCP connection from pfsense, nor can another host connect to the pfsense box via TCP.

I did a tcpdump on gif0 and vtnet1 (my WAN interface). It looks like when initiating a TCP connection from pfsense for example by "curl -v -6 http://ipv6.google.com", during the three-way handshake, the third ACK packet is sent out (visible on both dumps, see attached screenshots dump-gif0.png and dump-vtnet1.png) but somehow not received by the remote host (a retransmission of SYN/ACK packet is received after around 1 second).

When receiving a TCP connection directed at gif0 (by doing "nc -l -6 8080" on my pfsense box, and performing "curl -6 http://my_ipv6_address:8080" from another IPv6 capable host), it's slightly different: on gif0, there is the initial SYN received followed outgoing SYN/ACK, but the third ACK is not received so SYN/ACK is re-transmitted (screenshot dump-listening-gif0.png). However the pcap on vtnet1 does contain the incoming third ACK (screenshot dump-listening-vtnet1.png), suggesting some packet filtering issue. In both cases there is no indication of dropped packets related to the Ipv6 addresses in question in pfsense's system log/firewall.

I have set the firewall rule to very permissive ones (screenshots rule-ipv6.png, rule-lan.png, rule-ipv6.png): in the LAN tab, allowing IPv6 from LAN subnet; in the GIF interface tab, allowing IPv6 on any source or destination. The pfsense box's WAN interface sits behind my ISP's modem/router as a DMZ, but I doubt this matters as IPv6 ping over the HE tunnel works both ways.

The output of pfctl -sr is attached as pfctl.txt

Any help is appreciated. This problem has bugged me for days :(
« Last Edit: August 15, 2016, 04:18:55 pm by xurubin »

Offline bimmerdriver

  • Sr. Member
  • ****
  • Posts: 423
  • Karma: +12/-3
    • View Profile
Re: IPv6 with HE Tunnel: ping works, but TCP fails to establish
« Reply #1 on: August 15, 2016, 01:57:19 pm »
I have set up a HE IPv6 tunnel using the official guide. I'm able to ping any ipv6 addresses from pfsense, and gif0's IPv6 address is also pingable from another host. The problem is, I cannot initiate TCP connection from pfsense, nor can another host connect to the pfsense box via TCP.

[deleted]

Any help is appreciated. This problem has bugged me for days :(
I'm using pfsense with an HE tunnel. It has been working perfectly for months. The only thing I notice is that speedtest on ipv4 is faster than through the tunnel, which is not unexpected.

I'm not clear what your problem is. Do you literally mean connect to pfsense or do you really mean connect to a host on the lan?

If go to test-ipv6.com and ipv6-test.com, what results do you get? Do either of them show any issues? (You have to add pfsense firewall rules to pass icmp echo-request, as well as icmpv6 echo-request on the client.) Except for the lack of a host name, both of those websites show no problems.

Offline xurubin

  • Newbie
  • *
  • Posts: 5
  • Karma: +1/-0
    • View Profile
Re: IPv6 with HE Tunnel: ping works, but TCP fails to establish
« Reply #2 on: August 15, 2016, 02:13:21 pm »
I meant connecting to any outside IPv6 sites via TCP from pfsense does not work (for example curl -6 http://ipv6.google.com just timeout). The exact symptom is TCP three-way handshake fails to complete, which I found out via tcpdump as in my initial post.

Offline johnpoz

  • Hero Member
  • *****
  • Posts: 13099
  • Karma: +1147/-148
  • Not a pfSense employee, they cannot fire me...
    • View Profile
Re: IPv6 with HE Tunnel: ping works, but TCP fails to establish
« Reply #3 on: August 15, 2016, 03:51:12 pm »
I run HE tunnel on pfsense 2.3.2 and have no issues at all..  Have both /64 and multiple segments from my /48 running..

Code: [Select]
[2.3.2-RELEASE][root@pfSense.local.lan]/tmp: curl -6 http://ipv6.google.com
<!doctype html><html itemscope="" itemtype="http://schema.org/WebPage" lang="en"><head><meta content="Search the world's information, including webpages, images, videos and more. Google has many special features to help you find exactly what you're looking for." name="description"><meta content="noodp" name="robots"><meta content="text/html; charset=UTF-8" http-equiv="Content-Type"><meta content="/logos/doodles/2016/2016-doodle-fruit-games-day-11-5698592858701824-hp.gif" itemprop="image"><meta content="Check out the 2016 Doodle Fruit Games at g.co/fruit #GoogleDoodle" property="og:description"><meta content="http://www.google.com/logos/doodles/2016/2016-doodle-fruit-games-day-11-5698592858701824-thp.png" property="og:image"><meta content="454" property="og:image:width"><meta content="170" property="og:image:height"><title>Google</title><script

official guide where?  Can you actually post up your config and your firewall rules (screenshots are so much better than asci art) so we can see what you might have done wrong.  Your saying like pfsense itself can not do curl to some ipv6 site like you posted..

But you are saying you get back syn,ack from your syn or you don't get anything back at all?
- An intelligent man is sometimes forced to be drunk to spend time with his fools.
- If I have helped you and want to help back, https://www.freebsdfoundation.org/donate/
- Please don't PM me for personal help, info you don't want public sure. Link to thread you would like me to look at ok, etc.
1x SG-2440 2.3.4_p1 (work)
1x 2.4.0-BETA Aug 16 07:00:51 VM running on esxi 6.5 (home)

Offline xurubin

  • Newbie
  • *
  • Posts: 5
  • Karma: +1/-0
    • View Profile
Re: IPv6 with HE Tunnel: ping works, but TCP fails to establish
« Reply #4 on: August 15, 2016, 04:20:53 pm »
I uploaded screenshots for the firewall rules as well as wireshark captures in the initial post. Hope they make sense.

Offline grandrivers

  • Sr. Member
  • ****
  • Posts: 535
  • Karma: +11/-2
    • View Profile
Re: IPv6 with HE Tunnel: ping works, but TCP fails to establish
« Reply #5 on: August 15, 2016, 06:01:11 pm »
try setting gateway for your ipv6 lan rule
pfsense 2.3.3 super micro A1SRM-2558F
C2558 8gig ECC  60gig SSD
Dual Wan Dual ipv6 tunnels

Offline xurubin

  • Newbie
  • *
  • Posts: 5
  • Karma: +1/-0
    • View Profile
Re: IPv6 with HE Tunnel: ping works, but TCP fails to establish
« Reply #6 on: August 16, 2016, 04:42:22 am »
Thanks grandrivers. Tried to set gateway on both/either LAN/HENETV6 ipv6 rules,  but still no luck.

Offline johnpoz

  • Hero Member
  • *****
  • Posts: 13099
  • Karma: +1147/-148
  • Not a pfSense employee, they cannot fire me...
    • View Profile
Re: IPv6 with HE Tunnel: ping works, but TCP fails to establish
« Reply #7 on: August 16, 2016, 06:43:37 am »
what is the point of those rules on your lan below the any any for tcp to 46.101 ??  Anything below your any any rule never going to be seen..

I see no hits on your ipv6 rule in your lan?

Looks like your getting dupes of the syn,ack - so they are never seeing your ack.. Did you contact HE?

Can you post up config of your gif, and your he interface..  Your not getting any sort of IPv6 from your isp are you on your wan?  Your he tunnel should be listed as default gateway in routes section you should not setup any specific gateways on any rules.

Your testing this all from pfsense, what happens when you test from client on your lan?
« Last Edit: August 16, 2016, 06:51:38 am by johnpoz »
- An intelligent man is sometimes forced to be drunk to spend time with his fools.
- If I have helped you and want to help back, https://www.freebsdfoundation.org/donate/
- Please don't PM me for personal help, info you don't want public sure. Link to thread you would like me to look at ok, etc.
1x SG-2440 2.3.4_p1 (work)
1x 2.4.0-BETA Aug 16 07:00:51 VM running on esxi 6.5 (home)

Offline andrewknots

  • Newbie
  • *
  • Posts: 3
  • Karma: +0/-0
    • View Profile
Re: IPv6 with HE Tunnel: ping works, but TCP fails to establish
« Reply #8 on: August 16, 2016, 07:44:36 am »
I have had the same problem since I moved.  Last night I downloaded and re-flashed the very latest pfSense release just in case...

I attach 4 tcpdump run on the endpoints of attempts to run ssh by way of an HE tunnel terminated on my pfSense firewall.  SYN and Reset packets get through but the conversation gets no further

It may or may not be relevant that I'm connecting to an HE server in London 216.66.80.26, and it appears that the missing packets are disappearing before being delivered to my pfSense down the HE tunnel
« Last Edit: August 16, 2016, 09:09:38 am by andrewknots »

Offline yon

  • Sr. Member
  • ****
  • Posts: 308
  • Karma: +1/-0
    • View Profile
Re: IPv6 with HE Tunnel: ping works, but TCP fails to establish
« Reply #9 on: August 16, 2016, 10:16:58 am »
me too, my he.net ipv6 not work now. I had use he.net ipv6 with pfsense years. but this pf 2.3.1-2.3.2 version not normal work.

Offline bimmerdriver

  • Sr. Member
  • ****
  • Posts: 423
  • Karma: +12/-3
    • View Profile
Re: IPv6 with HE Tunnel: ping works, but TCP fails to establish
« Reply #10 on: August 16, 2016, 01:25:30 pm »
My system is working perfectly. I had to reboot the modem a few days ago and it came back up on its own. I'm currently running 2.3.1-RELEASE-p5.

Offline xurubin

  • Newbie
  • *
  • Posts: 5
  • Karma: +1/-0
    • View Profile
Re: IPv6 with HE Tunnel: ping works, but TCP fails to establish
« Reply #11 on: August 16, 2016, 06:56:38 pm »
After much Googling, I have now a suspicion that this has something to do with my ISP (UK Sky Fiber) or the router it provides (SR102): http://helpforum.sky.com/t5/Fibre/Hurricane-electric-IPv6-Tunnel-Broker-on-BSkyB/td-p/2366610. That would probably explain the missing ACK when connecting to remote sites, if the router/ISP is dropping it. However, it's not apparent why the incoming connection behaves strangely: the WAN port does receive the tunneled TCP handshake packets, but pfsense is not passing them through to gif0. Unless maybe the packet is mangled somehow by the ISP causing pfsense to consider it invalid and drop it?

I'm attaching pcap files on the WAN & gif0 interfaces for the incoming connection case, which are what the screenshots in #1 are based on.

Offline andrewknots

  • Newbie
  • *
  • Posts: 3
  • Karma: +0/-0
    • View Profile
Re: IPv6 with HE Tunnel: ping works, but TCP fails to establish
« Reply #12 on: August 17, 2016, 09:10:14 am »
After discussing with Alex of HE I dropped the MSS way down to no avail.

I then switched to HE's other London Tunnel server, again no joy

I can ping6 and traceroute6 (ie ICMP  and UDP), and syn packets get through.  However using the pfSense packet Capture on the tunnel I only get the inbound SYN packet, followed by the target host sending syn-ack packets back to the initiator, which does receive them, and sends an ack which does not appear in the tunnel packet capture.

I get no further traffic from the initiator of the connection INBOUND from HE

Offline Paint

  • Full Member
  • ***
  • Posts: 207
  • Karma: +32/-2
    • View Profile
Re: IPv6 with HE Tunnel: ping works, but TCP fails to establish
« Reply #13 on: August 17, 2016, 11:30:32 am »
After discussing with Alex of HE I dropped the MSS way down to no avail.

I then switched to HE's other London Tunnel server, again no joy

I can ping6 and traceroute6 (ie ICMP  and UDP), and syn packets get through.  However using the pfSense packet Capture on the tunnel I only get the inbound SYN packet, followed by the target host sending syn-ack packets back to the initiator, which does receive them, and sends an ack which does not appear in the tunnel packet capture.

I get no further traffic from the initiator of the connection INBOUND from HE

weird... maybe it is your ISP? I have no problems with pfSense and he.net tunnels in NYC
pfSense i7-4510U + 2x Intel 82574 + 2x Intel i350 Mini-ITX Build
940/880 mbit Fiber Internet from FiOS
Dell PowerConnect 2716 Gigabit Switch
Netgear R8000 AP (DD-WRT)
Asus RT-66U AP (DD-WRT)


Offline Gertjan

  • Hero Member
  • *****
  • Posts: 1855
  • Karma: +154/-6
    • View Profile
Re: IPv6 with HE Tunnel: ping works, but TCP fails to establish
« Reply #14 on: September 24, 2016, 01:14:44 am »
More weird, because : The 'tunnel' to London, UK 216.66.80.26 is nothing less or more as a TCPv4 stream, which contains the IPv6 connection.

Your ISP is filtering these kind of packets ?
Did you check your modem / device that coverts your local Ethernet to the cable / phone (ADSL) / whatever  ?