@viragomann It's a flat network so there is no other routing other than on the PfSence (unless VLANs are getting ignored).
Network design:
ESXi - vSwitch (standard, 1 host) - portgroup 107 (vlan) ... where all of kub vms exist -> Unifi USW 24 PoE -> trunk port -> TP-Link T1600G-28TS 3.0 -> Kub network on Switch as Untagged ports ] -> Pfsense and Workstation.
workstation (192.168.1.18). Can ping it's own gateway 192.168.1.1 and the 192.168.107.1 (pfsence vlan interface).
However, it cannot ping kub-master (192.168.107.10).
workstation:
$ netstat -nr
Kernel IP routing table
Destination Gateway Genmask Flags MSS Window irtt Iface
0.0.0.0 192.168.1.1 0.0.0.0 UG 0 0 0 ens192
192.168.1.0 0.0.0.0 255.255.255.0 U 0 0 0 ens192
$ ping 192.168.107.1
PING 192.168.107.1 (192.168.107.1) 56(84) bytes of data.
64 bytes from 192.168.107.1: icmp_seq=1 ttl=64 time=0.301 ms
64 bytes from 192.168.107.1: icmp_seq=2 ttl=64 time=0.435 ms
$ sudo tcpdump -n icmp and net 192.168.107.0/24
dropped privs to tcpdump
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on ens192, link-type EN10MB (Ethernet), snapshot length 262144 bytes
10:09:21.179729 IP 192.168.1.18 > 192.168.107.1: ICMP echo request, id 17, seq 1, length 64
10:09:21.179995 IP 192.168.107.1 > 192.168.1.18: ICMP echo reply, id 17, seq 1, length 64
10:09:22.224486 IP 192.168.1.18 > 192.168.107.1: ICMP echo request, id 17, seq 2, length 64
10:09:22.224855 IP 192.168.107.1 > 192.168.1.18: ICMP echo reply, id 17, seq 2, length 64
10:09:23.248402 IP 192.168.1.18 > 192.168.107.1: ICMP echo request, id 17, seq 3, length 64
10:09:23.248687 IP 192.168.107.1 > 192.168.1.18: ICMP echo reply, id 17, seq 3, length 64
10:09:25.115966 IP 192.168.1.18 > 192.168.107.10: ICMP echo request, id 18, seq 1, length 64
10:09:26.128484 IP 192.168.1.18 > 192.168.107.10: ICMP echo request, id 18, seq 2, length 64
10:09:27.152498 IP 192.168.1.18 > 192.168.107.10: ICMP echo request, id 18, seq 3, length 64
10:09:28.176534 IP 192.168.1.18 > 192.168.107.10: ICMP echo request, id 18, seq 4, length 64
10:09:29.200461 IP 192.168.1.18 > 192.168.107.10: ICMP echo request, id 18, seq 5, length 64
10:09:30.224490 IP 192.168.1.18 > 192.168.107.10: ICMP echo request, id 18, seq 6, length 64
10:09:31.248498 IP 192.168.1.18 > 192.168.107.10: ICMP echo request, id 18, seq 7, length 64
The kub-master routing is a bit more complicated:
$ netstat -nr
Destination Gateway Genmask Flags MSS Window irtt Iface
0.0.0.0 192.168.107.1 0.0.0.0 UG 0 0 0 ens160
10.107.65.0 192.168.107.11 255.255.255.0 UG 0 0 0 tunl0
10.107.66.0 192.168.107.12 255.255.255.0 UG 0 0 0 tunl0
10.107.75.0 192.168.107.13 255.255.255.0 UG 0 0 0 tunl0
10.107.103.0 0.0.0.0 255.255.255.0 U 0 0 0 *
192.168.107.0 0.0.0.0 255.255.255.0 U 0 0 0 ens160
$ cat /etc/netplan/00-static.yaml
network:
version: 2
renderer: networkd
ethernets:
myinterface:
match:
name: ens*
dhcp4: false
dhcp6: false
addresses: [192.168.107.10/24]
nameservers:
addresses: [192.168.107.1]
routes:
- to: default
via: 192.168.107.1