pfSense Support Subscription

Author Topic: pfflowd and duplicate flow records  (Read 715 times)

0 Members and 1 Guest are viewing this topic.

Offline wallabybob

  • Hero Member
  • *****
  • Posts: 5262
    • View Profile
pfflowd and duplicate flow records
« on: March 16, 2012, 07:03:57 pm »
Though pfflowd is involved I have posted this topic in this forum since the possibility of duplicate flow records being passed to pfflowd may be of more general interest, particularly to those people who have reported significant anomolies in various places where byte counts are reported.

I use pfflowd to send flow records to a Linux system on which I use the flow-tools package to capture and analyse those records. Some time ago I noticed duplicate flow records. I'll give an example later. Thinking it might be a quirk of pfflowd I went and looked at the options available through the web GUI and saw pf rule direction restriction - Restrict creation of flow records to states matching a certain direction (in, out, or any). and noted it was set to Any. The explanation wasn't particularly helpful to understanding what setting I should choose nor was the corresponding FreeBSD man page so I decided to change to in which seemed to stop the duplicate records. However big downloads did not show up in the records. In particular, a 12MB download from 72.21.194.22 (from states reported by pftop) seemed to have been lost from the records (in the following reports flow-cat concatenates a number of flow files, removing file headers and flow-nfilter selects specified flow record, in these examples selecting a specific IP address and flow-print displays the selected flow records):
Quote
# flow-cat ft-v05.2012-03-17.* | flow-nfilter -F ip-dst-addr -v ADDR=72.21.194.22 | flow-print -f 1   
Sif  SrcIPaddress     DIf  DstIPaddress      Pr SrcP DstP  Pkts  Octets
 StartTime          EndTime             Active   B/Pk Ts Fl

000c 192.168.211.241  000a 72.21.194.22      06 d6b6 50    4          180       
 0317.07:24:02.498  0317.07:25:43.498    101.000 45  00 00

# flow-cat ft-v05.2012-03-17.* | flow-nfilter -F ip-src-addr -v ADDR=72.21.194.22 | flow-print -f 1
Sif  SrcIPaddress     DIf  DstIPaddress      Pr SrcP DstP  Pkts  Octets
 StartTime          EndTime             Active   B/Pk Ts Fl

000a 72.21.194.22     000c 192.168.211.241   06 50   d6b6  2          92       
 0317.07:24:02.498  0317.07:25:43.498    101.000 46  00 00

#
I changed the pfflowd rule direction restriction to Out and downloaded a (different) 12MB file from 63.173.70.10 (again, IP address taken from state shown by pftop) and this time saw:
Quote
# flow-cat ft-v05.2012-03-17.* | flow-nfilter -F ip-dst-addr -v ADDR=63.173.70.10 | flow-print -f 1
Sif  SrcIPaddress     DIf  DstIPaddress      Pr SrcP DstP  Pkts  Octets
 StartTime          EndTime             Active   B/Pk Ts Fl

000c 192.168.211.241  000a 63.173.70.10      06 b052 50    5843       314821   
 0317.07:42:18.614  0317.07:48:14.614    356.000 53  00 00

# flow-cat ft-v05.2012-03-17.* | flow-nfilter -F ip-src-addr -v ADDR=63.173.70.10 | flow-print -f 1
Sif  SrcIPaddress     DIf  DstIPaddress      Pr SrcP DstP  Pkts  Octets
 StartTime          EndTime             Active   B/Pk Ts Fl

000a 63.173.70.10     000c 192.168.211.241   06 50   b052  9933       14410509 
 0317.07:42:18.614  0317.07:48:14.614    356.000 1450 00 00

#
Next I changed the pfflowd rule direction restriction to Any and downloaded the same file (though this time pftop reported it coming from 80.239.224.51) and saw duplicate flow records:
Quote
# flow-cat ft-v05.2012-03-17.* | flow-nfilter -F ip-dst-addr -v ADDR=80.239.224.51 | flow-print -f 1
Sif  SrcIPaddress     DIf  DstIPaddress      Pr SrcP DstP  Pkts  Octets
 StartTime          EndTime             Active   B/Pk Ts Fl

000c 192.168.211.241  000a 80.239.224.51     06 8cc4 50    5864       314565   
 0317.08:02:48.917  0317.08:10:53.917    485.000 53  00 00

000c 192.168.211.241  000a 80.239.224.51     06 8cc4 50    5864       314565   
 0317.08:02:48.917  0317.08:10:53.917    485.000 53  00 00

# flow-cat ft-v05.2012-03-17.* | flow-nfilter -F ip-src-addr -v ADDR=80.239.224.51 | flow-print -f 1
Sif  SrcIPaddress     DIf  DstIPaddress      Pr SrcP DstP  Pkts  Octets
 StartTime          EndTime             Active   B/Pk Ts Fl

000a 80.239.224.51    000c 192.168.211.241   06 50   8cc4  9949       14440741 
 0317.08:02:48.917  0317.08:10:53.917    485.000 1451 00 00

000a 80.239.224.51    000c 192.168.211.241   06 50   8cc4  9949       14440741 
 0317.08:02:48.917  0317.08:10:53.917    485.000 1451 00 00

#

1. What is the meaning of each of the settings of the pfflowd rule direction restriction? (It seems to need a fair bit of knowledge of firewall rule generation hidden by the web GUI and the kernel pf component to know the meaning of the different settings.)
2. What is the correct setting of of the pfflowd rule direction restriction to record at least the flows through the WAN interface? (Any seems to mean more than both in and out).
3. Does the apparent duplicate flow reporting have any implications for other packages interested in byte usage? (I don't know if pfflowd is generating the duplicate or its just passing on a duplicate reported by the kernel.)