next »

[Alan87i]

I've been running PRTG on a windows XP box and have had poor results with softflowd V9 data. PRTG displays a small percentage of the actual data used.
I have switched back to the PFsense package Pfflowd. Witch seems to send data V9 that PRTG can interpret correctly. But PRTG has issues with the flow time outs when using Pfflowd.
I also have an issue with filtering local traffic where a PC running pf sense has 2 lans. One being a static route too another network with it's own WAN gateway.
If a package that exported flow data could be configured too only export flows between X and Y interfaces leaving Z interface out of the picture I would love to try it!

It seems that PRTG can not filter an interface with Letters in it's name. Mine for example is (EM1) and (EM2) and (BGE0).

[wallabybob]

If you have a Linux or Unix system available you could run flow-tools (home page http://code.google.com/p/flow-tools) to analyse your flow records. There is a variety of filter and report options. There is a reasonable writeup in Network Flow Analysis by Michael W Lucas, ISBN 978-1-59327-203-6

I run pfflowd on pfSense, direct the flow records to a collector program (flow-capture) on a Linux system and use flow-report and flow-nfilter on the Linux system to generate reports.

[RedRep]

I would kick in $20.

[serialdie]

Ill pitch in another $20.00

[slth]

Any updates on this matter? I find it hard to believe something as simple as bandwidth usage statistics per IP would be so hard to develop..

I'll trow in another 10$

[xbipin]

that makes it $280 but still no1 to do it, probably some1 should mention what their target amount is and then can try and meet that

[Alan87i]

that makes it $280 but still no1 to do it, probably some1 should mention what their target amount is and then can try and meet that

Yes please someone give us an idea of what it would take to get this rolling. I think a package like this should be a part of PFsense.

[slth]

Yesterday, I had a chat with the vnStat author. I asked him if he had any plans to implement this feature directly into vnStat, here are some excerpts from our conversation:

<Vergo> that would require a complete rewrite since the linux kernel doesn't provide that information directly. I wouldn't integrate that sort of feature into vnStat
<Vergo> the thing is, the kernel provides the information about traffic per interface directly so vnStat can just query it and sleep between the queries
<Vergo> filtering traffic per ip would require inspecting every packet and that's a totally different thing
<Vergo> it might be possible to cheat a little bit and use iptables for getting the data but the end result wouldn't work in anything else than linux and even that would have some restrictions

I asked if he had any idea of something we are looking for already exists for FreeBSD:

<Vergo> I've had some plans for writing at least some kind of proof of concept program for doing per ip stats with a console based program but haven't so far found time to start it
<Vergo> darkstats is the closest there currently is and it isn't exactly what you are searching since it's also filtering target ips, doesn't provide simple stats from console and can't survive a restart

So I guess we are pretty much out of luck with this bounty, as far as my understanding goes, an entirely new package is needed to accomplish the listing of usage per IP..  

[Alan87i]

Thanks for the Info.!
I'll update the topic subject and revise this bounty for those keeping track.

A vnstat "like" package to monitor bandwidth usage PER LAN IP

[Jimmy_uk]

I will post a further $20.00 for the development of this feature/package.

[wallabybob]

I was asked to elaborate on my earlier post about flow tools to get per IP usage stats.

pfSense needs a flow collector installed. I used pfflowd. pfflowd sends flow records to a collector. I used flow-capture from the flow-tools package which I installed on a Linux system. flow-capture stores its flow records in directories, one for each day.  Mostly I'm interested in finding out who has used the most data during a day so I can take appropriate action if the monthly download quota looks like being exceeded. My ISP makes available daily usage stats and from them I can see about 9GB was downloaded on 19 Nov. So to see who was downloading and from where on the Linux system holding my flow records I can:

[root@sme ~]# pushd /var/db/flows/2011/2011-11/2011-11-19/
/var/db/flows/2011/2011-11/2011-11-19 ~
[root@sme 2011-11-19]# flow-cat * | flow-report -v TYPE=ip-source/destination-address/ip-source/destination-port -v SORT=+octets | more
#  --- ---- ---- Report Information --- --- ---
# build-version:        flow-tools 0.68
# name:                 default
# type:                 ip-source/destination-address/ip-source/destination-port
# options:              +header,+xheader,+totals
# ip-src-addr-type:     address
# ip-dst-addr-type:     address
# sort_field:           +octets
# fields:               +key1,+key2,+key3,+key4,+flows,+octets,+packets,+duration,+other
# records:              165068
# first-flow:           1321624808 Sat Nov 19 00:00:08 2011
# last-flow:            1321711187 Sat Nov 19 23:59:47 2011
# now:                  1322602258 Wed Nov 30 07:30:58 2011
#
# mode:                 streaming
# compress:             off
# byte order:           little
# stream version:       3
# export version:       5
#
#  ['/usr/bin/flow-rptfmt', '-f', 'ascii']
ip-source-address ip-destination-address ip-source-port ip-destination-port flows octets    packets duration
64.188.166.206    192.168.211.244        6881           6881                4     282428402 205832  4594000
173.194.28.84     192.168.211.244        80             51905               2     52110568  35902   468000 
58.174.20.228     192.168.211.244        25565          58525               2     38788562  61226   2782000
12.129.255.100    192.168.211.244        3724           56975               2     31560842  357082  7366000
74.125.109.182    192.168.211.244        80             52042               2     25368268  17660   376000 
125.252.225.176   192.168.211.244        80             58396               2     25020948  17238   978000 
117.121.249.80    192.168.211.244        80             52876               2     23684584  16464   522000 
195.8.214.79      192.168.211.244        80             50283               2     21343766  14708   578000 
12.120.15.208     192.168.211.244        80             52877               2     21147556  14578   520000 
125.252.225.176   192.168.211.244        80             58423               2     18952452  13060   2258000
125.252.225.176   192.168.211.244        80             58380               2     18219946  12566   866000 
117.121.249.75    192.168.211.244        80             57241               2     17291682  11948   462000 
173.194.28.106    192.168.211.244        80             51947               2     16064040  11196   392000 
195.8.214.37      192.168.211.244        80             58489               2     15804278  10890   445000 
117.121.249.81    192.168.211.244        80             52620               2     15645356  10894   386000 
125.252.225.151   192.168.211.244        80             52377               2     14250122  9828    354000 
12.120.14.206     192.168.211.244        80             52606               2     14162172  9764    422000 
125.252.225.152   192.168.211.244        80             52431               2     13742162  9576    466000 
125.252.225.152   192.168.211.244        80             52432               2     13539082  9430    466000 
74.125.109.143    192.168.211.244        80             49399               2     13522672  9392    298000 
125.252.225.151   192.168.211.244        80             52874               2     11739240  8098    526000 
74.125.10.15      192.168.211.244        80             49470               2     11368880  7976    278000 
125.252.225.152   192.168.211.244        80             52600               2     11335216  7818    450000 
192.168.211.244   222.154.97.65          6881           6881                22    10993548  13430   4250000
64.233.183.132    192.168.211.216        443            42135               2     10846542  7822    1804000
192.168.211.244   64.188.166.206         6881           6881                4     10631398  207608  4594000
121.223.82.76     192.168.211.244        6881           6881                2     10495864  9388    3062000
125.252.225.151   192.168.211.244        80             52397               2     9478840   6536    360000 
12.129.255.91     192.168.211.244        3724           57334               2     9141684   103534  2410000
192.168.211.244   12.129.255.100         56975          3724                2     8131908   178936  7366000
222.154.97.65     192.168.211.244        6881           6881                20    7941028   13162   4184000
195.8.214.22      192.168.211.244        80             58467               2     6694386   4618    440000 
173.194.28.113    192.168.211.244        80             51989               2     6375462   4456    344000 
125.252.225.151   192.168.211.244        80             53094               2     5851168   4186    1024000

flow-cat reads a bunch of flow files and removes headers and writes a stream of flow records to stdout.  There is a flow-nfilter program which can strip specified flow records from the stream (e.g. flows between LAN and OPT1, flows over specified time intervals). flow-report has a number of reporting and sorting options. There are more advanced reporting options discussed in the book including graphing options.

I would like to take a look at software described in http://www.manageengine.com/products/netflow/ which seems to have much more extensive reporting capability. It is commercial software but there is a free edition which handles a limited number of interfaces.

[Alan87i]

If you sort this forum but Most views count this topic is at the top of the list under the sticky's .
Any response from the pfsense team?
Is there a chance you could work this feature into the OS. I'm sick of running 2 computers just to monitor bandwidth. I just want a list per month of all lan too wan traffic sorted by lan IP.
If you commit I'll send the coin asap! 

[xbipin]

im ready to pledge some more coins if some1 seriously is willing to complete this and im sure others will add more of the coins once we know what it would take to complete it

[akghetto]

As I understand it, said package would monitor total bandwidth by IP across multiple NICs, so if I have multiple internal subnets routing through pfsense, I can monitor how much IPs on the multiple LANs are transferring in and out across my WAN link(s).  If this understanding is accurate, count me in for $50 *if* said package would also support IPv6/pfsense 2.1.  I need IPv6 accounting as well.

[Alan87i]

Well It's been just over 1 year and over 7000 views on this topic . And not one hint of anything from the pf team.