pfSense Support Subscription

Author Topic: New userland per-ip bandwidth manager for pfSense.  (Read 10441 times)

0 Members and 1 Guest are viewing this topic.

Offline JorgeAldoBR

  • Jr. Member
  • **
  • Posts: 64
  • Karma: +0/-0
    • View Profile
New userland per-ip bandwidth manager for pfSense.
« on: January 19, 2008, 07:23:00 am »
I started a little project for pfSense : An userland traffic shaper (per ip bandwidth) using virtual tun/tap interfaces.

I have a concept program already running and needing some tests.

Im stuck now at developing a traffic shapping algorithm, i want one similar to the one used on dummynet, and i need to do some tests with a linux box (easier to program with the tools im using, but its portable to freebsd once ready...)

Who can help me ?

I can send a copy of the program to anyone willing to test this (it currently doesnt do any traffic shaping at all, i just need to know if this project is worth as i dont know if the kernel <-> userland ovehead is too costly to pay on current hardware).
« Last Edit: January 19, 2008, 07:24:34 am by JorgeAldoBR »

Offline sullrich

  • Hero Member
  • *****
  • Posts: 5110
  • Karma: +3/-0
    • View Profile
    • pfSense
Re: New userland per-ip bandwidth manager for pfSense.
« Reply #1 on: January 19, 2008, 02:43:08 pm »
I would visit the algo's used in dummynet.  WFQ+ and there are a few others.

Offline JorgeAldoBR

  • Jr. Member
  • **
  • Posts: 64
  • Karma: +0/-0
    • View Profile
Re: New userland per-ip bandwidth manager for pfSense.
« Reply #2 on: January 20, 2008, 09:36:27 am »
I think i will use a multithreaded leacky bucket, its the most used algorithm anyway...

Lets explore our options :

Im thinking about the following usage :

Tap0 is the downstream
Tap1 is the upstream

We bridge Tap0 with all LAN-like devices (Tap0, Lan, Opt interfaces, etc).

We make pfSense think that Tap1 is the real LAN device and run our captive portal there.

So we endup solving two problems : Now we can have the captive portal running across multiple interfaces (Even if they are bridged with Tap0, Tap1 isnt bridged with anything so the captive portal will not complain...)

Later we block traffic comming-in/out on the Lan interfaces so all traffic allowed will pass thru Tap0 into the traffic shaper.

On the traffic shaper daemon I plan to put a socket on a port (bound to the Tap1) so the current captive portal can change traffic shaping rules dinamically...

This setup looks reasonable ?
« Last Edit: January 20, 2008, 09:48:26 am by JorgeAldoBR »

Online ermal

  • Administrator
  • Hero Member
  • *****
  • Posts: 3364
  • Karma: +1/-0
    • View Profile
Re: New userland per-ip bandwidth manager for pfSense.
« Reply #3 on: January 20, 2008, 12:47:16 pm »
If you're doing this in C.

1- Can't you just port ALTQ_WFQ to PF and that it is. It's not even that hard to do it.
Just don't reinvent the wheel and in userland it will be slow, belive me.

2- IF you want a utility to control take a look at ng_car and make it a normal kernel module, not hard too.
3- I have ported dummynet to PF if you feel to have the knowledge to do some tests on it, ping me back(or search the freebsd-pf lists for dummynet+PF).

I strongly suggest you focus on something usable and worth its time and consider my suggestions.

The ALTQ_WFQ is on my TODO with some nice additions to ALTQ, but i have to find time for it yet.


Offline JorgeAldoBR

  • Jr. Member
  • **
  • Posts: 64
  • Karma: +0/-0
    • View Profile
Re: New userland per-ip bandwidth manager for pfSense.
« Reply #4 on: January 20, 2008, 04:08:05 pm »
I dont have enough knowledge to do kernel programing.

I started this because i really need that my pfsense box do all the work, my clients dont like when i say they need to setup two boxes (one running pfsense the other running monowall).

I will finish this project as a userland traffic shaper and use it until someone solves the pf rdr + ipfw(dummynet) incompatibility.

Doesnt look like a too hard project, its now more of a problem about selecting the right algorithm.

Here in Brazil bandwidth is not cheap, and the speed of the links are slow.

An ISP selling 128kb to a client is very common. I dont think a small WISP will notice the overhead of a userland application...

Online ermal

  • Administrator
  • Hero Member
  • *****
  • Posts: 3364
  • Karma: +1/-0
    • View Profile
Re: New userland per-ip bandwidth manager for pfSense.
« Reply #5 on: January 20, 2008, 04:25:01 pm »
If you fill like playing with it.
Here is the patch.
Full syntax of dummynet the only change is that pipe is called dnpipe and that queue is called dnqueue.

Look here too.
http://lists.freebsd.org/pipermail/freebsd-pf/2007-October/003849.html

also take a look at this link:
http://mpd.sourceforge.net/doc/mpd62.html

on the radius section it says it already has per user ip-bandwidth limiting.

But it is your choice anyway.
« Last Edit: January 20, 2008, 04:35:37 pm by eri-- »

Online ermal

  • Administrator
  • Hero Member
  • *****
  • Posts: 3364
  • Karma: +1/-0
    • View Profile
Re: New userland per-ip bandwidth manager for pfSense.
« Reply #6 on: January 22, 2008, 03:38:13 am »
Follow the discussion on the sticky tags where HFSC is being discussed something useful can be contructed from that which will give you what you want without coding a thing.

Offline JorgeAldoBR

  • Jr. Member
  • **
  • Posts: 64
  • Karma: +0/-0
    • View Profile
Re: New userland per-ip bandwidth manager for pfSense.
« Reply #7 on: January 31, 2008, 10:02:04 pm »
Well, if someone is porting dummynet for pf then i will stop developing this...

Im working now on a transproxy for monowall.

My idea is to run a program inside monowall that intercepts http traffic and transfer it to a DMZ squid (While keeping track if squid is running).

If squid is down (ping) then traffic is transfered directly to the intended server.

It will do more, it will detect if the user is logged and transfer the request to the monowall internal server if not.

As i can run multiple instances (multithreaded), i can make (side effect) the captive portal run across multiple interfaces too...

So we end up with three features : Squid transparent + captive portal + multiple interfaces.

The end result is that if this is implemented in monowall i will drop pfsense and use Squid on a OpenBSD machine at the DMZ...
« Last Edit: January 31, 2008, 10:05:02 pm by JorgeAldoBR »

Online ermal

  • Administrator
  • Hero Member
  • *****
  • Posts: 3364
  • Karma: +1/-0
    • View Profile
Re: New userland per-ip bandwidth manager for pfSense.
« Reply #8 on: February 01, 2008, 03:26:17 am »
Quote
The end result is that if this is implemented in monowall i will drop pfsense and use Squid on a OpenBSD machine at the DMZ...
What is wrong with you people what do you try to say with such stupid sentences.

Dummynet is already ported to PF. Search the forum i have posted the patch. If you don't find it ping me i will give you the patch.

Quote
Im working now on a transproxy for monowall.
There is such a thing in freebsd ports, strangely with the same name!

Quote
It will do more, it will detect if the user is logged and transfer the request to the monowall internal server if not.

As i can run multiple instances (multithreaded), i can make (side effect) the captive portal run across multiple interfaces too...

So we end up with three features : Squid transparent + captive portal + multiple interfaces.
Sometimes programming is more than just writing code!
Captve portal with multiple interfaces cannot be done now since of PF not supporting mac addresses, which will be corrected.

But, as always, go for what works for you and just keep in mind before posting stupid sentences what that actually mean:
1- Is ofending others job worth it?
2- Is saying thank you so hard?
3- What i will do if anybody wrote that sentence for me?

and do not get me wrong, but it just an ethical thing i am discussing here.

Offline JorgeAldoBR

  • Jr. Member
  • **
  • Posts: 64
  • Karma: +0/-0
    • View Profile
Re: New userland per-ip bandwidth manager for pfSense.
« Reply #9 on: February 01, 2008, 08:03:22 am »
English is not my native language.

Im trying to help the community donating my spare time. If you solved a problem, i think its useless for me to continue developing a inferior solution.

As for dropping pfSense, the only reason for me to use pfSense is because it does squid. Everything else i can do with monowall. If i manage to build a good monowall based solution i can drop pfSense. (Instaling pfSense just for the sake of squid is "bad investiment").

On the day that pfSense can do squid + captive portal + traffic shapping, on a single machine, i will restart to use it.
« Last Edit: February 01, 2008, 09:42:12 am by JorgeAldoBR »

Offline JorgeAldoBR

  • Jr. Member
  • **
  • Posts: 64
  • Karma: +0/-0
    • View Profile
Re: New userland per-ip bandwidth manager for pfSense.
« Reply #10 on: February 01, 2008, 10:37:42 am »
[Captve portal with multiple interfaces cannot be done now since of PF not supporting mac addresses, which will be corrected.]

I my humble opnion, based on experiences, MAC filtering is useless in the environments i work with. The way to go is certificate + ip filtering. Most of my clients are small wireless ISP and most APs filters out the clients MAC address. So the only thing important is to the intercepting proxy to redirect the user to the portal page and be able to know afterwards wich page the user wanted to go (This is where im stuck now). Im thinking about some sort of popup with the captive portal page, while the main page keeps track of the original page.

Something like :

"you are not logged."
"Click HERE to open the login popup"
"Click HERE to continue after you log in"

But this is another question not related to this topic.

Online ermal

  • Administrator
  • Hero Member
  • *****
  • Posts: 3364
  • Karma: +1/-0
    • View Profile
Re: New userland per-ip bandwidth manager for pfSense.
« Reply #11 on: February 01, 2008, 10:45:38 am »
If you have the resources why don;t you open a bounty for these features to get implemented.

IMHO, squid should not be on the pfSense box cause of they orthogonal issues.