pfSense Gold Subscription

Author Topic: Traffic shaper changes [90% completed, please send money to complete bounty]  (Read 190537 times)

0 Members and 1 Guest are viewing this topic.

Offline msamblanet

  • Newbie
  • *
  • Posts: 21
  • Karma: +0/-0
    • View Profile
Re: Traffic shaper changes
« Reply #15 on: November 20, 2006, 06:48:06 am »
Just sent in $100 to Paypal...while I would like you to consider it part of this bounty, please use it as/when the project needs...you've earned it with or without a multi-interface traffic shaper!

Offline Perry

  • Hero Member
  • *****
  • Posts: 1152
  • Karma: +0/-0
    • View Profile
Re: Traffic shaper changes
« Reply #16 on: December 02, 2006, 12:32:24 pm »
Bill what paypal account do i send my donation to?
/Perry
doc.pfsense.org

Offline aldo

  • Full Member
  • ***
  • Posts: 202
  • Karma: +0/-0
    • View Profile
Re: Traffic shaper changes
« Reply #17 on: December 02, 2006, 12:56:00 pm »
ok that was confusing could you clarify these points.
is it possible?

Possible, yes.  With the current way interfaces are configured, no.  The shaper changes I'm working on won't directly help here, but would be considered a prereq to being able to do this.

would you include it?

OK i think i understand what are the overall thoughts on this. should i start up a bounty on it.
we use pppoe server for all our wireless concentration. if this change looks achievable outside of the shaper scope i will make a bounty for it.

maybe you or scott can clarify the scope of the change a little more clearly and i can brief it

One thing at a time :)  If PPPoE server 'nics' (all the ng interfaces) were already individually assignable for rule management in pfSense, you'd get the shaper changes "for free" so to speak.  The changes I'm looking at would just come along for the ride.  As it sits, I'd consider this a different project, but one that relies on this change before it can be seriously thought of.  Depending on how the code ends up getting written, it may be possible to hack up a config.xml that'll create the correct rules - not sure, I'm still researching the proper way to do the queues as it is (it's looking like we'll have a number of nasty recursive loops).

all i am trying to do is hfsc with them no other gaurentees all have equal preferance.

well look forward to the clarification

Hope that helps.

--Bill

Offline billm

  • Administrator
  • Hero Member
  • *****
  • Posts: 731
  • Karma: +1/-0
    • View Profile
    • UCSecurity - Technology discovery and ramblings
Re: Traffic shaper changes
« Reply #18 on: December 03, 2006, 07:07:25 pm »
Bill what paypal account do i send my donation to?

paypal _at_ chrisbuechler.com if you want pfSense to hold onto it until I'm done, or billm _at_ pfsense.org if you wish to send it direct to me sooner.

--Bill
pfSense core developer
blog - http://www.ucsecurity.com/
twitter - billmarquette

Offline billm

  • Administrator
  • Hero Member
  • *****
  • Posts: 731
  • Karma: +1/-0
    • View Profile
    • UCSecurity - Technology discovery and ramblings
Re: Traffic shaper changes
« Reply #19 on: December 03, 2006, 07:09:11 pm »
Aldo, didn't see any content in that post...did I miss something?

--Bill

ok that was confusing could you clarify these points.
is it possible?

Possible, yes.  With the current way interfaces are configured, no.  The shaper changes I'm working on won't directly help here, but would be considered a prereq to being able to do this.

would you include it?

OK i think i understand what are the overall thoughts on this. should i start up a bounty on it.
we use pppoe server for all our wireless concentration. if this change looks achievable outside of the shaper scope i will make a bounty for it.

maybe you or scott can clarify the scope of the change a little more clearly and i can brief it

One thing at a time :)  If PPPoE server 'nics' (all the ng interfaces) were already individually assignable for rule management in pfSense, you'd get the shaper changes "for free" so to speak.  The changes I'm looking at would just come along for the ride.  As it sits, I'd consider this a different project, but one that relies on this change before it can be seriously thought of.  Depending on how the code ends up getting written, it may be possible to hack up a config.xml that'll create the correct rules - not sure, I'm still researching the proper way to do the queues as it is (it's looking like we'll have a number of nasty recursive loops).

all i am trying to do is hfsc with them no other gaurentees all have equal preferance.

well look forward to the clarification

Hope that helps.

--Bill
pfSense core developer
blog - http://www.ucsecurity.com/
twitter - billmarquette

Offline billm

  • Administrator
  • Hero Member
  • *****
  • Posts: 731
  • Karma: +1/-0
    • View Profile
    • UCSecurity - Technology discovery and ramblings
Re: Traffic shaper changes
« Reply #20 on: December 22, 2006, 10:16:10 pm »
Just wanted to update the thread.  I'm still working on this, had some issues with some of the new gui libraries that we needed to get fixed as well as some VM issues that are now resolved.  I'm hoping to spend some time during my vacation to get a new wizard completed which should allow me to generate configs that I can use to create the backend :)  Due to the use of the new gui library, I can pretty easily say that this won't appear in the RELENG_1 branch at all, but I'll attempt to backport it for those that have pledged and donated for this so it can get tested and have some eyes on it earlier (and of course so you can have a new toy :)).

--Bill
pfSense core developer
blog - http://www.ucsecurity.com/
twitter - billmarquette

Offline sai

  • Sr. Member
  • ****
  • Posts: 383
  • Karma: +0/-0
    • View Profile
Re: Traffic shaper changes
« Reply #21 on: December 22, 2006, 11:06:23 pm »
Thanks Bill.

I have a request: could you make the wizard optional please?

I realize that altq is really difficult to understand, but sometimes you just want to set things up yourself. This is especially true when you are trying to learn about the software.


Offline billm

  • Administrator
  • Hero Member
  • *****
  • Posts: 731
  • Karma: +1/-0
    • View Profile
    • UCSecurity - Technology discovery and ramblings
Re: Traffic shaper changes
« Reply #22 on: December 23, 2006, 09:23:35 am »
Thanks Bill.

I have a request: could you make the wizard optional please?

I realize that altq is really difficult to understand, but sometimes you just want to set things up yourself. This is especially true when you are trying to learn about the software.

The wizard is already optional.  I do plan on making the manual configuration a little more reliable and less prone to easy breakage (the real problem) though.

--Bill
pfSense core developer
blog - http://www.ucsecurity.com/
twitter - billmarquette

Offline Leoandru

  • Full Member
  • ***
  • Posts: 111
  • Karma: +0/-0
    • View Profile
Re: Traffic shaper changes
« Reply #23 on: December 27, 2006, 11:22:38 am »
Hey Bill mind if I chip in on this project? I'm finding more free time on my hand these days, so I'm specifically interested in helping with transparent shaping and investigating the muliwan/multinterface shaping of altq.

Offline billm

  • Administrator
  • Hero Member
  • *****
  • Posts: 731
  • Karma: +1/-0
    • View Profile
    • UCSecurity - Technology discovery and ramblings
Re: Traffic shaper changes
« Reply #24 on: December 28, 2006, 08:44:47 am »
Hey Bill mind if I chip in on this project? I'm finding more free time on my hand these days, so I'm specifically interested in helping with transparent shaping and investigating the muliwan/multinterface shaping of altq.

You might check out http://wiki.pfsense.com/wikka.php?wakka=NewShaperNotes.  I think I can handle bridge, and multi-lan w/out too much problem.  Multi-wan is going to be a tad more challenging I think. 

--Bill
pfSense core developer
blog - http://www.ucsecurity.com/
twitter - billmarquette

Offline Leoandru

  • Full Member
  • ***
  • Posts: 111
  • Karma: +0/-0
    • View Profile
Re: Traffic shaper changes
« Reply #25 on: December 28, 2006, 01:11:10 pm »
Hey Bill mind if I chip in on this project? I'm finding more free time on my hand these days, so I'm specifically interested in helping with transparent shaping and investigating the muliwan/multinterface shaping of altq.

You might check out http://wiki.pfsense.com/wikka.php?wakka=NewShaperNotes.  I think I can handle bridge, and multi-lan w/out too much problem.  Multi-wan is going to be a tad more challenging I think. 

--Bill

cool, I'll experiment with altq and multi-wan shaping and update the wiki with my findings and ideas. Off the bat though I'm not sure if this can be done without modifying altq itself. Also I'll experiment with the ideas you currently have to see if I can add any additional info. What about transparent/l7 shaping? have any ideas or wiki entry on that? I have a few idea's I'd like to share on that, I probably make a wiki entry once I setup a testing platform this weekend and put together some notes.

Offline sullrich

  • Hero Member
  • *****
  • Posts: 5110
  • Karma: +3/-0
    • View Profile
    • pfSense
Re: Traffic shaper changes
« Reply #26 on: December 28, 2006, 03:01:47 pm »
cool, I'll experiment with altq and multi-wan shaping and update the wiki with my findings and ideas. Off the bat though I'm not sure if this can be done without modifying altq itself. Also I'll experiment with the ideas you currently have to see if I can add any additional info. What about transparent/l7 shaping? have any ideas or wiki entry on that? I have a few idea's I'd like to share on that, I probably make a wiki entry once I setup a testing platform this weekend and put together some notes.

Yay!  Glad to see you have some free time Leo!

Offline Leoandru

  • Full Member
  • ***
  • Posts: 111
  • Karma: +0/-0
    • View Profile
Re: Traffic shaper changes
« Reply #27 on: January 09, 2007, 10:33:03 am »
just a little update: The multiple interface shaping feature is starting to look a bit daunting, altq was not designed for it. The queuing hierarchy created on each interface are totally unrelated. So if you try to shape 1 wan interface over two lans then altq simple can't do it. Probably some combination of dummynet and altq would solve the problem, I'll post my opinions on the wiki later.

Offline billm

  • Administrator
  • Hero Member
  • *****
  • Posts: 731
  • Karma: +1/-0
    • View Profile
    • UCSecurity - Technology discovery and ramblings
Re: Traffic shaper changes
« Reply #28 on: January 10, 2007, 10:02:58 pm »
Thanks Leon...I'll check out the wiki, the configs apply, but I'm not terribly surprised it doesn't work quite as advertised :-/

--Bill
pfSense core developer
blog - http://www.ucsecurity.com/
twitter - billmarquette

Offline hoba

  • Administrator
  • Hero Member
  • *****
  • Posts: 5837
  • Karma: +3/-0
  • What was the problem to this solution again?
    • View Profile
    • pfSense
Re: Traffic shaper changes
« Reply #29 on: January 11, 2007, 01:19:07 am »
As dummynet can shape incoming on an interface this would be an option to shape traffic inside tunnels as well (before the traffic on the outgoing interface is only seen as encrypted traffic only). I have some setups that work this way pretty well with m0n0wall. However, getting this all under control and even crunching all that logic in a wizard will be a hard task I guess and considering multiple interfaces...