My "custom" QoS that really works

Post new topic   Reply to topic    DD-WRT Forum Forum Index -> Contributions Upload
Goto page 1, 2, 3, 4  Next
Author Message
CaScAdE
DD-WRT Guru


Joined: 18 Jun 2006
Posts: 1110
Location: Kiel (54.4247,10.1721)

PostPosted: Tue Aug 15, 2006 7:03    Post subject: My "custom" QoS that really works Reply with quote
Since many people are having problems with QoS, myself included, and I have been asked for my >custom< sulution which works I decided to put it on here for people who want to test it.

I attached it gzipped to make sure the forum does not mess it up with newlines or whatnot...

It works for me for about tweo months now, gaming and voip without any disturbing laggs during 85k/s uploads, accesses to my webserver, fast downloads and whatnot.
Do not forget that you have to tune it according to your linespeeds, QoS is nearly NEVER a out of box sulution. Read out your modem to see you connection stats!
And as important as the linespeeds is the modem you are using, had really big problems like dropping DSL-Connections, modem timeouts and whatnot with the supplied "Arcor Speed Modem 200" and now I have a "SpeedTouch 536i v5" I have a DSL-Uptime (not PPPoE because of 24h disconnects) of 14Days (there were some powerlosses before when moving the modem to its final destination).
Problems with your DSL-Connection? Try http://forum.mhilfe.de/ (german)

I am hoping this will help some people :D

Note: I am using jffs and placed following files:
/jffs/etc/shaper.sh - the script itself
/jffs/etc/config/shaper.ipup
#!/bin/sh

/jffs/etc/shaper.sh start

/jffs/etc/config/shaper.ipdown
#!/bin/sh

/jffs/etc/shaper.sh stop




shaper.sh
Code:
#!/bin/sh

TC=/usr/sbin/tc
IPTABLES=/usr/sbin/iptables


# PPPoE, DSL16000, 18140/925 sync at my modem, NEVER expect to have for exeample 6000 when you ordered DSL6000
# RATE should be somewhat about 85% of you upstream, this works well for me, do not complain about "losing" 15%
DEV=ppp0
DOWNLINK=17800
RATE=760

# we want just the status when we call ./shaper.sh status
if [ "$1" = "status" ]
then
       $TC -s qdisc ls dev $DEV
       $TC -s class ls dev $DEV
       exit
fi

# clean existing down- and uplink qdiscs, hide errors
$TC qdisc del dev $DEV root    2> /dev/null > /dev/null
$TC qdisc del dev $DEV ingress 2> /dev/null > /dev/null

# kill the qdiscs to disable QoS (well iptables will still mark packets but there are no handles for them)
if [ "$1" = "stop" ]
then
        exit
fi

###### marking ######

## flush the handler
$IPTABLES -t mangle --flush

# rtp/voice get mark 3
$IPTABLES -t mangle -A POSTROUTING -p udp -m length --length 150:250 -j MARK --set-mark 3
$IPTABLES -t mangle -A POSTROUTING -p udp --source-port 11000:11009 -j MARK --set-mark 3
$IPTABLES -t mangle -A POSTROUTING -m layer7 --l7proto sip -j MARK --set-mark 3
$IPTABLES -t mangle -A POSTROUTING -m layer7 --l7proto teamspeak --set-mark 3

# gaming gets mark 3 well
$IPTABLES -t mangle -A POSTROUTING -m layer7 --l7proto quake-halflife -j MARK --set-mark 3
$IPTABLES -t mangle -A POSTROUTING -m layer7 --l7proto counterstrike-source -j MARK --set-mark 3

# p2p + ftp gets mark 6
$IPTABLES -t mangle -A POSTROUTING -m layer7 --l7proto edonkey -j MARK --set-mark 6
$IPTABLES -t mangle -A POSTROUTING -m layer7 --l7proto gnutella -j MARK --set-mark 6
$IPTABLES -t mangle -A POSTROUTING -m layer7 --l7proto bittorrent -j MARK --set-mark 6
$IPTABLES -t mangle -A POSTROUTING -m layer7 --l7proto ftp -j MARK --set-mark 6

# for shaping (modem queue never gets full)
# this is VERY important, would ruin all our work
$TC qdisc add dev ${DEV} root handle 1: tbf rate ${RATE}kbit burst 4k latency 30ms

# priorization 1: voip/gaming | 2: default | 3: p2p
$TC qdisc add dev ${DEV} parent 1: handle 10: prio bands 3 priomap 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
$TC qdisc add dev ${DEV} parent 10:1 handle 100: pfifo
$TC qdisc add dev ${DEV} parent 10:2 handle 200: pfifo limit 200
$TC qdisc add dev ${DEV} parent 10:3 handle 300: pfifo limit 700

#-------------------------------------
# filtering
#-------------------------------------
# 10:1 voip
$TC filter add dev ${DEV} parent 10:0 protocol ip prio 10 \
   handle 0x3 fw flowid  10:1


# 10:3 p2p
$TC filter add dev ${DEV} parent 10:0 protocol ip prio 20 \
        handle 0x6 fw flowid  10:3


# default: All non VoIP traffic on band 2
$TC filter add dev ${DEV} parent 10:0 prio 30 protocol ip u32 \
    match ip src 0.0.0.0/0 \
    flowid 10:2

########## downlink #############
# slow downloads down to somewhat less than the real speed  to prevent
# queuing at our ISP. Tune to see how high you can set it.
# ISPs tend to have *huge* queues to make sure big downloads are fast
#
# attach ingress policer:

$TC qdisc add dev $DEV handle ffff: ingress

# filter *everything* to it (0.0.0.0/0), drop everything that's
# coming in too fast:

$TC filter add dev $DEV parent ffff: protocol ip prio 50 u32 match ip src \
   0.0.0.0/0 police rate ${DOWNLINK}kbit burst 10k drop flowid :1

_________________
DD-WRT F.A.Q.

Webcam Kiel


Last edited by CaScAdE on Sun Nov 12, 2006 14:59; edited 4 times in total
Sponsor




Joined: 01 Jan 1970
Posts:

PostPosted: Tue Aug 15, 2006 7:11    Post subject: Reply with quote
Thanks for this contribution. I think QOS is indeed totally broken in DD-WRT, but I will try to mention that again.

For those writing their own QoS scripts, may I make a suggestion I've made before...

Instead of using these inefficient and sometimes unreliable L7 filters for matching p2p content, make all high numbered ports (above 1024, or above 3072, whatever..) default to a bulk rate priority and then add any exceptions necessary. This acts as a good catch-all for known and unknown p2p traffic since it almost always occurs on high port numbers.

EDIT: Noticed a minor type in your script's comments:

"### p2p + ftp get mark 3" -> "### p2p + ftp get mark 6"

I'm no iptables and tc guru, so I'm going to take your script and tweak it in accordance with the behavior I desire above. I'll post the result here.
CaScAdE
DD-WRT Guru


Joined: 18 Jun 2006
Posts: 1110
Location: Kiel (54.4247,10.1721)

PostPosted: Tue Aug 15, 2006 7:16    Post subject: Reply with quote
db90h wrote:
L7 filters for matching p2p content, make all high numbered ports (above 1024, or above 3072, whatever..) default to a bulk rate priority and then add any exceptions necessary. This acts as a good catch-all for known and unknown p2p traffic since it almost always occurs on high port numbers.


Thanx for the advice but that came to my mind as well but I have so many custom ports open for once and a while using whatnot and the script above just works for me without jamming my Router, so I did not think of tweaking it...
Maybe if I ever get a faster connection I will try...

db90h wrote:
EDIT: Noticed a minor type in your script's comments:

"### p2p + ftp get mark 3" -> "### p2p + ftp get mark 6"

I'm no iptables and tc guru, so I'm going to take your script and tweak it in accordance with the behavior I desire above. I'll post the result here.


Hehe, damn me, I just added those comments to make it easier to understand Wink
But corrected it, thanx...
And let us see what you made better than me after you have finished it *g*

_________________
DD-WRT F.A.Q.

Webcam Kiel


Last edited by CaScAdE on Tue Aug 15, 2006 7:20; edited 1 time in total




Joined: 01 Jan 1970
Posts:

PostPosted: Tue Aug 15, 2006 7:18    Post subject: Reply with quote
Also, anyone given any thoughts to porting nbd's OpenWrt QoS scripts? They are awesome in their ease of configuration (they use a configuration file of stanzas). Not sure how much work would be involved to port it.




Joined: 01 Jan 1970
Posts:

PostPosted: Tue Aug 15, 2006 7:20    Post subject: Reply with quote
CaScAdE wrote:
Thanx for the advice but that came to my mind as well but I have so many custom ports open for once and a while using whatnot and the script above just works for me without jamming my Router, so I did not think of tweaking it...
Maybe if I ever get a faster connection I will try...


Yea, as you said, QoS is hardly ever an out-of-the-box endeavor .. everybody needs a different config.

I wish my connection was as fast as yours is currently... ;p.

Does anyone know what the fuck is wrong with DD-WRT's QOS? I see a series of these "iptables: No chain/target/match by that name", but not sure of their relation to its problems. I meant to look into it sometime, but I have so many bugs of my own to fix ;p.




Joined: 01 Jan 1970
Posts:

PostPosted: Tue Aug 15, 2006 7:23    Post subject: Reply with quote
CaScAdE wrote:

And let us see what you made better than me after you have finished it *g*


What, you think my changes would make it better?!? That is hoping for too much Wink. Now that this requirement has been stated, I must refrain from editing as I will surely only make it worse ;p.
TexHex
DD-WRT User


Joined: 28 Jul 2006
Posts: 93

PostPosted: Tue Aug 15, 2006 11:40    Post subject: Reply with quote
thx for posting CaScAdE,
db90h i don't think it's a good way to deal with all ports over 3072 is p2p, many games and also voip software like skype or teamspeak are using ports over it.




Joined: 01 Jan 1970
Posts:

PostPosted: Tue Aug 15, 2006 12:03    Post subject: Reply with quote
TexHex wrote:

db90h i don't think it's a good way to deal with all ports over 3072 is p2p, many games and also voip software like skype or teamspeak are using ports over it.


Yea, you'd have to add exclusions for any practical use. For me, who doesn't use VOIP or games, I wouldn't have but a handful of exclusions.
cyberde
DD-WRT Guru


Joined: 07 Jun 2006
Posts: 1488
Location: the Netherlands

PostPosted: Tue Aug 15, 2006 19:30    Post subject: Reply with quote
db90h wrote:
Thanks for this contribution. I think QOS is indeed totally broken in DD-WRT, but I will try to mention that again.


What do you mean? It seems to work fine on my router. If you need any outputs please let me know.

NNTP gets thottled when downloading with http, and my ping is still good within Counter-Strike when downloading at full speed with NNTP.
But there's something weird; NNTP download also gets throttled to 0kb/s when I'm uploading at full speed. So that's the weirdest part I notice.

These are my settings:

_________________
Firmware: DD-WRT v24-sp2 (latest available) mega
WRT320N

Donater




Joined: 01 Jan 1970
Posts:

PostPosted: Thu Aug 17, 2006 1:56    Post subject: Reply with quote
cyberde wrote:
What do you mean? It seems to work fine on my router. If you need any outputs please let me know.


Maybe it's working for you, or maybe you are imagining it .. it doesn't work for me, I can assure you of that. I suppose it's possible it's due to my particular config, but I suspect it's just broken.

I am going to try to port Nbd's QOS scripts over to DD-WRT, it should hopefully be no big deal. This, I think, the ultimate QOS scripts. Integrating them with the UI is another pain in the ass matter all-together.
cyberde
DD-WRT Guru


Joined: 07 Jun 2006
Posts: 1488
Location: the Netherlands

PostPosted: Thu Aug 17, 2006 9:38    Post subject: Reply with quote
That might me possible too, but there are some things I still find weird. But it does give some services priority over other services, so that's why I think it works Smile
_________________
Firmware: DD-WRT v24-sp2 (latest available) mega
WRT320N

Donater
lawnmowerguy1
Donator


Joined: 06 Jun 2006
Posts: 163

PostPosted: Sun Aug 20, 2006 7:25    Post subject: Reply with quote
I cannot comment on the current status of qos, but I never liked the DD-WRT implementation

nbd's qos scripts are the best, if we could get those into the gui it would be fantastic

_________________
marcosscriven
DD-WRT Novice


Joined: 25 Jul 2006
Posts: 43

PostPosted: Wed Aug 23, 2006 14:40    Post subject: Reply with quote
Sorry for the newbie question - just searched on Google for 'DD-WRT QoS', and found this thread. Found the official documentation for it, but still can't really get the working. The thing I want to do is this:

I download movies from a newsgroup server, over NNTP, with a program called Grabbit. I allow it to make 15 connections, which mayes out my connection at 1710 KB/s or 13680kbps (official rate 16mbps) . I haven't measured my upload, but the official rate is 1024 (so I'm guessing around 800-850 kbps)

While that's going on, I want all other traffic to 'just work'. Skype, for instance, is really bad when I'm doing a Grabbit NNTP download. So, SSH, Telnet, Skype etc, I want to be responsive when I'm using them.

At all other times I want applications to use the full bandwidth if they need it and noone else does - I see it as pointless limiting otherwise.

I see the script above - but I don't really know how to deploy it. I tried just enabling QoS on the router, but Grabbit drops to around 350kbps! I can SSH in ok, but there is no /etc dir in the /jffs/ dir as mentioned above...

Any help appreciated! I just want a point and click, plug it in solution. Please :)

Marcos
CaScAdE
DD-WRT Guru


Joined: 18 Jun 2006
Posts: 1110
Location: Kiel (54.4247,10.1721)

PostPosted: Wed Aug 23, 2006 16:26    Post subject: Reply with quote
Did you read: http://www.dd-wrt.com/wiki/index.php/Jffs ?

You need a device with >4MB flash or the mini version when havin =4MB flash. Otherwise there is not space left on the jffs partition.

And if you jamm you downstream this script won't help that mouch cause whatever is in the queue at your ISP is there and you cannot handle the ISP's side of the line. This script just does what you can do (afair), dropping the rate to enable smaller packages to get through.

As I said, it works for me (and for my brother, who is using native linux and not embedded, as well). Since many others have problems with built in QoS I posted it.

In addition we could rewrite the script so it fits in the "Save Firewall" box, but there the size is limited and so I did not try it.

But for first tests:
Disable you QoS in the web interface.
SSH into your router.
put the script in /tmp and execute it.
( vi, chmod +x, ./foo.sh )
Will still works, no matter where you place it.

_________________
DD-WRT F.A.Q.

Webcam Kiel
speed_phreak
DD-WRT Novice


Joined: 06 Nov 2006
Posts: 3

PostPosted: Mon Nov 06, 2006 0:49    Post subject: errors on script execution Reply with quote
wrt54gs v3.0
dd-wrt v23 sp2


/jffs/etc # ./shaper.sh start
RTNETLINK answers: No such file or directory
RTNETLINK answers: No such file or directory
RTNETLINK answers: No such file or directory
RTNETLINK answers: No such file or directory

what is this trying to tell me? should i be concerned?
Goto page 1, 2, 3, 4  Next Display posts from previous:    Page 1 of 4
Post new topic   Reply to topic    DD-WRT Forum Forum Index -> Contributions Upload All times are GMT

Navigation

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You cannot download files in this forum