Based on my experience with the QoS settings, QoS is broken in both v24 sp1 and v24 sp2. It actually appears to be worse in v24 sp2. I am using the http://www.whichvoip.com/voip/speed_test/ppspeed.html to test the results of my QoS settings, and doing real tests with my VOIP device while loading my Internet connection. Even when I enable a single IP based rule under QoS I experience high packet loss and terrible jitter. Something isn't right. [...] Even when I'm using only 40kbps of upstream traffic my DD-WRT router's CPU will go to 100%.
Thanks for that test link. I used it to perform a few tests myself. You might be interested in the results, but first, here's a recap of my setup:
- connection rated at 1.5Mbps/384Kbps
- actual tested rate of 1232/320
- QoS Enabled in the GUI
- Port using default WAN setting
- Packet Scheduler using default HTB setting
- actual tested rates entered in Uplink and Downlink fields
- Optimize for Gaming using default unchecked setting
- MAC Priority entry for "work" PC using Express Priority
- Ethernet Port Priority for all ports using the default Premium Priority
- "work" PC and "play" PC both plugged directly into router
- no QoS scripts or explicit nvram settings
So with the settings noted above I ran two sets of several tests each on both the "work" (MAC-prioritized) PC and the "play" PC (no particular priority). The first set of tests was run non-concurrently to get a clean baseline for each PC, while the second set was run concurrently (starting within a second of each other), to see how QoS settings translate into real-world performance. I won't bore you with the detailed results except for the cases where the test results were not Green...
For the non-concurrent tests, results for the non-prioritized PC were Green across the board. Results for the MAC-prioritized PC were also Green across the board, with the exception of the QoS results, which were usually Red and were typically in the 50% range. While these results may seem counter-intuitive, they do accord with the fact that the MAC priority for that PC is only set to Express, not Premium (more on this issue later).
For the concurrent tests, results for both PCs were Green across the board except for Upload Speed and QoS. In this case the non-prioritized PC typically scored Yellow for Upload Speed and always scored Red for QoS, typically in the 30% range. The MAC-prioritized PC only occasionally scored Yellow for Upload Speed and also consistently scored Red for QoS, but that score seemed to hold its ground in the 50% range.
BTW, I also had a telnet connection to the router while performing all of these tests, and top (with the default polling interval) typically showed 5 to 8% CPU utilization, which seems perfectly acceptable.
FYI, after noting that the MAC-prioritized QoS rating was only 50%, I tried resetting that priority to Premium and repeating the tests. Non-concurrent testing then initially showed a Green QoS rating for the MAC-prioritized PC, but after repeated testing that rating started fluctuating wildly and ultimately ended up consistently Red. Initial concurrent testing also gave the prioritized PC a Green QoS rating, but the non-prioritized PC got a Red QoS rating of 0% (again, I think this is expected behavior based on the definition of Premium priority). At that point I decided that it wouldn't be practical to keep the MAC-prioritized PC on Premium priority, so I reverted it back to Express and stopped my testing.
My interpretation of the results:
QoS is definitely doing something in the latest builds of DD-WRT (I've been using it since 12220 and currently have 12262 installed). It seems to work correctly if you use the default settings (WAN, HTB, and no Gaming optimization), allocate enough "slack" in the Uplink/Downlink settings, and follow the Help guidelines of using Express priority for devices (i.e. MACs, and probably IP addresses or ports) that you want to have priority. I can't say if it works correctly when targeting protocols using the "Layer 7" filters built-in to DD-WRT. It also seems that giving Premium priority to any particular object (e.g. VoIP streams) may actually be counter-productive (algorithm contention?). And it doesn't appear that a "low" QoS score on the VoIP test page means anything by itself (i.e. when the test was only taken with no other devices concurrently running the test).
:
Edited to correct "initially ended up consistently Red" to instead say "ultimately ended up consistently Red". Also edited to add this editing note.
.
Last edited by SNR on Sun Jun 07, 2009 7:13; edited 2 times in total
SNR,
Thanks for posting these great results. Tomorrow I will have a chance to do a similar test with my setup which has an older revision of DD-WRT than yours. I will post what I can.
I will report that at this moment I am using a solo connection to a DSL modem (740kbps dn/384kbs up). I also get a red (30%) QoS even when I am the only one using the connection. So its unclear to me what this QoS rating means.
So it seems that in your case at least we do not have the 100% CPU utilization reported by scottlindner.
Lots of ideas come to mind. For example in your setup what would happen if you turned off QoS altogether? What would happen if you use HFSC instead of HTB? What would happened if one machine was uploading a large file while the other machine was doing the test?
SNR,
Thanks for posting these great results. Tomorrow I will have a chance to do a similar test with my setup which has an older revision of DD-WRT than yours. I will post what I can.
I will report that at this moment I am using a solo connection to a DSL modem (740kbps dn/384kbs up). I also get a red (30%) QoS even when I am the only one using the connection. So its unclear to me what this QoS rating means.
So it seems that in your case at least we do not have the 100% CPU utilization reported by scottlindner.
Lots of ideas come to mind. For example in your setup what would happen if you turned off QoS altogether? What would happen if you use HFSC instead of HTB? What would happened if one machine was uploading a large file while the other machine was doing the test?
If you dig into the details you'll see high packet losses cause that QoS rating to be very low.
I have turned off the QoS button in DD-WRT and the results on that page were very good.
If you dig into the details you'll see high packet losses cause that QoS rating to be very low.
Do you mean on the advanced tab? In my case at least my packet loss was 0.0% but I still get a red (36%) QoS. My results follow:
VoIP test statistics
--------------------
Jitter: you --> server: 0.6 ms
Jitter: server --> you: 4.4 ms
Packet loss: you --> server: 0.0 %
Packet loss: server --> you: 0.0 %
Packet discards: 0.0 %
Packets out of order: 0.0 %
Estimated MOS score: 4.0
Is there a place to learn more about what this QoS rating means?
As SNR pointed out, the HFSC problem is explained and fixed in my ticket. To put it simply, the HFSC script sets both upload and download rates according to whatever you enter for your download rate.
For all QoS the problem is a bit more complicated but I'll try to explain. Originally there was only simple ingress (incoming) traffic filtering in linux but then a wonderful hack of a method was developed by created a fake device that linux's robust egress (outgoing) filters could be applied to for filtering ingress traffic. These devices are called imq# but in order for the hack to work you also have to have iptables rules that send packets to the fake device where they'll then be put in the tc queues that you've set up for the imq device.
The bold output below shows that even though QoS is set to use the WAN port, a rule that sends all traffic coming in from the LAN side gets sent to imq0. This traffic won't have any mark at all since it gets sent to imq0 in the very first rule of the PREROUTING chain and so it will be put in the default tc queue (standard rate) for ingress traffic even though it is really egress. This same rule causes all your lan traffic that isn't handled by the internal switch (wlan-lan or usb drive attached to the router itself) to be put in your ingress queues.
Chain POSTROUTING (policy ACCEPT 2346 packets, 1030K bytes)
pkts bytes target prot opt in out source destination
58 6547 SVQOS_OUT 0 -- * eth1 0.0.0.0/0 0.0.0.0/0 _________________ Read the forum announcements thoroughly! Be cautious if you're inexperienced.
Available for paid consulting. (Don't PM about complicated setups otherwise)
Looking for bricks and spare routers to expand my collection. (not interested in G spec models)
I will report that at this moment I am using a solo connection to a DSL modem (740kbps dn/384kbs up). I also get a red (30%) QoS even when I am the only one using the connection. So its unclear to me what this QoS rating means.
I really don't think it means much by itself. What's more important is how it behaves with other concurrent connections.
pbix wrote:
So it seems that in your case at least we do not have the 100% CPU utilization reported by scottlindner.
My understanding (from another thread) is that Scott is seeing this high CPU utilization when actually placing VoIP calls. All I've done with these tests is (supposedly) see how well VoIP should work for me (but I can't test actual VoIP usage, since I don't use such a service).
pbix wrote:
Lots of ideas come to mind. For example in your setup what would happen if you turned off QoS altogether? What would happen if you use HFSC instead of HTB? What would happened if one machine was uploading a large file while the other machine was doing the test?
As I noted earlier in this thread, actual testing proves to me that my MAC-prioritized PC really does get priority over other clients on my LAN. For my previous testing I simply ran concurrent line speed tests via DSLReports.com on both my prioritized and non-prioritized PC. These tests download a large MP3 format file and upload a smaller file. During these tests I saw very little increase in latency on the MAC-prioritized PC, and it was using about twice the download bandwidth of the non-prioritized PC. I've also noticed that after enabling QoS in this manner I've never had to ask my kids to knock off the YouTube while I'm working over VPN. :wink:
But my work requirements are mostly download intensive (screen scrapes from remote control programs), and my results don't necessarily correlate well with someone else's requirements, especially for something like VoIP service. And I probably won't try HFSC, since VoIP prioritization isn't a requirement of mine. Instead, I would think that VoIP users should give HTB a try instead.
Edited to replace "VPN" with "VoIP" in this last paragraph. It's getting late over here...
In my case at least my packet loss was 0.0% but I still get a red (36%) QoS.
Me too. Good stats all around, except for a Red QoS rating (which is what I expect for an "Express" priority). These particular stats are from my non-prioritized PC but my MAC-prioritized PC looks similar:
VoIP test statistics
--------------------
Jitter: you --> server: 0.1 ms
Jitter: server --> you: 3.6 ms
Packet loss: you --> server: 0.0 %
Packet loss: server --> you: 0.0 %
Packet discards: 0.0 %
Packets out of order: 0.0 %
Estimated MOS score: 4.0
The bold output below shows that even though QoS is set to use the WAN port, a rule that sends all traffic coming in from the LAN side gets sent to imq0. This traffic won't have any mark at all since it gets sent to imq0 in the very first rule of the PREROUTING chain and so it will be put in the default tc queue (standard rate) for ingress traffic even though it is really egress. This same rule causes all your lan traffic that isn't handled by the internal switch (wlan-lan or usb drive attached to the router itself) to be put in your ingress queues
I see the same rule in the mangle table PREROUTING chain for my setup:
I can't say that I really understand what I'm seeing here (I'm still learning iptables) but it's clear that this rule precedes my MAC priority rule. Yet it seems that the MAC rule is still having some sort of QoS effect in my setup. In my case the MAC-prioritized device is plugged directly into the router, i.e. on LAN/vlan0, bridged via br0 to eth1/WLAN, so the first rule (which you are saying is erroneous) should apply to traffic from this device.
Can this be fixed with an iptables command(s) in a Firewall script?
I got back to my house and had a chance to run this test on my PC. At my house I get all greens.
VoIP test statistics
--------------------
Jitter: you --> server: 1.1 ms
Jitter: server --> you: 4.4 ms
Packet loss: you --> server: 0.0 %
Packet loss: server --> you: 0.0 %
Packet discards: 0.0 %
Packets out of order: 0.0 %
Estimated MOS score: 4.0
Speed test statistics
---------------------
Download speed: 1387544 bps
Upload speed: 260032 bps
Download quality of service: 99 %
Upload quality of service: 8 %
Download test type: socket
Upload test type: socket
Maximum TCP delay: 37 ms
Average download pause: 25 ms
Minimum round trip time to server: 92 ms
Average round trip time to server: 97 ms
Estimated download bandwidth: 1387544bps
Route concurrency: --
Download TCP forced idle: --
Maximum route speed: 5698688bps
Now its interesting that the "Upload Qos" is listed as only 8% above but the summary of test says green and 99% (the download QoS). This is with HFSC (WAN) or HTB (WAN) QoS enabled on my router.
If I turn off QoS entirely the QoS scores both become 99%.
Another interesting fact is that I am using v24pre-SP2 1099 which is circa Nov 2008. Can someone clue me into the location of the QOS script with the incorrect IPTABLES command? I can check this version and see if it has the same script error.
Last edited by pbix on Sun Jun 07, 2009 12:45; edited 1 time in total
So it seems that in your case at least we do not have the 100% CPU utilization reported by scottlindner.
My understanding (from another thread) is that Scott is seeing this high CPU utilization when actually placing VoIP calls. All I've done with these tests is (supposedly) see how well VoIP should work for me (but I can't test actual VoIP usage, since I don't use such a service).
That is correct. I apologize if I'm confusing things. This VOIP test site seems to give the best information regarding the quality of service of our Internet service, and since I'm using QoS on my router for VOIP, this is perfect for testing my situation.
My VOIP call is ~45kbps (it fluctuates between 38kbps and 48kbps) both upstream and downstream. Nothing radical at all when you consider the percentage of my total bandwidth available. I have read that DD-WRT can support up to 43Mbps throughput. I don't know if this is with the QoS settings in effect or not. Seeing my CPU utilization going to 100% on 45kbps traffic concerns me. Seeing my QoS scores on that test site being terrible when I enable QoS in DD-WRT with only one simple IP based rule setting the QoS to Premium really alarms me.
I experimented with another open source firmware that has far fewer features than DD-WRT and turned on the QoS. My scores are perfect every time on that test site. Everything else in my network is the same. Only difference is the firmware. So I do believe something is really wrong with DD-WRT's QoS implementation, but I don't know enough to help you guys figure it out.
Another interesting fact is that I am using v24pre-SP2 1099 which is circa Nov 2008. Can someone clue me into the location of the QOS script with the incorrect IPTABLES command? I can check this version and see if it has the same script error.
It has existed since the time wshaper.c was added to svn.
http://svn.dd-wrt.com:8000/dd-wrt/browser/src/router/services/networking/wshaper.c?rev=6627#L240 _________________ Read the forum announcements thoroughly! Be cautious if you're inexperienced.
Available for paid consulting. (Don't PM about complicated setups otherwise)
Looking for bricks and spare routers to expand my collection. (not interested in G spec models)
: