Lockup under heavy load

Post new topic   Reply to topic    DD-WRT Forum Index -> Atheros WiSOC based Hardware
Goto page 1, 2, 3  Next
Author Message
Tomasas
DD-WRT Novice


Joined: 26 Jun 2015
Posts: 15

PostPosted: Fri Jun 26, 2015 14:44    Post subject: Lockup under heavy load Reply with quote
Every time I try to download a popular torrent the router ends up freezing up, no auto-reboot or anything. I've tried messing around with Maximum Ports, TCP Timeouts.

Here is what I'm running:
Build: 27453 (06/25/15)
Router: Buffalo WHR-HP-G300N
Maximum ports: 2560 (doesn't seem to do much in terms of stability)
TCP timeout: 400
UDP timeout: 120
Web Access disabled. (to save some memory)

Now the weird part is that it crashes (at least according to top) even when there is still a lot of free memory.

Anyone have any idea what's going on? Can I find some error logs in the router?

Below I added the final top update for 3 tries with different Maximum port variables:

TRY 1:
Mem: 16372K used, 13052K free, 0K shrd, 1940K buff, 5640K cached
CPU: 0.0% usr 0.0% sys 0.0% nic 52.6% idle 0.0% io 0.0% irq 47.3% sirq
Load average: 0.00 0.02 0.04 1/30 1329
PID PPID USER STAT VSZ %VSZ CPU %CPU COMMAND
1225 1136 root R 1148 3.9 0 0.4 top
702 1 root S 1612 5.4 0 0.0 hostapd -B -P /var/run/ath0_hosta
514 1 root S 1524 5.1 0 0.0 watchdog
1125 1 root S 1504 5.1 0 0.0 wland
922 1 root S 1480 5.0 0 0.0 resetbutton
865 1 root S 1400 4.7 0 0.0 ttraff
1 0 root S 1284 4.3 0 0.0 /sbin/init
1119 1 root S 1280 4.3 0 0.0 process_monitor
747 1 root S 1148 3.9 0 0.0 telnetd
1136 747 root S 1144 3.8 0 0.0 -sh
1127 1 root S 1140 3.8 0 0.0 udhcpc -i eth0 -p /var/run/udhcpc
756 1 root S 1136 3.8 0 0.0 dnsmasq -u root -g root --conf-fi
470 1 root S 936 3.1 0 0.0 /sbin/hotplug2 --set-rules-file /
1126 1 root S 936 3.1 0 0.0 igmprt /tmp/igmpproxy.conf
1135 1 root S 912 3.0 0 0.0 cron
4 2 root SW 0 0.0 0 0.0 [kworker/0:0]
3 2 root SW 0 0.0 0 0.0 [ksoftirqd/0]
6 2 root SW 0 0.0 0 0.0 [kworker/u2:0]
98 2 root SW 0 0.0 0 0.0 [kworker/0:1]
70 2 root SW< 0 0.0 0 0.0 [crypto]
TRY 2:
Mem: 14488K used, 14936K free, 0K shrd, 1536K buff, 4164K cached
CPU: 0.0% usr 0.3% sys 0.0% nic 51.9% idle 0.0% io 0.0% irq 47.6% sirq
Load average: 0.05 0.03 0.05 1/30 1346
PID PPID USER STAT VSZ %VSZ CPU %CPU COMMAND
1142 1140 root R 1148 3.9 0 0.7 top
706 1 root S 1612 5.4 0 0.0 hostapd -B -P /var/run/ath0_hosta
515 1 root S 1524 5.1 0 0.0 watchdog
1128 1 root S 1504 5.1 0 0.0 wland
924 1 root S 1480 5.0 0 0.0 resetbutton
869 1 root S 1400 4.7 0 0.0 ttraff
1 0 root S 1284 4.3 0 0.0 /sbin/init
1123 1 root S 1280 4.3 0 0.0 process_monitor
751 1 root S 1148 3.9 0 0.0 telnetd
1140 751 root S 1144 3.8 0 0.0 -sh
1131 1 root S 1140 3.8 0 0.0 udhcpc -i eth0 -p /var/run/udhcpc
757 1 root S 1136 3.8 0 0.0 dnsmasq -u root -g root --conf-fi
483 1 root S 936 3.1 0 0.0 /sbin/hotplug2 --set-rules-file /
1130 1 root S 936 3.1 0 0.0 igmprt /tmp/igmpproxy.conf
1139 1 root S 912 3.0 0 0.0 cron
4 2 root SW 0 0.0 0 0.0 [kworker/0:0]
3 2 root SW 0 0.0 0 0.0 [ksoftirqd/0]
6 2 root SW 0 0.0 0 0.0 [kworker/u2:0]
105 2 root SW 0 0.0 0 0.0 [kswapd0]
70 2 root SW< 0 0.0 0 0.0 [crypto]
TRY 3:
Mem: 17512K used, 11912K free, 0K shrd, 1996K buff, 5944K cached
CPU: 0.1% usr 0.5% sys 0.0% nic 54.0% idle 0.0% io 0.0% irq 45.1% sirq
Load average: 0.02 0.05 0.05 1/30 3757
PID PPID USER STAT VSZ %VSZ CPU %CPU COMMAND
3705 3703 root R 1148 3.9 0 0.7 top
3294 1 root S 1148 3.9 0 0.2 telnetd
3257 1 root S 1612 5.4 0 0.0 hostapd -B -P /var/run/ath0_hosta
514 1 root S 1524 5.1 0 0.0 watchdog
3676 1 root S 1504 5.1 0 0.0 wland
3489 1 root S 1480 5.0 0 0.0 resetbutton
3418 1 root S 1400 4.7 0 0.0 ttraff
1 0 root S 1284 4.3 0 0.0 /sbin/init
3672 1 root S 1280 4.3 0 0.0 process_monitor
3703 3294 root S 1144 3.8 0 0.0 -sh
3680 1 root S 1140 3.8 0 0.0 udhcpc -i eth0 -p /var/run/udhcpc
3307 1 root S 1136 3.8 0 0.0 dnsmasq -u root -g root --conf-fi
479 1 root S 936 3.1 0 0.0 /sbin/hotplug2 --set-rules-file /
3679 1 root S 936 3.1 0 0.0 igmprt /tmp/igmpproxy.conf
3304 1 root S 912 3.0 0 0.0 cron
3 2 root SW 0 0.0 0 0.0 [ksoftirqd/0]
4 2 root SW 0 0.0 0 0.0 [kworker/0:0]
6 2 root SW 0 0.0 0 0.0 [kworker/u2:0]
8 2 root SW 0 0.0 0 0.0 [kworker/u2:1]
98 2 root SW 0 0.0 0 0.0 [kworker/0:1]
Sponsor
KrypteX
DD-WRT Guru


Joined: 18 Jan 2014
Posts: 2411

PostPosted: Fri Jun 26, 2015 14:54    Post subject: Reply with quote
Use the following:
Maximum ports: 16384
TCP timeout: 3600 (Default)
UDP timeout: 120 (Default)

If you still have issues, then flash back to 27396 or wait for the new build 27456+
Builds 27413-27453 have all kinds of issues, see the changelog in my signature for more details.

_________________
THERE ARE NO STRANGERS HERE; ONLY FRIENDS YOU HAVEN'T YET MET.
________________________________________________________________________________________________________
DD-WRT CHANGELOG | DEVICES | DD-WRT BUILDS | KONG BUILDS | UNOFFICIAL BUILDS | DD-WRT in VIRTUALBOX
Tomasas
DD-WRT Novice


Joined: 26 Jun 2015
Posts: 15

PostPosted: Fri Jun 26, 2015 15:56    Post subject: Reply with quote
KrypteX wrote:
Use the following:
Maximum ports: 16384
TCP timeout: 3600 (Default)
UDP timeout: 120 (Default)

If you still have issues, then flash back to 27396 or wait for the new build 27456+
Builds 27413-27453 have all kinds of issues, see the changelog in my signature for more details.


Tried your settings, but it still crashed. Flashed 27396 and it still ends up crashing BUT it took a lot longer, so it sure is more stable.
Though the TCP timeout of 3600 seems long, shouldn't around 400-460 be enough?
EDIT:
Oh and I flashed without resetting settings, should I try resetting them?
Also this run, top showed more or less the same:
Mem: 18252K used, 11164K free, 0K shrd, 1964K buff, 5868K cached
CPU: 0.0% usr 0.3% sys 0.0% nic 54.7% idle 0.0% io 0.0% irq 44.8% sirq
Mile-Lile
DD-WRT Guru


Joined: 24 Feb 2013
Posts: 1634
Location: Belgrade

PostPosted: Fri Jun 26, 2015 16:48    Post subject: Reply with quote
Try to disable UPnP... I had big problems in past with ddwrt UPnP ... I noticed that UPnP is not needed anymore for running torrents...
Tomasas
DD-WRT Novice


Joined: 26 Jun 2015
Posts: 15

PostPosted: Fri Jun 26, 2015 16:52    Post subject: Reply with quote
Mile-Lile wrote:
Try to disable UPnP... I had big problems in past with ddwrt UPnP ... I noticed that UPnP is not needed anymore for running torrents...

I don't use it either, so it isn't enabled. QoS is also disabled, I only use 1 port range forward.
tatsuya46
DD-WRT Guru


Joined: 03 Jan 2010
Posts: 7568
Location: YWG, Canada

PostPosted: Fri Jun 26, 2015 17:10    Post subject: Reply with quote
watch the free ram as it freezes till it freezes, does it run out? copy/paste into a txt file the output from "dmesg" command as it starts freezing again
_________________
LATEST FIRMWARE(S)

BrainSlayer wrote:
we just do it since we do not like any restrictions enforced by stupid cocaine snorting managers

[x86_64] Haswell i3-4150/QCA9984/QCA9882 ------> r55797 std
[QUALCOMM] DIR-862L --------------------------------> r55797 std
▲ ACTIVE / INACTIVE ▼
[QUALCOMM] WNDR4300 v1 --------------------------> r50485 std
[BROADCOM] DIR-860L A1 ----------------------------> r50485 std


Sigh.. why do i exist anyway.. | I love you Anthony.. never forget that.. my other 99% that ill never see again..

Tomasas
DD-WRT Novice


Joined: 26 Jun 2015
Posts: 15

PostPosted: Fri Jun 26, 2015 17:28    Post subject: Reply with quote
tatsuya46 wrote:
watch the free ram as it freezes till it freezes, does it run out? copy/paste into a txt file the output from "dmesg" command as it starts freezing again

According to the last top update before it locks up there is still free memory (as noted at the bottom of my Original Post), so I don't think it's an issue with memory running out, unless top isn't accurate.
tatsuya46
DD-WRT Guru


Joined: 03 Jan 2010
Posts: 7568
Location: YWG, Canada

PostPosted: Fri Jun 26, 2015 17:58    Post subject: Reply with quote
Tomasas wrote:
tatsuya46 wrote:
watch the free ram as it freezes till it freezes, does it run out? copy/paste into a txt file the output from "dmesg" command as it starts freezing again

According to the last top update before it locks up there is still free memory (as noted at the bottom of my Original Post), so I don't think it's an issue with memory running out, unless top isn't accurate.


i saw it but i didnt know how close to the freeze they were observed at

dmesg output?

_________________
LATEST FIRMWARE(S)

BrainSlayer wrote:
we just do it since we do not like any restrictions enforced by stupid cocaine snorting managers

[x86_64] Haswell i3-4150/QCA9984/QCA9882 ------> r55797 std
[QUALCOMM] DIR-862L --------------------------------> r55797 std
▲ ACTIVE / INACTIVE ▼
[QUALCOMM] WNDR4300 v1 --------------------------> r50485 std
[BROADCOM] DIR-860L A1 ----------------------------> r50485 std


Sigh.. why do i exist anyway.. | I love you Anthony.. never forget that.. my other 99% that ill never see again..

Tomasas
DD-WRT Novice


Joined: 26 Jun 2015
Posts: 15

PostPosted: Fri Jun 26, 2015 19:08    Post subject: Reply with quote
tatsuya46 wrote:

dmesg output?

Last dmesg calls visible in terminal (Called every second):
http://pastebin.com/JNnQWWpS (didn't fit in the post)
Shryp
DD-WRT User


Joined: 22 Jun 2008
Posts: 138
Location: Cleveland, Ohio USA

PostPosted: Fri Jun 26, 2015 23:41    Post subject: Reply with quote
It could be your max ports get filled. If you disable your torrents and then wait a while does it free up? I used to have that problem on an old WRT54Gv5 running micro.

Once the max ports get full nothing else can connect to the router so you can't even log into the web ui. After some of them time out and free up you can open new ones.
tatsuya46
DD-WRT Guru


Joined: 03 Jan 2010
Posts: 7568
Location: YWG, Canada

PostPosted: Fri Jun 26, 2015 23:53    Post subject: Reply with quote
when was dmesg run? just before it froze or during its freeze? or a random guess & it froze later (minutes or hours)?
_________________
LATEST FIRMWARE(S)

BrainSlayer wrote:
we just do it since we do not like any restrictions enforced by stupid cocaine snorting managers

[x86_64] Haswell i3-4150/QCA9984/QCA9882 ------> r55797 std
[QUALCOMM] DIR-862L --------------------------------> r55797 std
▲ ACTIVE / INACTIVE ▼
[QUALCOMM] WNDR4300 v1 --------------------------> r50485 std
[BROADCOM] DIR-860L A1 ----------------------------> r50485 std


Sigh.. why do i exist anyway.. | I love you Anthony.. never forget that.. my other 99% that ill never see again..

KrypteX
DD-WRT Guru


Joined: 18 Jan 2014
Posts: 2411

PostPosted: Sat Jun 27, 2015 5:36    Post subject: Reply with quote
Shryp wrote:
It could be your max ports get filled.

This might happen, but if you've set to 16384 max ports, it should suffice.
However, I've seen the freezing on my routers with uTorrent 2.2.x (older, but a lot of people still use it) even with max ports set to 32768.
A single torrent is enough to completely fill the ports in 5-10 minutes and crash the router.
If that's the case, upgrade to a newer torrent client software.

You can check the number of TCP & UDP connections at any given time (before the crash of course) by running commands:
grep -c ^tcp /proc/net/ip_conntrack
or
grep -c ^udp /proc/net/ip_conntrack

They should give you the number of active TCP and UDP connections, respectively. I'd check several times with them, while the router is still functional. With uTorrent 2.2.1 I see continuous increase in the UDP connections until the router freezes (it takes ~10-20 minutes).

_________________
THERE ARE NO STRANGERS HERE; ONLY FRIENDS YOU HAVEN'T YET MET.
________________________________________________________________________________________________________
DD-WRT CHANGELOG | DEVICES | DD-WRT BUILDS | KONG BUILDS | UNOFFICIAL BUILDS | DD-WRT in VIRTUALBOX
Tomasas
DD-WRT Novice


Joined: 26 Jun 2015
Posts: 15

PostPosted: Sat Jun 27, 2015 8:32    Post subject: Reply with quote
tatsuya46 wrote:
when was dmesg run? just before it froze or during its freeze? or a random guess & it froze later (minutes or hours)?

It was run every 1 second. I used a while true loop with a sleep of 1 to execute it. The pastebin I posted is the last outputs I could see in my terminal.

As for max ports filling up, I don't think that it reaches it. I'm using transmissionQT(for reference) and it ends up crashing near 2k connections according to the web GUI.
Shryp
DD-WRT User


Joined: 22 Jun 2008
Posts: 138
Location: Cleveland, Ohio USA

PostPosted: Sat Jun 27, 2015 8:32    Post subject: Reply with quote
Oh yes, speaking of torrents, there are configurations you can do to your client to help. Disabling DHT if you can get by without it will help as it uses a lot of UDP connections.

Limiting your connections will also help. Most clients have limits globally, per torrent and attempts per second.
tatsuya46
DD-WRT Guru


Joined: 03 Jan 2010
Posts: 7568
Location: YWG, Canada

PostPosted: Sat Jun 27, 2015 9:25    Post subject: Reply with quote
Tomasas wrote:
tatsuya46 wrote:
when was dmesg run? just before it froze or during its freeze? or a random guess & it froze later (minutes or hours)?

It was run every 1 second. I used a while true loop with a sleep of 1 to execute it. The pastebin I posted is the last outputs I could see in my terminal.

As for max ports filling up, I don't think that it reaches it. I'm using transmissionQT(for reference) and it ends up crashing near 2k connections according to the web GUI.


not connections getting filled or itll say nf_conntrack_full repeatadly in dmesg

_________________
LATEST FIRMWARE(S)

BrainSlayer wrote:
we just do it since we do not like any restrictions enforced by stupid cocaine snorting managers

[x86_64] Haswell i3-4150/QCA9984/QCA9882 ------> r55797 std
[QUALCOMM] DIR-862L --------------------------------> r55797 std
▲ ACTIVE / INACTIVE ▼
[QUALCOMM] WNDR4300 v1 --------------------------> r50485 std
[BROADCOM] DIR-860L A1 ----------------------------> r50485 std


Sigh.. why do i exist anyway.. | I love you Anthony.. never forget that.. my other 99% that ill never see again..

Goto page 1, 2, 3  Next Display posts from previous:    Page 1 of 3
Post new topic   Reply to topic    DD-WRT Forum Index -> Atheros WiSOC based Hardware All times are GMT

Navigation

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You cannot download files in this forum