K2.6 Increase Maximum Connections ip_conntrack_max hashsize

Post new topic   Reply to topic    DD-WRT Forum Index -> Broadcom SoC based Hardware
Goto page 1, 2  Next
Author Message
ev1te
DD-WRT User


Joined: 03 Jan 2007
Posts: 76
Location: Sweden, Stockholm

PostPosted: Tue Jan 26, 2010 14:10    Post subject: K2.6 Increase Maximum Connections ip_conntrack_max hashsize Reply with quote
Here is a solution for increasing the maximum number of connections in Kernel 2.6:
K2.6 is very different regarding ip_conntrack_max (IP Filter Maximum Ports in the GUI) and it's hash table compared to the older kernel.

ip_conntrack_max sets the maximum number of connections that can be kept at one time. Most people here seems to decrease the default timeout (3600s) for these connections (which results in other problems) instead of increasing the maximum value.

On routers with large amount of RAM (32 MB for example) one could increase this by 10 times from the default max of 4096 without any problem.

From what I understand you also have to increase the hash table where these are stored in order to benefit from this increase. In the older kernel this was not possible since one would have to set this value before booting (and there was no nvram setting for that?).

Although in K2.6 one can change booth of these in realtime dynamically without restarting any process. A hashsize equaly large to the conntrack_max, has the best performance as I understand. Also since K2.4.21 the hashsize performs best with a value that is a power of 2 ex. 2^14 = 16384. (I have also used this as an example)


Since K2.4.23 (and newer), to change ip_conntrack_max:
    echo "16384" > /proc/sys/net/ipv4/netfilter/ip_conntrack_max

Since K2.6.20 (and newer), to change the hashsize:
    echo "16384" > /sys/module/nf_conntrack/parameters/hashsize

But none of these will be stored after a reboot, since the values are stored in RAM.


Permanently change ip_conntrack_max, ether use the webgui or write:
    nvram set ip_conntrack_max=16384
    nvram commit
    reboot

I haven't found any way to permanently change the hashsize parameter. Although since you now in K2.6 can change it in realtime, you can just add it to your startup script and it should do the job.
    echo "16384" > /sys/module/nf_conntrack/parameters/hashsize

Proof of it working: Very Happy


Further reading about this subject and also formulas to calculate the RAM usage of different settings:
http://www.wallfire.org/misc/netfilter_conntrack_perf.txt

During the test you see in the image (14-16k connections), the router used up some 5 MB of the free RAM (so make sure you have enough RAM). Also my computer almost crashed when it opened up 16k connections (froze for some seconds), could be a software problem though. Rolling Eyes

_________________
WRT320N


Last edited by ev1te on Sat Jan 30, 2010 13:04; edited 3 times in total
Sponsor
ev1te
DD-WRT User


Joined: 03 Jan 2007
Posts: 76
Location: Sweden, Stockholm

PostPosted: Sat Jan 30, 2010 12:59    Post subject: Reply with quote
I have seen several people before, that have had problems with the maximum number of connections becoming reached...

But there hasn't been any response to this solution that I came up with for the K2.6?

_________________
WRT320N
nolar
DD-WRT Novice


Joined: 31 Jan 2010
Posts: 6

PostPosted: Sat Feb 06, 2010 1:12    Post subject: Breaks the WebUI Reply with quote
Max Connections (ip_conntrack_max) limited to only 4k in the default firmware is a non-starter where I am, as we tend to have tens of thousands of connections at any given moment (we develop p2p software).

I'm not sure where the best place to report this is, but once you increase the ip_conntrack_max value it breaks the web ui in several places. I've attached a screenshot of this. I'm guessing it's something to do with the value being more than four digits...

I'm not sure why the limit is set to max 4096 by default, as that's extremely low, especially given the current crop of hardware, but we're not the only ones complaining: http://www.dd-wrt.com/phpBB2/viewtopic.php?t=63623

If max conns could be increased without breaking the webui, I'd switch to ddwrt for our office in a heartbeat.



rtn16.png
 Description:
 Filesize:  40.25 KB
 Viewed:  55422 Time(s)

rtn16.png


DHC_DarkShadow
DD-WRT Guru


Joined: 22 Jun 2008
Posts: 2440
Location: Am now Dark_Shadow

PostPosted: Sat Feb 06, 2010 2:39    Post subject: Reply with quote
Clear your browser cache and/or use another browser.
_________________
The New Me
nolar
DD-WRT Novice


Joined: 31 Jan 2010
Posts: 6

PostPosted: Sat Feb 06, 2010 2:59    Post subject: Reply with quote
Oh, it definitely has nothing to do with stale browser cache. Here's some captures on a totally different browser on a totally different os. The Active Clients list also never gets filled.

I can reproduce 100% reliably by starting with a virgin ddwrt image, and simply running

Code:

nvram set ip_conntrack_max=16384
nvram commit



missing_stats.png
 Description:
 Filesize:  37.56 KB
 Viewed:  55380 Time(s)

missing_stats.png



missing_clients.png
 Description:
 Filesize:  91.66 KB
 Viewed:  55380 Time(s)

missing_clients.png


ev1te
DD-WRT User


Joined: 03 Jan 2007
Posts: 76
Location: Sweden, Stockholm

PostPosted: Sun Feb 07, 2010 16:12    Post subject: Reply with quote
I am using 2.6 (you have 2.4) which might explain a few things.

From what i understand you can not only increase the ip_conntrack_max value in linux, you also have to increase the hash table for these values. Which is impossible to to in the 2.4 version kernel without rebuilding the firmware (but still possible in 2.6 since you can modify the value while the router is running).

I have been running 16384 connections max without any setbacks for a couple of weeks now.

_________________
WRT320N
phuzi0n
DD-WRT Guru


Joined: 10 Oct 2006
Posts: 10141

PostPosted: Sun Feb 07, 2010 16:35    Post subject: Reply with quote
The ability to dynamically adjust the hash size settings only applies to kernel 2.6.14 and higher. Kernel 2.4 builds can only adjust it as a compilation option so you would have to compile your own build to gain more than 4096 conntrack entries.
_________________
Read the forum announcements thoroughly! Be cautious if you're inexperienced.
Available for paid consulting. (Don't PM about complicated setups otherwise)
Looking for bricks and spare routers to expand my collection. (not interested in G spec models)
birnie
DD-WRT Novice


Joined: 21 Dec 2009
Posts: 26

PostPosted: Sun Feb 07, 2010 20:16    Post subject: Reply with quote
I have the same problem as Nolar, running DD-WRT v24-sp2 (01/02/10) big - build 13575M NEWD-2 K2.6 Eko.

RT-N16,

nvram set ip_conntrack_max=16384
nvram commit
reboot

Run: echo "16384" > /sys/module/nf_conntrack/parameters/hashsize

After reboot webui does not display active connections or memory information.

Thoughts?
ev1te
DD-WRT User


Joined: 03 Jan 2007
Posts: 76
Location: Sweden, Stockholm

PostPosted: Sun Feb 07, 2010 21:49    Post subject: Reply with quote
birnie wrote:
I have the same problem as Nolar, running DD-WRT v24-sp2 (01/02/10) big - build 13575M NEWD-2 K2.6 Eko.

RT-N16,

nvram set ip_conntrack_max=16384
nvram commit
reboot

Run: echo "16384" > /sys/module/nf_conntrack/parameters/hashsize

After reboot webui does not display active connections or memory information.

Thoughts?


If you read my first post, you will see that that it only works for the new Linux kernel (2.6.20 and forward), and you have DD-WRT v24-sp2 (which is 2.4.x something)

Look if the new firmware with the new kernel is available for your router. Otherwise I don't know if it is possible to fix it.

_________________
WRT320N
nolar
DD-WRT Novice


Joined: 31 Jan 2010
Posts: 6

PostPosted: Sun Feb 07, 2010 22:14    Post subject: Um... Reply with quote
Correct me if I'm wrong, but as I understand it, we ARE running kernel 2.6 builds, as the v24 refers to the ddwrt version and the K26 part refers to the kernel version: http://www.dd-wrt.com/wiki/index.php/What_is_DD-WRT%3F#File_Versions

Here's where I got my revision from: ftp://dd-wrt.com/others/eko/BrainSlayer-V24-preSP2/01-16-10-r13637/broadcom_K26/
birnie
DD-WRT Novice


Joined: 21 Dec 2009
Posts: 26

PostPosted: Sun Feb 07, 2010 22:39    Post subject: Reply with quote
That was my understanding as well Smile
phuzi0n
DD-WRT Guru


Joined: 10 Oct 2006
Posts: 10141

PostPosted: Sun Feb 07, 2010 22:40    Post subject: Reply with quote
Yeah, RT-N16 only supports k2.6 so there's no way you have k2.4. Iirc k2.6 explicitly say it in the version string on the status page but it seems to have been missing in the first screenshot for whatever reason.

It might be that the nvram variable + startup script is causing trouble because it's doing things in the wrong order. Try removing all of it, reboot, and then run these commands via telnet in the order below so that the hash size is increased before you try raising the conntrack limit.


echo "16384" > /sys/module/nf_conntrack/parameters/hashsize
echo "16384" > /proc/sys/net/ipv4/netfilter/ip_conntrack_max

_________________
Read the forum announcements thoroughly! Be cautious if you're inexperienced.
Available for paid consulting. (Don't PM about complicated setups otherwise)
Looking for bricks and spare routers to expand my collection. (not interested in G spec models)
ev1te
DD-WRT User


Joined: 03 Jan 2007
Posts: 76
Location: Sweden, Stockholm

PostPosted: Mon Feb 08, 2010 9:30    Post subject: Reply with quote
Sorry for the misunderstanding, I must have mixed up the v24 with K24. Wink

I am using a nvram value of 16384 for ip_conntrak_max.
My startup script begins a "sleep" in order for all the vital functions to start properly first:
    sleep 20
    echo "16384" > /sys/module/nf_conntrack/parameters/hashsize

_________________
WRT320N
nolar
DD-WRT Novice


Joined: 31 Jan 2010
Posts: 6

PostPosted: Tue Feb 09, 2010 0:13    Post subject: Reply with quote
OK, got it working, without the broken ui!

1) In order for the UI to pick up the max connection increase, one must use 'nvram set' (at least on the rt-n16), although I suspect that setting the proc value via 'echo' is enough to get things working behind the scenes.

2) It appears that issuing 'nvram commit' afterwards is what breaks things for me, so I instead saved the config via the gui save/apply settings/reboot router buttons.

I'm now running with hashsize and ip_conntrack_max both set to 65536, which I'll be testing this week with some real world load.
ev1te
DD-WRT User


Joined: 03 Jan 2007
Posts: 76
Location: Sweden, Stockholm

PostPosted: Wed Feb 10, 2010 20:28    Post subject: Reply with quote
nolar wrote:
OK, got it working, without the broken ui!

1) In order for the UI to pick up the max connection increase, one must use 'nvram set' (at least on the rt-n16), although I suspect that setting the proc value via 'echo' is enough to get things working behind the scenes.

2) It appears that issuing 'nvram commit' afterwards is what breaks things for me, so I instead saved the config via the gui save/apply settings/reboot router buttons.

I'm now running with hashsize and ip_conntrack_max both set to 65536, which I'll be testing this week with some real world load.


That seems weird, doesn't the gui use nvram commit to save values as well?

Also 65536 connections would use a lot of RAM, probably around 20 MB, so try and keep track of the RAM usage while you benchmark the settings.

_________________
WRT320N


Last edited by ev1te on Thu Feb 11, 2010 21:29; edited 1 time in total
Goto page 1, 2  Next Display posts from previous:    Page 1 of 2
Post new topic   Reply to topic    DD-WRT Forum Index -> Broadcom SoC based Hardware All times are GMT

Navigation

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You can attach files in this forum
You can download files in this forum