MEMORY: "privoxy" eating up memory on R7000

Post new topic   Reply to topic    DD-WRT Forum Index -> Broadcom SoC based Hardware
Author Message
rnio
DD-WRT User


Joined: 21 Apr 2012
Posts: 94

PostPosted: Tue Aug 22, 2017 23:09    Post subject: MEMORY: "privoxy" eating up memory on R7000 Reply with quote
New Kong's test build: DD-WRT 33010M - 2017/08/09

Router Model Netgear R7000
Firmware Version DD-WRT v3.0-r33010M kongac (08/09/17)
Kernel Version Linux 4.4.80 #422 SMP Wed Aug 9 20:13:10 CEST 2017 armv7l
RESET: Yes
SERVICES USED: USB / NAS (Samba) / Adblocking

Privoxy

Privoxy: Enable
Provide Proxy Autoconfig: Enable
Transparent Mode: Disable
Custom Configuration: Disable


Code:
S   UID   PID  PPID   VSZ   RSS TTY   STIME TIME     CMD

S     0  1091     1  155m 62192 0:0   Aug10 01:12:14 privoxy /tmp/privoxy.conf


Hi There,

over the last 11 Days I have LOST ~50 MB of available (FREE) Memory ... from the "PS -l"report it looks like the PRIVOXY process grows out of control.

Q: Anyone else seeing this? Remedies?

Thanks in advance for any pointers / help you can provide.

Robert
Sponsor
yoyoma2
DD-WRT User


Joined: 24 Sep 2016
Posts: 373

PostPosted: Wed Dec 06, 2017 19:01    Post subject: Reply with quote
Noticed privoxy growing more than twelvefold in a week using DD-WRT v3.0-r33675M kongac.

After running for a week:
Code:
  PID  PPID USER     STAT   VSZ %VSZ CPU %CPU COMMAND
 3045     1 root     S    26168 10.2   0  0.0 privoxy /tmp/privoxy.conf


After running for a few hours:
Code:
  PID  PPID USER     STAT   VSZ %VSZ CPU %CPU COMMAND
 3128     1 root     S     2040  0.8   0  0.2 privoxy /tmp/privoxy.conf


Since privoxy is a third party application, dd-wrt has no control over its memory leaks. What would be the best way in a daily cron job to cleanly stop/restart privoxy. How does dd-wrt do it when privoxy is enabled/disabled from the web interface "Services" tab?
rnio
DD-WRT User


Joined: 21 Apr 2012
Posts: 94

PostPosted: Wed Dec 06, 2017 21:02    Post subject: Reply with quote
For me it "calmed down" ... privoxy still uses a chunk of memory ~30MB ... but it seems to be stable.

I guess it's also in part due to the clients I have / had.

At the time when I experienced the "memory leak" / "extreme buffering", I used to just STOP the privoxy service / restart it ... through the web gui. (once a week or so)

Hope this helps,

Robert
Alozaros
DD-WRT Guru


Joined: 16 Nov 2015
Posts: 6446
Location: UK, London, just across the river..

PostPosted: Wed Dec 06, 2017 21:02    Post subject: Reply with quote
yoyoma2 wrote:
Noticed privoxy growing more than twelvefold in a week using DD-WRT v3.0-r33675M kongac.

After running for a week:
Code:
  PID  PPID USER     STAT   VSZ %VSZ CPU %CPU COMMAND
 3045     1 root     S    26168 10.2   0  0.0 privoxy /tmp/privoxy.conf


After running for a few hours:
Code:
  PID  PPID USER     STAT   VSZ %VSZ CPU %CPU COMMAND
 3128     1 root     S     2040  0.8   0  0.2 privoxy /tmp/privoxy.conf


Since privoxy is a third party application, dd-wrt has no control over its memory leaks. What would be the best way in a daily cron job to cleanly stop/restart privoxy. How does dd-wrt do it when privoxy is enabled/disabled from the web interface "Services" tab?


i gave up on privoxy and use start up adblocking script instead works fine...

_________________
Atheros
TP-Link WR740Nv1 ---DD-WRT 55630 WAP
TP-Link WR1043NDv2 -DD-WRT 55723 Gateway/DoT,Forced DNS,Ad-Block,Firewall,x4VLAN,VPN
TP-Link WR1043NDv2 -Gargoyle OS 1.15.x AP,DNS,QoS,Quotas
Qualcomm-Atheros
Netgear XR500 --DD-WRT 55779 Gateway/DoH,Forced DNS,AP Isolation,4VLAN,Ad-Block,Firewall,Vanilla
Netgear R7800 --DD-WRT 55819 Gateway/DoT,AD-Block,Forced DNS,AP&Net Isolation,x3VLAN,Firewall,Vanilla
Netgear R9000 --DD-WRT 55779 Gateway/DoT,AD-Block,AP Isolation,Firewall,Forced DNS,x2VLAN,Vanilla
Broadcom
Netgear R7000 --DD-WRT 55460 Gateway/SmartDNS/DoH,AD-Block,Firewall,Forced DNS,x3VLAN,VPN
NOT USING 5Ghz ANYWHERE
------------------------------------------------------
Stubby DNS over TLS I DNSCrypt v2 by mac913
droid-I
DD-WRT User


Joined: 10 Oct 2014
Posts: 101
Location: 60th parallel N.

PostPosted: Thu Dec 07, 2017 8:44    Post subject: Reply with quote
I had enabled privoxy@transparent on my EA6900 and it had run for a few days.
All of a sudden the traffic stopped. The 4G dongle indicated net connection still active.
The router did not answer to http and ping, seemed locked up.
This never happened before, I suspected privoxy.
If it was due to memory I do not know.
build : r33555 std

_________________
EA6900 v3.0-r52217 std ░ WZR-HP-AG300H-DD-WRT v3.0-r52095 std ░ WHR3-G54 v3.0-r29193 mini ░ WLI-TX4-G54HP ░ WBMR-HP-GNV2 ░ TL-MR3420v²
<Kong>
DD-WRT Guru


Joined: 15 Dec 2010
Posts: 4339
Location: Germany

PostPosted: Thu Dec 07, 2017 10:38    Post subject: Re: MEMORY: "privoxy" eating up memory on R70 Reply with quote
rnio wrote:
New Kong's test build: DD-WRT 33010M - 2017/08/09

Router Model Netgear R7000
Firmware Version DD-WRT v3.0-r33010M kongac (08/09/17)
Kernel Version Linux 4.4.80 #422 SMP Wed Aug 9 20:13:10 CEST 2017 armv7l
RESET: Yes
SERVICES USED: USB / NAS (Samba) / Adblocking

Privoxy

Privoxy: Enable
Provide Proxy Autoconfig: Enable
Transparent Mode: Disable
Custom Configuration: Disable


Code:
S   UID   PID  PPID   VSZ   RSS TTY   STIME TIME     CMD

S     0  1091     1  155m 62192 0:0   Aug10 01:12:14 privoxy /tmp/privoxy.conf


Hi There,

over the last 11 Days I have LOST ~50 MB of available (FREE) Memory ... from the "PS -l"report it looks like the PRIVOXY process grows out of control.

Q: Anyone else seeing this? Remedies?

Thanks in advance for any pointers / help you can provide.

Robert


There is some bug in mem calculation in busybox, privoxy does not use that amount of ram, on mine:

1086 1 root S 294m117.7 1 0.0 privoxy /tmp/privoxy.conf

it says 295MB, which is impossible, as my unit only has 256MB.

At the same time:

free
total used free shared buffers cached
Mem: 254536 180468 74068 0 7732 23752

Says only 180MB used in total.

_________________
KONG PB's: http://www.desipro.de/ddwrt/
KONG Info: http://tips.desipro.de/
yoyoma2
DD-WRT User


Joined: 24 Sep 2016
Posts: 373

PostPosted: Sat Dec 09, 2017 4:57    Post subject: Reply with quote
It appears that's just how busybox calculates the percentage process virtual size/physical memory size. It appears privoxy has a memory leak and the virtual size is growing over time and can exceed physical memory size (117% in your example) if you have swap enabled. I have noticed the "Inactive" bar grows over time in dd-wrt's Status->Router->Memory web page ever since privoxy was turned on.

What would be the best way in a daily cron job to cleanly stop/restart privoxy.
James2k
DD-WRT Guru


Joined: 23 Oct 2011
Posts: 549

PostPosted: Sat Dec 09, 2017 8:37    Post subject: Reply with quote
Something like:

Code:
0 0 * * * root stopservice privoxy && startservice privoxy


Would stop and start the Privoxy process at 12 AM daily.

_________________
James

Main router:

Netgear R7000 overclocked to 1.2GHz - DD-WRT v3.0-r35965M kongac

IPv6 6in4 (HE.net), OpenVPN (with PBR and split tunnelling), Entware, dnsmasq with ipset

Easy ipset support for the R7000

VPN speed: Download: 77.96 Mbps Upload: 5.00 Mbps (AES-128-CBC HMAC-SHA1)

Yes you can get 50 Mbps+ with OpenVPN on a R7000 if you configure it properly!

Previous routers:

ASUS RT-N66U - The Dark Knight
WNR2000v3 - Bought on the cheap for someone else, neutered crap
WNR3500Lv1 - First venture into the DD-WRT world
<Kong>
DD-WRT Guru


Joined: 15 Dec 2010
Posts: 4339
Location: Germany

PostPosted: Sat Dec 09, 2017 10:39    Post subject: Reply with quote
yoyoma2 wrote:
It appears that's just how busybox calculates the percentage process virtual size/physical memory size. It appears privoxy has a memory leak and the virtual size is growing over time and can exceed physical memory size (117% in your example) if you have swap enabled. I have noticed the "Inactive" bar grows over time in dd-wrt's Status->Router->Memory web page ever since privoxy was turned on.

What would be the best way in a daily cron job to cleanly stop/restart privoxy.


I don't have swap enabled and I have tested privoxy with valgrind before, not a single leak.

_________________
KONG PB's: http://www.desipro.de/ddwrt/
KONG Info: http://tips.desipro.de/
yoyoma2
DD-WRT User


Joined: 24 Sep 2016
Posts: 373

PostPosted: Sat Dec 09, 2017 19:18    Post subject: Reply with quote
@Kong: Since you have no swap perhaps the second answer makes more sense. If the size of linked but not loaded dynamic libraries are responsible (not memory leaks), that would explain why rnio observed: "For me it "calmed down" ... privoxy still uses a chunk of memory ~30MB ... but it seems to be stable. ". Edit: On second thought it's probably a memory fragmentation issue so not technically a memory leak. The malloc code needs to make a tradeoff between speed and memory usage and this implementation is probably leaning towards speed at the cost of more fragmentation.

Thanks James2k that does restart privoxy nicely.

@Kong: The privoxy service doesn't stop cleanly. That could be a concern if port 80 isn't properly restored to normal when privoxy is stopped.
Code:
# stopservice privoxy
Bad argument `80'
Try `iptables -h' or 'iptables --help' for more information.


Last edited by yoyoma2 on Tue Mar 20, 2018 19:22; edited 1 time in total
yoyoma2
DD-WRT User


Joined: 24 Sep 2016
Posts: 373

PostPosted: Fri Dec 22, 2017 22:08    Post subject: Reply with quote
Created a ticket 6132 for this error which includes a possible solution.
Display posts from previous:    Page 1 of 1
Post new topic   Reply to topic    DD-WRT Forum Index -> Broadcom SoC based Hardware All times are GMT

Navigation

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You can attach files in this forum
You can download files in this forum