Posted: Thu May 03, 2018 3:48 Post subject: Re: Tip: workaround to restart YAMon periodically
bytebandit01 wrote:
aloss wrote:
Since I installed release 3.4 I've noticed strange behavior with YAMon that stops fetching data, forcing me to restart YAMon, but the data history is definitely lost.
Looking at logs, it looks like YAMon is trying to add rules somewhere and keeps trying without success indeterminately.
I'm running YAMon on Linksys WRT3200ACM, with the latest version of DD-WRT running on it (35681, because the newer ones are bugged). I have already forwarded the logs for Al's analysis.
However, I am registering here to share the problem with you who may be going through a similar situation and to suggest a workaround tip I am using to get around the problem until it is solved by some new version of YAMon or DD-WRT.
What I'm doing is restarting YAMon every 2 hours, keeping it alive. For this, I'm using the following cron job (i.e.:Administration-->Management-->Cron):
Cron: enabled
Additional Cron Jobs:
15 * / 2 * * * root /opt/YAMon3/restart.sh
This causes YAMon to restart at 00h15, 02h15, 04h15 ...
I hope this tip can help some of you.
hi,
thanks for the input regarding cron jobs, i'm now having a similar concern..
previously with just the dsl connection and using the isp's modem not in bridge mode with the router (wrt 1200ac) i only get to restart yamon once in a week or maybe 2-3x in a month.
recently i upgraded from dsl to fiber connection and set the modem to bridge mode to the same router, keeping all previous dd-wrt settings unchanged and yamon, i now need to restart yamon on a daily basis because it just stops working at any random time of the day.
do you suggest having to restart yamon every 2 hours just like what you did? i'm planning to restart the script every 12 hours instead unfortunately i don't have knowledge about cron/scripts.
can i just simply replace 2 in your cron/script with 12 to have yamon restart every 12 hours?
Alexandre sent me a log file and it appears that something is causing a problem in iptables...
and for some unknown reason, the code that tries to re-add the missing entry does not work?!?
When things are in a bad way,what do you get if you run
iptables -L -vnx | grep Chain
Do you see these entries?
Chain YAMON34v4 (1 references)
Chain YAMON34v4Entry (3 references)
Chain YAMON34v4Local (15 references)
I'm flying blind because I've never seen this on my routers... but what happens if you edit yamon3.3.4.sh (starting at line #1452:
local nr=$(eval $1 $_tMangleOption -nL "$2" | wc -l)
if [ "$nr" -lt '3' ] ; then
$send2log "checkChainEntries: $2 returned only $nr entries?!? Resetting $cmd rules" 2
setupIPChains #Add this line
setUsers
fi
}
i have a question, I have several rules of QOS or Access Restrictions, if I change something and push the button apply settings, the Yamon stops working and tells me all the data as if they were from the router and not from each computer.
I have to enter by ssh and restart the yamon and there if it works again.
Is there any way to create some script or something so that every time I aproto the apply button, execute the option to reboot the yamon ?????
Or I do not know if it is normal that I should restart the Yamon every time.
i have a question, I have several rules of QOS or Access Restrictions, if I change something and push the button apply settings, the Yamon stops working and tells me all the data as if they were from the router and not from each computer.
I have to enter by ssh and restart the yamon and there if it works again.
Is there any way to create some script or something so that every time I aproto the apply button, execute the option to reboot the yamon ?????
Or I do not know if it is normal that I should restart the Yamon every time.
Thank you
Just guessing but I suspect that your iptables entries get clobbered after you update your QOS or Access Restrictions...
To confirm this can you
1. update the QOS or Access Restrictions
2. in a PuTTY (or equivalent terminal app) window, enter:
iptables -L -vnx
and then
iptables -L YAMON34v4 -vnx
(I'm guessing the latter will contain few entries if any)
Posted: Sun May 06, 2018 11:59 Post subject: Re: Tip: workaround to restart YAMon periodically
al_c wrote:
bytebandit01 wrote:
aloss wrote:
Since I installed release 3.4 I've noticed strange behavior with YAMon that stops fetching data, forcing me to restart YAMon, but the data history is definitely lost.
Looking at logs, it looks like YAMon is trying to add rules somewhere and keeps trying without success indeterminately.
I'm running YAMon on Linksys WRT3200ACM, with the latest version of DD-WRT running on it (35681, because the newer ones are bugged). I have already forwarded the logs for Al's analysis.
However, I am registering here to share the problem with you who may be going through a similar situation and to suggest a workaround tip I am using to get around the problem until it is solved by some new version of YAMon or DD-WRT.
What I'm doing is restarting YAMon every 2 hours, keeping it alive. For this, I'm using the following cron job (i.e.:Administration-->Management-->Cron):
Cron: enabled
Additional Cron Jobs:
15 * / 2 * * * root /opt/YAMon3/restart.sh
This causes YAMon to restart at 00h15, 02h15, 04h15 ...
I hope this tip can help some of you.
hi,
thanks for the input regarding cron jobs, i'm now having a similar concern..
previously with just the dsl connection and using the isp's modem not in bridge mode with the router (wrt 1200ac) i only get to restart yamon once in a week or maybe 2-3x in a month.
recently i upgraded from dsl to fiber connection and set the modem to bridge mode to the same router, keeping all previous dd-wrt settings unchanged and yamon, i now need to restart yamon on a daily basis because it just stops working at any random time of the day.
do you suggest having to restart yamon every 2 hours just like what you did? i'm planning to restart the script every 12 hours instead unfortunately i don't have knowledge about cron/scripts.
can i just simply replace 2 in your cron/script with 12 to have yamon restart every 12 hours?
Alexandre sent me a log file and it appears that something is causing a problem in iptables...
and for some unknown reason, the code that tries to re-add the missing entry does not work?!?
When things are in a bad way,what do you get if you run
iptables -L -vnx | grep Chain
Do you see these entries?
Chain YAMON34v4 (1 references)
Chain YAMON34v4Entry (3 references)
Chain YAMON34v4Local (15 references)
I'm flying blind because I've never seen this on my routers... but what happens if you edit yamon3.3.4.sh (starting at line #1452:
local nr=$(eval $1 $_tMangleOption -nL "$2" | wc -l)
if [ "$nr" -lt '3' ] ; then
$send2log "checkChainEntries: $2 returned only $nr entries?!? Resetting $cmd rules" 2
setupIPChains #Add this line
setUsers
fi
}
Al
yes, i have those entries above..
_________________ Router: Linksys WRT1200AC v2
Firmware: DD-WRT v3.0-r44048 std [08/02/20]
I started rebooting YAMon every 12 hours, but even then I lost traffic. So you decide to reboot every 2 hours because the restarting process is very fast.
However, on 07-may I installed the DD-WRT r35848 and since this day, around 20:00,YAMon is running without any interruption with cron disabled.
I'm following closely and I hope it has stabilized.
Al, I didn't answer you here because I had already answered you in pvt. Tks.
Joined: 12 May 2018 Posts: 31 Location: Broomfield, Colorado USA
Posted: Sat May 12, 2018 19:01 Post subject: Re: Tip: workaround to restart YAMon periodically
YAMon related issues discussed here: Suspected USB flash drive wearout, NTP Client, YAMon stops running and/or recording data, DD-WRT timeline. My first post so apply appropriate idiot filters, and I apologize for the length.
I have about 2 years experience using YAMon with great success on the e4200. The only other relatively unique things I do are implementation of 4 VLANs and pass through of Wake-On-LAN for a family-only webserver. I’m now in the process of moving to the r7800. DD-WRT router functionality is up and running, but so far I'm having a devil of a time with YAMon. On to the topics:
Suspected USB flash drive wearout: During my time with the e4200 I used Kingston 16GB thumb drives for /opt YAMon storage with ext2. I had two devices fail, the symptom being YAMon failure (white page for YAMon reports, can’t get YAMon running again, can't reinstall). So each time my reaction was to replace the drive and then successfully re-install YAMon. Attempts to reformat the drives revealed the drive failure. Might just be a problem with the drive brand, but I was suspicious of the YAMon write frequency or locality on the drive causing wearout, so I’ve switched to using an HDD (with ext4) on my USB port. If that is still running in a year I will be happy.
NTP Client: I had the same problems as others with the YAMon install complaining about NTP not being enabled even though the DD-WRT GUI said it was enabled. Fixed it by leaving the NTP Server/IP name blank. I had no problem with this back on my e4200 where I used a US NTP server name, but I was using a much older DD-WRT build on that router. With the v3.0-r34900M kongat ( 02/14/18 ) release on my new r7800 the NTP server name had to be blank so that DD-WRT would use it’s internal default NTP server name. See my notes below regarding DD-WRT timeline.
YAMon stops running and/or recording data: My problems appear to be identical to that being discussed by aloss and al_c. I have tried the aloss cron job suggestion, with the same result he gets: the cron job keeps YAMon running, but after a few hours all device data drops away. Side note: I see devices continue to appear in the “Live” report, but all have gone to an “unknown” source name in the “live” data – although all devices are still present and properly named in the “devices” tab. So I suspect Al is right about the iptables modifications going south.
DD-WRT timeline: I was going crazy trying to find some resource that would give me a clue about what was being addressed in DD-WRT releases. For other noobs like myself this link (http://svn.dd-wrt.com/timeline ) is hugely helpful in that regard. It is particularly helpful if you use Kong code as he is not identifying the build number in the names of his releases (you get to find out after you install – but using the timeline you can get a pretty good idea about where he is on the code in his test directory). The only reason I mention the “timeline” link is back to the NTP issue: Note a lot of time related fixes have occurred in the last 60 days, with a few mentioning use of NTP servers. So, this may be fixed in later test builds. Unfortunately I didn’t notice anything that made me think “Ah – iptables fixes”. Maybe I missed it.
A “by the way”: The e4200 was working well, but my ISP bumped me up from 100 mbps to 150 mbps (for free!). The e4200 with YAMon ON was only passing about 75 mbps. I could get about 90 mbps with YAMon OFF … telling me that the e4200 processor was my limitation. With the r7800 and YAMon ON I’m regularly measuring 180 to 190 mbps through the r7800 router, so I’m now hitting ISP limitations rather than router limits. This is perfect as I can now take full advantage of my ISP bandwidth, and I am sure I will get YAMon back in order soon.
I will go on to experimenting with more recent DD-WRT code to see if that helps clear up my YAMon issues – I will used the aloss r35848 (or later) suggestion. I am hopeful this is a DD-WRT bug that is recently fixed. I will report back, in much shorter form.
I currently run Yamon on 4 routers, 3 R9000 and 1 R8500 using various DD-WRT Kong and BS versions.
I have suffered a few USB stick failures over the time I've used Yamon and finally switched a couple of them to a Kingspec Z3 64gb USB SSD I found on Amazon.
One on a cool running USB 2 port on the 8500 and 3.0 port on the 9000.
Partitioned to 32gb for opt and jffs.
So far no issues and runs much faster than previous sticks.
For NTP I pinged my ISP NTP server to get the ip and use the ip in my setup to bypass any DNS issues at boot time. Works well.
I also use a sleep 60 or 90 command prior to starting Yamon to give the routers time to settle after booting before starting Yamon. This eliminated Yamon startup issues. _________________ Segment 1 XR700 10Gb LAN, 1Gb WAN ISP BS
Wired AP 1 Unifi Wifi 6 LR US 1Gb LAN
Wired AP 2 Unifi Wifi 6 LR US 1Gb LAN
Wired AP 3 Unifi Wifi 6 LR US 1Gb LAN
Syslog Services Asustor 7110T NAS 10GB
NetGear XS716T 10GB Switch
download1.dd-wrt.com/dd-wrtv2/downloads/betas/ (Brain Slayer)
YAMon https://usage-monitoring.com/index.php
Posted: Sun May 13, 2018 3:59 Post subject: Re: Tip: workaround to restart YAMon periodically
ColoradoUser wrote:
YAMon related issues discussed here: Suspected USB flash drive wearout, NTP Client, YAMon stops running and/or recording data, DD-WRT timeline. My first post so apply appropriate idiot filters, and I apologize for the length.
I have about 2 years experience using YAMon with great success on the e4200. The only other relatively unique things I do are implementation of 4 VLANs and pass through of Wake-On-LAN for a family-only webserver. I’m now in the process of moving to the r7800. DD-WRT router functionality is up and running, but so far I'm having a devil of a time with YAMon. On to the topics:
Glad to hear you like it!!!
ColoradoUser wrote:
Suspected USB flash drive wearout: During my time with the e4200 I used Kingston 16GB thumb drives for /opt YAMon storage with ext2. I had two devices fail, the symptom being YAMon failure (white page for YAMon reports, can’t get YAMon running again, can't reinstall). So each time my reaction was to replace the drive and then successfully re-install YAMon. Attempts to reformat the drives revealed the drive failure. Might just be a problem with the drive brand, but I was suspicious of the YAMon write frequency or locality on the drive causing wearout, so I’ve switched to using an HDD (with ext4) on my USB port. If that is still running in a year I will be happy.
I had a number of glitches when I used ext2... generally involving an unexpected power cycle on the router (I fully believe that problems occurred because the file system was left in a bad state rather than as a consequence of wearout... although that is an issue to consider). Once I started formatting to ext4, those problems have not re-occurred. Some users have found that firmware variants for older memory constrained routers may not support ext4...
ColoradoUser wrote:
NTP Client: I had the same problems as others with the YAMon install complaining about NTP not being enabled even though the DD-WRT GUI said it was enabled. Fixed it by leaving the NTP Server/IP name blank. I had no problem with this back on my e4200 where I used a US NTP server name, but I was using a much older DD-WRT build on that router. With the v3.0-r34900M kongat ( 02/14/18 ) release on my new r7800 the NTP server name had to be blank so that DD-WRT would use it’s internal default NTP server name. See my notes below regarding DD-WRT timeline.
What do you get if you enter the following commands in a PuTTY (or equivalent terminal app) window?
1. nvram get ntp_enable
2. nvram show | grep ntp
ColoradoUser wrote:
YAMon stops running and/or recording data: My problems appear to be identical to that being discussed by aloss and al_c. I have tried the aloss cron job suggestion, with the same result he gets: the cron job keeps YAMon running, but after a few hours all device data drops away. Side note: I see devices continue to appear in the “Live” report, but all have gone to an “unknown” source name in the “live” data – although all devices are still present and properly named in the “devices” tab. So I suspect Al is right about the iptables modifications going south.
This bug scares me... I cannot reproduce it my environment. I'm holding off on v3.4.5 to see if anything should be added for this issue.
Alexander said things got better when he updated is firmware... have you tried that as well?
When things get to a bad state, please open a PuTTY window and run
DD-WRT timeline: I was going crazy trying to find some resource that would give me a clue about what was being addressed in DD-WRT releases. For other noobs like myself this link (http://svn.dd-wrt.com/timeline ) is hugely helpful in that regard. It is particularly helpful if you use Kong code as he is not identifying the build number in the names of his releases (you get to find out after you install – but using the timeline you can get a pretty good idea about where he is on the code in his test directory). The only reason I mention the “timeline” link is back to the NTP issue: Note a lot of time related fixes have occurred in the last 60 days, with a few mentioning use of NTP servers. So, this may be fixed in later test builds. Unfortunately I didn’t notice anything that made me think “Ah – iptables fixes”. Maybe I missed it.
A “by the way”: The e4200 was working well, but my ISP bumped me up from 100 mbps to 150 mbps (for free!). The e4200 with YAMon ON was only passing about 75 mbps. I could get about 90 mbps with YAMon OFF … telling me that the e4200 processor was my limitation. With the r7800 and YAMon ON I’m regularly measuring 180 to 190 mbps through the r7800 router, so I’m now hitting ISP limitations rather than router limits. This is perfect as I can now take full advantage of my ISP bandwidth, and I am sure I will get YAMon back in order soon.
My ISP increased my speeds too... but I quickly learned that it just meant my kids could get us over our bandwidth cap faster! I finally moved to an unlimited plan and then my kids left home for university and miraculously, our bandwidth usage (and electricity and hotwater bill) has gone way down.
ColoradoUser wrote:
I will go on to experimenting with more recent DD-WRT code to see if that helps clear up my YAMon issues – I will used the aloss r35848 (or later) suggestion. I am hopeful this is a DD-WRT bug that is recently fixed. I will report back, in much shorter form.
Thanks for all of the info and details and for your help!
Joined: 12 May 2018 Posts: 31 Location: Broomfield, Colorado USA
Posted: Sun May 13, 2018 14:58 Post subject: Re: Tip: workaround to restart YAMon periodically
Hi Al – Yesterday evening I updated to Kong’s v3.0-r35900M kongat ( 05/08/18 ) from his test directory. From a little over 12 hours of run time it appears I have a dramatic improvement as YAMon kept running and there has been no apparent loss of data. I say “dramatic improvement” as I am still seeing a few infrequent entries in the “Live / Active Connections” data where the source is identified as “unknown”, yet the IP address is in fact a known device and address. That same device source name and IP address will show up properly in the active connections data on other prior and subsequent samples. So, something is still not quite right, but the problem is now very infrequent. To me this suggests a race condition of some kind, but that is a noob comment and not of much value without more information or code knowledge.
All this suggests the issue (and apparent fix) was/is in the DD-WRT code somewhere between r34900 and r35900. I looked over the DD-WRT timeline again and did find an iptables issue (see ticket 6277). It does not show as closed yet, so either some other fix is helping us, or perhaps some fixes for 6277 have been applied but it just isn’t complete at this time.
Since there is still some kind of issue I will keep an eye on this and let you know if it changes. I will also keep up with Kong’s test builds for awhile, although that carries the risk of getting to experience other in-development issues. All part of the process.
Regarding NTP you had asked me to send you my setup log and to run a couple of commands. But since I got around it by leaving the NTP field blank is that still needed? I thought I would re-try filling the NTP field with a server name to see if it was fixed with the DD-WRT update … if we were just dealing with a DD-WRT bug maybe it is now in the past.
As FYI my network has over 50 devices on it. 6 or 7 are active 24 hours a day with very little traffic (mostly NTP updates and minor maintenance traffic). Another dozen are heavy traffic generators during the late afternoon (grandkid’s computers and devices after school). So I’ve got a pretty good test bed if you need me to try stuff.
Posted: Sun May 13, 2018 15:23 Post subject: Re: Tip: workaround to restart YAMon periodically
Mike - thanks for the update (and the digging)
ColoradoUser wrote:
Hi Al – Yesterday evening I updated to Kong’s v3.0-r35900M kongat ( 05/08/18 ) from his test directory. From a little over 12 hours of run time it appears I have a dramatic improvement as YAMon kept running and there has been no apparent loss of data. I say “dramatic improvement” as I am still seeing a few infrequent entries in the “Live / Active Connections” data where the source is identified as “unknown”, yet the IP address is in fact a known device and address. That same device source name and IP address will show up properly in the active connections data on other prior and subsequent samples. So, something is still not quite right, but the problem is now very infrequent. To me this suggests a race condition of some kind, but that is a noob comment and not of much value without more information or code knowledge.
I've been looking at my live usage recent because I've got another user with some `live` issues and I also noticed an `unknown` mixed in with the (expected) tcp and udp entries... those come directly from from DD-WRT - because YAMon simply parses the contents of /proc/net/ip_conntrack or /proc/net/nf_conntrack (depending on settings & firmware).
In the background, I'm adding the ability to better filter the contents of the active connections table (i.e., show/hide TCP/UDP entries) and a few other tweaks
ColoradoUser wrote:
All this suggests the issue (and apparent fix) was/is in the DD-WRT code somewhere between r34900 and r35900. I looked over the DD-WRT timeline again and did find an iptables issue (see ticket 6277). It does not show as closed yet, so either some other fix is helping us, or perhaps some fixes for 6277 have been applied but it just isn’t complete at this time.
Since there is still some kind of issue I will keep an eye on this and let you know if it changes. I will also keep up with Kong’s test builds for awhile, although that carries the risk of getting to experience other in-development issues. All part of the process.
Again, thanks for this - I will definitely look at #6277 to see if that might be the issue... let me know how things go with the testing
ColoradoUser wrote:
Regarding NTP you had asked me to send you my setup log and to run a couple of commands. But since I got around it by leaving the NTP field blank is that still needed? I thought I would re-try filling the NTP field with a server name to see if it was fixed with the DD-WRT update … if we were just dealing with a DD-WRT bug maybe it is now in the past.
Yes please send the log, etc. (and perhaps the show where/how error message in the setup screen)... That way I can better check to see whether or not YAMon will have an issue with the configuration... at this point I use a simplistic nvram check.
ColoradoUser wrote:
As FYI my network has over 50 devices on it. 6 or 7 are active 24 hours a day with very little traffic (mostly NTP updates and minor maintenance traffic). Another dozen are heavy traffic generators during the late afternoon (grandkid’s computers and devices after school). So I’ve got a pretty good test bed if you need me to try stuff.
I may take you up on that! I've made a number of small changes for 3.4.5 and will want to get guinea pigs for testing soon.
Joined: 12 May 2018 Posts: 31 Location: Broomfield, Colorado USA
Posted: Sun May 13, 2018 15:45 Post subject:
ddaniel51 wrote:
I currently run Yamon on 4 routers, 3 R9000 and 1 R8500 using various DD-WRT Kong and BS versions.
I have suffered a few USB stick failures over the time I've used Yamon and finally switched a couple of them to a Kingspec Z3 64gb USB SSD I found on Amazon.
One on a cool running USB 2 port on the 8500 and 3.0 port on the 9000.
Partitioned to 32gb for opt and jffs.
So far no issues and runs much faster than previous sticks.
Hi ddaniel51 - When I was having USB stick failures I did some searches (on the internet so it's all true, right?) - and it seems some USB stick chips implement wear leveling (like an SSD) and others do not. I find that believable, but didn't keep digging to find out which ones do and which ones don't. But since I had two nearly identical failures with these Kingston devices on my router, but NO failures using them anywhere else, and the attempt to format showed un-writeable space in the same place on both, my best guess was wearout probably due to writing the same spot for months on end.
This does lead me to worry about using the router onboard flash for jffs (which I am still doing). Now that I've switched to an HDD for my USB drive I will likely move my jffs out there like you did.
