YAMon 3.0... Usage Reporting by device per hr/day/mo

Post new topic   This topic is locked: you cannot edit posts or make replies.    DD-WRT Forum Index -> General Questions
Goto page Previous  1, 2, 3 ... 107, 108, 109 ... 149, 150, 151  Next
Author Message
ddaniel51
DD-WRT Guru


Joined: 19 Feb 2013
Posts: 1464

PostPosted: Wed Jul 26, 2017 4:49    Post subject: Reply with quote
Al, there are 3 places in yamon3.3.html that point to config3.js instead of config3.3.js. :;
_________________
Segment 1 XR700 10Gb LAN, 1Gb WAN ISP BS
Wired AP 1 Unifi Wifi 6 LR US 1Gb LAN
Wired AP 2 Unifi Wifi 6 LR US 1Gb LAN
Wired AP 3 Unifi Wifi 6 LR US 1Gb LAN
Syslog Services Asustor 7110T NAS 10GB
NetGear XS716T 10GB Switch
download1.dd-wrt.com/dd-wrtv2/downloads/betas/ (Brain Slayer)
YAMon https://usage-monitoring.com/index.php
Sponsor
sribou
DD-WRT Novice


Joined: 15 Jun 2016
Posts: 7

PostPosted: Wed Jul 26, 2017 12:48    Post subject: Re: TypeError: Cannot set property '23' of undefined Reply with quote
al_c wrote:
sribou wrote:


Thanks for your message, i updated mine. Today, i have no error but when i will have I will do it.

Unfortunatelly, i have another repeating issue. Sometimes I have 16 in volume of data download or upload but it is corresponing to nothing and this value seems bigger than the others. how can I fix it ? Many thanks. This time I have it on unknow but sometime it is the same on one of my devices.

Regards.


That error indicates that you have bad values in your hourly usage file... data for a particular device is likely ridiculously big. The best way that I've been able to replicate this issue is to have two instances of the script running simultaneously. Please make sure you have just one instance running by entering
Code:
ps | grep =v grep | grep yam


If there is more that one instance of yamon3.3.x, then you should stop all instances and then run restart.sh

Does this help?

Al


Hi,

There were 4 process. I shudowns all and restart, will see.

For your info, below in hourly js :
TODAY
hu({"mac":"xx:xx:xx:xx","hour":"07","down":18446744073709597076,"up":54436})

YESTERDAY
hu({"mac":"un:kn:ow:n0:0m:ac","hour":"04","down":18446744073709586574,"up":0})


And I confirm that it is not possible

Regards.
al_c
DD-WRT Guru


Joined: 13 Apr 2013
Posts: 2134
Location: Ottawa Canada

PostPosted: Wed Jul 26, 2017 12:57    Post subject: Reply with quote
ddaniel51 wrote:
Al, there are 3 places in yamon3.3.html that point to config3.js instead of config3.3.js. :;


And you left finding them as an exercise for me... Smile

I've posted a(nother) minor update to yamon3.3.html to http://usage-monitoring.com/manualInstall.php.

You don't heed to rush out and get it though. I've also updated `yamon3.3.js` so that the changes are automagically made within `yamon3.3.html` (jQuery is *sooo* powerful!!!)

Thanks for bringing this to my attention.

Al
al_c
DD-WRT Guru


Joined: 13 Apr 2013
Posts: 2134
Location: Ottawa Canada

PostPosted: Wed Jul 26, 2017 13:13    Post subject: Re: TypeError: Cannot set property '23' of undefined Reply with quote
sribou wrote:
al_c wrote:
sribou wrote:


Thanks for your message, i updated mine. Today, i have no error but when i will have I will do it.

Unfortunatelly, i have another repeating issue. Sometimes I have 16 in volume of data download or upload but it is corresponing to nothing and this value seems bigger than the others. how can I fix it ? Many thanks. This time I have it on unknow but sometime it is the same on one of my devices.

Regards.


That error indicates that you have bad values in your hourly usage file... data for a particular device is likely ridiculously big. The best way that I've been able to replicate this issue is to have two instances of the script running simultaneously. Please make sure you have just one instance running by entering
Code:
ps | grep =v grep | grep yam


If there is more that one instance of yamon3.3.x, then you should stop all instances and then run restart.sh

Does this help?

Al


Hi,

There were 4 process. I shudowns all and restart, will see.

For your info, below in hourly js :
TODAY
hu({"mac":"xx:xx:xx:xx","hour":"07","down":18446744073709597076,"up":54436})

YESTERDAY
hu({"mac":"un:kn:ow:n0:0m:ac","hour":"04","down":18446744073709586574,"up":0})


And I confirm that it is not possible

Regards.

Are there multiple instances running again?
Code:
ps | grep =v grep | grep yam


If no, set loglevel=-1 in `config.file` and let things run for a while. Then send the zipped log file to questions@usage-monitoring.com

If yes, we have to figure out why.
NB - the main script periodically fires child processes so you might occasionally see two entries in the above results. If you do, wait a couple of seconds and then run the command again.

If there are still two or more, go to `/tmp` and delete the `YAMon3.3-running` directory. Wait a few seconds and run the ps command again... in theory, they should all disappear within 30s. If no, stop the processes (via kill <proc-id>).

Then run restart.sh (/opt/YAMon3/restart.sh)... you should see just one instance now...

You will have to clear the bad values out of the hourly data file manually (I replace them with zeroes... in theory you could likely go through the log file and find the last good value but I don't have time for that).

Coincidentally, I had the same issue arise on my router last night. I updated my firmware and when the router came back online, things were in a bad state. It might be safest to run shutdown.sh before updating firmware. Were you updating settings on your router when this behaviour started?

Thx
sribou
DD-WRT Novice


Joined: 15 Jun 2016
Posts: 7

PostPosted: Wed Jul 26, 2017 13:42    Post subject: Re: TypeError: Cannot set property '23' of undefined Reply with quote
al_c wrote:
sribou wrote:
al_c wrote:
sribou wrote:


Thanks for your message, i updated mine. Today, i have no error but when i will have I will do it.

Unfortunatelly, i have another repeating issue. Sometimes I have 16 in volume of data download or upload but it is corresponing to nothing and this value seems bigger than the others. how can I fix it ? Many thanks. This time I have it on unknow but sometime it is the same on one of my devices.

Regards.


That error indicates that you have bad values in your hourly usage file... data for a particular device is likely ridiculously big. The best way that I've been able to replicate this issue is to have two instances of the script running simultaneously. Please make sure you have just one instance running by entering
Code:
ps | grep =v grep | grep yam


If there is more that one instance of yamon3.3.x, then you should stop all instances and then run restart.sh

Does this help?

Al


Hi,

There were 4 process. I shudowns all and restart, will see.

For your info, below in hourly js :
TODAY
hu({"mac":"xx:xx:xx:xx","hour":"07","down":18446744073709597076,"up":54436})

YESTERDAY
hu({"mac":"un:kn:ow:n0:0m:ac","hour":"04","down":18446744073709586574,"up":0})


And I confirm that it is not possible

Regards.

Are there multiple instances running again?
Code:
ps | grep =v grep | grep yam


If no, set loglevel=-1 in `config.file` and let things run for a while. Then send the zipped log file to questions@usage-monitoring.com

If yes, we have to figure out why.
NB - the main script periodically fires child processes so you might occasionally see two entries in the above results. If you do, wait a couple of seconds and then run the command again.

If there are still two or more, go to `/tmp` and delete the `YAMon3.3-running` directory. Wait a few seconds and run the ps command again... in theory, they should all disappear within 30s. If no, stop the processes (via kill <proc-id>).

Then run restart.sh (/opt/YAMon3/restart.sh)... you should see just one instance now...

You will have to clear the bad values out of the hourly data file manually (I replace them with zeroes... in theory you could likely go through the log file and find the last good value but I don't have time for that).

Coincidentally, I had the same issue arise on my router last night. I updated my firmware and when the router came back online, things were in a bad state. It might be safest to run shutdown.sh before updating firmware. Were you updating settings on your router when this behaviour started?

Thx


I changed config.file and restarted and yet, sometimes I have 2 or 1 process :

root@unknown:/cifs1/YAMon3# ps | grep -v grep | grep yam
12920 root 1724 S {yamon3.3.1.sh} /bin/sh ./yamon3.3.1.sh
24638 root 1724 S {yamon3.3.1.sh} /bin/sh ./yamon3.3.1.sh
root@unknown:/cifs1/YAMon3# ps | grep -v grep | grep yam
12920 root 1724 S {yamon3.3.1.sh} /bin/sh ./yamon3.3.1.sh
root@unknown:/cifs1/YAMon3# ps | grep -v grep | grep yam
12920 root 1724 S {yamon3.3.1.sh} /bin/sh ./yamon3.3.1.sh
root@unknown:/cifs1/YAMon3#


I will wait and check.

Regards.

Many thanks again.
NiTrus
DD-WRT User


Joined: 25 Dec 2010
Posts: 295
Location: Twin Cities, MN

PostPosted: Wed Jul 26, 2017 17:18    Post subject: Reply with quote
ddaniel51 wrote:
In the latest dd-wrt builds with busybox 1.27 the -e option
has been replaced with --full-time.

This requires a change to line 141 in yamon3.3.1.sh

local ulmt=$(ls -e "$_usersFile" | tr -s ' ' ' ' )
to
local ulmt=$(ls --full-time "$_usersFile" | tr -s ' ' ' ' )

Al, would it help to put both lines in?


Al, this change still isnt present in latest yamon3.3.1.sh
i changed manually

_________________
NETGEAR R9000 | RT | 40134
NETGEAR R7800 | AP | 40134

al_c
DD-WRT Guru


Joined: 13 Apr 2013
Posts: 2134
Location: Ottawa Canada

PostPosted: Thu Jul 27, 2017 2:36    Post subject: Reply with quote
NiTrus wrote:
ddaniel51 wrote:
In the latest dd-wrt builds with busybox 1.27 the -e option
has been replaced with --full-time.

This requires a change to line 141 in yamon3.3.1.sh

local ulmt=$(ls -e "$_usersFile" | tr -s ' ' ' ' )
to
local ulmt=$(ls --full-time "$_usersFile" | tr -s ' ' ' ' )

Al, would it help to put both lines in?


Al, this change still isnt present in latest yamon3.3.1.sh
i changed manually


You are right. I made the change in my as-yet-not-available 3.3.2... and I found a better solution to the one suggested above.

If you want to make the updates yourself
1. delete lines #142-149:
Code:
local ulmt=$(ls -e "$_usersFile" | tr -s ' ' ' ' )
local lmy=$(echo "$ulmt" | cut -d' ' -f10)
local lmm=$(echo "$ulmt" | cut -d' ' -f7)
local lmd=$(echo "$ulmt" | cut -d' ' -f8)
local lmt=$(echo "$ulmt" | cut -d' ' -f9)
local months="Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec"
local string="${months%$lmm*}"
local lmmno=$(printf %02d $((${#string}/4 + 1)))



2. And replace line #165:
Code:
_usersLastMod="$lmy-$lmmno-$lmd $lmt"

with
Code:
_usersLastMod=$(date -r "$_usersFile" "+%Y-%d-%m %T")


(9 lines of code replaced with just one... much better Smile)
al_c
DD-WRT Guru


Joined: 13 Apr 2013
Posts: 2134
Location: Ottawa Canada

PostPosted: Thu Jul 27, 2017 3:10    Post subject: Reply with quote
NiTrus wrote:
just a fyi, in newer fw builds with SFE(Shortcut Forwarding Engine) enabled, yamon doesnt record download/upload usage correctly, so i turned sfe off & everything started working correctly..anyone else experience this?


I can confirm this issue... if the Shortcut Forwarding Engine (SFE) option is enabled, the usage totals in YAMon are incorrect. The `measured at the router values seem to be correct.

It appears tat SFE interferes with the `normal` flow of traffic through iptables. But, I've taken a quick look at my iptables entries with and without the option enabled and there were no obvious changes that I could see.

This feature was added in the most recent firmware build that I downloaded earlier this week (r32753) and it is enabled by default.

The only fix that I am aware of at the moment is to disable the option (in the DD-WRT admin UI-->Setup-->Basic Setup-->Optional Settings).

If anyone knows more about this, please get it touch.

Thanks!

Al
ddaniel51
DD-WRT Guru


Joined: 19 Feb 2013
Posts: 1464

PostPosted: Thu Jul 27, 2017 8:59    Post subject: Reply with quote
al_c wrote:

You are right. I made the change in my as-yet-not-available 3.3.2... and I found a better solution to the one suggested above.

If you want to make the updates yourself
1. delete lines #142-149:
Code:
local ulmt=$(ls -e "$_usersFile" | tr -s ' ' ' ' )
local lmy=$(echo "$ulmt" | cut -d' ' -f10)
local lmm=$(echo "$ulmt" | cut -d' ' -f7)
local lmd=$(echo "$ulmt" | cut -d' ' -f8)
local lmt=$(echo "$ulmt" | cut -d' ' -f9)
local months="Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec"
local string="${months%$lmm*}"
local lmmno=$(printf %02d $((${#string}/4 + 1)))



2. And replace line #165:
Code:
_usersLastMod="$lmy-$lmmno-$lmd $lmt"

with
Code:
_usersLastMod=$(date -r "$_usersFile" "+%Y-%d-%m %T")


(9 lines of code replaced with just one... much better Smile)



Al, 2 things:

What editor are you using? My 2 editors (Prolix 32 and WinSCP) have different line counts than yours, either 7 or 1 lines difference.

Second, I used the posted code snippets as a guide and then yamon3.3.1.sh would not run. Permissions ck'd ok.
Funny, second time worked ok. Fumble fingers.....

Would you mind sending out your own corrected yamon3.3.1.sh please?

_________________
Segment 1 XR700 10Gb LAN, 1Gb WAN ISP BS
Wired AP 1 Unifi Wifi 6 LR US 1Gb LAN
Wired AP 2 Unifi Wifi 6 LR US 1Gb LAN
Wired AP 3 Unifi Wifi 6 LR US 1Gb LAN
Syslog Services Asustor 7110T NAS 10GB
NetGear XS716T 10GB Switch
download1.dd-wrt.com/dd-wrtv2/downloads/betas/ (Brain Slayer)
YAMon https://usage-monitoring.com/index.php


Last edited by ddaniel51 on Fri Jul 28, 2017 3:50; edited 2 times in total
ddaniel51
DD-WRT Guru


Joined: 19 Feb 2013
Posts: 1464

PostPosted: Thu Jul 27, 2017 16:35    Post subject: Reply with quote
Al, here's another head scratcher.

After a random amount of time, yamon3.3.html stops updating
from the hourly.js and live usage js.

It holds a snapshot of the frozen js's and starts flashing it's time missing warnings.

While it's doing this the hourly in /data and live js's are continuing to be updated as usual.

The refresh timer continues to run, but isn't actually refreshing anything.

Same in chrome and firefox. Ctl/F5 has no effect.

Js errors are :
index.html:1 This page includes a password or credit card input in a non-secure context. A warning has been added to the URL bar. For more information, see https://goo.gl/zmWq3m.
4jsapi_compiled_default_module.js:119 [Violation] Added non-passive event listener to a scroll-blocking 'mousewheel' event. Consider marking event handler as 'passive' to make the page more responsive.
jquery-latest.min.js:4 [Violation] 'readystatechange' handler took 252ms
[Violation] Forced reflow while executing JavaScript took 37ms
4jsapi_compiled_default_module.js:119 [Violation] Added non-passive event listener to a scroll-blocking 'mousewheel' event. Consider marking event handler as 'passive' to make the page more responsive.
jquery-latest.min.js:4 [Violation] 'readystatechange' handler took 174ms

_________________
Segment 1 XR700 10Gb LAN, 1Gb WAN ISP BS
Wired AP 1 Unifi Wifi 6 LR US 1Gb LAN
Wired AP 2 Unifi Wifi 6 LR US 1Gb LAN
Wired AP 3 Unifi Wifi 6 LR US 1Gb LAN
Syslog Services Asustor 7110T NAS 10GB
NetGear XS716T 10GB Switch
download1.dd-wrt.com/dd-wrtv2/downloads/betas/ (Brain Slayer)
YAMon https://usage-monitoring.com/index.php
NiTrus
DD-WRT User


Joined: 25 Dec 2010
Posts: 295
Location: Twin Cities, MN

PostPosted: Fri Jul 28, 2017 0:16    Post subject: Reply with quote
al_c wrote:
NiTrus wrote:
ddaniel51 wrote:
In the latest dd-wrt builds with busybox 1.27 the -e option
has been replaced with --full-time.

This requires a change to line 141 in yamon3.3.1.sh

local ulmt=$(ls -e "$_usersFile" | tr -s ' ' ' ' )
to
local ulmt=$(ls --full-time "$_usersFile" | tr -s ' ' ' ' )

Al, would it help to put both lines in?


Al, this change still isnt present in latest yamon3.3.1.sh
i changed manually


You are right. I made the change in my as-yet-not-available 3.3.2... and I found a better solution to the one suggested above.

If you want to make the updates yourself
1. delete lines #142-149:
Code:
local ulmt=$(ls -e "$_usersFile" | tr -s ' ' ' ' )
local lmy=$(echo "$ulmt" | cut -d' ' -f10)
local lmm=$(echo "$ulmt" | cut -d' ' -f7)
local lmd=$(echo "$ulmt" | cut -d' ' -f8)
local lmt=$(echo "$ulmt" | cut -d' ' -f9)
local months="Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec"
local string="${months%$lmm*}"
local lmmno=$(printf %02d $((${#string}/4 + 1)))



2. And replace line #165:
Code:
_usersLastMod="$lmy-$lmmno-$lmd $lmt"

with
Code:
_usersLastMod=$(date -r "$_usersFile" "+%Y-%d-%m %T")


(9 lines of code replaced with just one... much better Smile)


works for me, although it was line 141-148 & 164...using textastic for ios..

_________________
NETGEAR R9000 | RT | 40134
NETGEAR R7800 | AP | 40134

al_c
DD-WRT Guru


Joined: 13 Apr 2013
Posts: 2134
Location: Ottawa Canada

PostPosted: Fri Jul 28, 2017 13:46    Post subject: Reply with quote
ddaniel51 wrote:
al_c wrote:

You are right. I made the change in my as-yet-not-available 3.3.2... and I found a better solution to the one suggested above.

If you want to make the updates yourself
1. delete lines #142-149:
Code:
local ulmt=$(ls -e "$_usersFile" | tr -s ' ' ' ' )
local lmy=$(echo "$ulmt" | cut -d' ' -f10)
local lmm=$(echo "$ulmt" | cut -d' ' -f7)
local lmd=$(echo "$ulmt" | cut -d' ' -f8)
local lmt=$(echo "$ulmt" | cut -d' ' -f9)
local months="Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec"
local string="${months%$lmm*}"
local lmmno=$(printf %02d $((${#string}/4 + 1)))



2. And replace line #165:
Code:
_usersLastMod="$lmy-$lmmno-$lmd $lmt"

with
Code:
_usersLastMod=$(date -r "$_usersFile" "+%Y-%d-%m %T")


(9 lines of code replaced with just one... much better Smile)



Al, 2 things:

What editor are you using? My 2 editors (Prolix 32 and WinSCP) have different line counts than yours, either 7 or 1 lines difference.

Second, I used the posted code snippets as a guide and then yamon3.3.1.sh would not run. Permissions ck'd ok.
Funny, second time worked ok. Fumble fingers.....

Would you mind sending out your own corrected yamon3.3.1.sh please?

I use Notepad++ on WinX. I was working from a local copy... I think I was off by one line... look for the text.

I will update the current file tonight or tomorrow with the change (but without a minor version bump)
sigmax
DD-WRT Novice


Joined: 22 Nov 2011
Posts: 29

PostPosted: Fri Jul 28, 2017 16:27    Post subject: Yamon and PPPoE Overhead Reply with quote
Al,

I've been quiet all these days because -after going back a couple of dd-wrt firmware releases, and manually installing your 3.3.1 (3.3.2 html) YAMon, it successfully started working.

So I could finally compare its results against my TELCO data consumption page.

The telco shows about 15% - 18% bigger numbers.

I DID ask them about it, and their unofficial response was something that YOU MAY WANT TO TAKE INTO CONSIDERATION:

PPPoE Overhead.

Measuring the way YAMon does, is not resulting into the real data, it seems, as the router has to encapsulate all those packages into a PPPoE stream. In such process, several bytes are added into the equation for each packet.

It may be a nice idea to allow adding bytes per packet (Packet size may be determined somehow ... maybe MTU parameter).

What do you think ? I do not know enough about TELCO business in order to rebut or confirm their saying.

Regards,

Enrique
gainestr
DD-WRT User


Joined: 06 Jul 2017
Posts: 78

PostPosted: Fri Jul 28, 2017 21:51    Post subject: Reply with quote
Al,
After installing DD-WRT build 32868 and re-installing YAMon3, my live tab in YAMon is only updating once every 10 minutes. I can not seem to get it to update every 30 seconds like it used to before.

Any help with this?
ddaniel51
DD-WRT Guru


Joined: 19 Feb 2013
Posts: 1464

PostPosted: Sat Jul 29, 2017 0:28    Post subject: Reply with quote
Dropped in the yamon3.3.html with the config3 changes and the updates started working again.

4 hrs later it quit again.

Where is this thing hiding these frozen hourly updates?

var hourly_updated="2017-07-28 10:10:02"
var users_updated="2017-27-07 20:44:42"
var disk_utilization="1%"
var serverUptime="86670.01"
var freeMem="346848",availMem="452236",totMem="514692"
serverloads("0.04","01:22:43","0.20","00:03:07")

_________________
Segment 1 XR700 10Gb LAN, 1Gb WAN ISP BS
Wired AP 1 Unifi Wifi 6 LR US 1Gb LAN
Wired AP 2 Unifi Wifi 6 LR US 1Gb LAN
Wired AP 3 Unifi Wifi 6 LR US 1Gb LAN
Syslog Services Asustor 7110T NAS 10GB
NetGear XS716T 10GB Switch
download1.dd-wrt.com/dd-wrtv2/downloads/betas/ (Brain Slayer)
YAMon https://usage-monitoring.com/index.php
Goto page Previous  1, 2, 3 ... 107, 108, 109 ... 149, 150, 151  Next Display posts from previous:    Page 108 of 151
Post new topic   This topic is locked: you cannot edit posts or make replies.    DD-WRT Forum Index -> General Questions All times are GMT

Navigation

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You can attach files in this forum
You can download files in this forum