Yet Another Monitor... usage by user/IP by hour, day & m

Post new topic   This topic is locked: you cannot edit posts or make replies.    DD-WRT Forum Index -> General Questions
Goto page 1, 2, 3 ... 46, 47, 48  Next
Author Message
al_c
DD-WRT Guru


Joined: 13 Apr 2013
Posts: 2134
Location: Ottawa Canada

PostPosted: Thu Jul 25, 2013 5:46    Post subject: Yet Another Monitor... usage by user/IP by hour, day & m Reply with quote
NOTE:

This is version: 1.0.17B (I do not plan to make any additional changes to the v1 code base)

Instead, I am now working on an improved version 2 of YAMon - see http://www.dd-wrt.com/phpBB2/viewtopic.php?t=259806 for downloads and more info

Thx!

Al



In short, "YAMon" (for Yet Another Monitor) allows you to see which devices on your network are using how much bandwidth and when.

It consists of a `bash` script and HTML reports that run on your [DD-WRT] router. The script generates small user and usage files which are processed into meaningful reports within your browser. The usage is summarized in a variety of tables and graphs allowing you to see hourly, daily and monthly totals for all of the devices on your network.
The hourly graphs will let you see when your kids are actually shutting off their devices... no wonder they're tired in the AM! Smile
The monthly totals may help you prevent overages charges if your ISP caps your bandwidth... stop the heavy downloaders before they burn through your allowance.
A simple UI allows you to compare the usage as record on your router with what is reported by your ISP... you get some interesting differences there!
And, new in the 1.0.15 release, there are also `live` usage updates... server loads, number of connections, active users, etc. More features will be added to this section shortly.

I've tried to make things as simple as possible. The script can be run without any additional parameters (save for the path to the configuration file if you choose to store it some where other than the same folder as the script itself). The configuration file includes a variety of options that allow you to manage how frequently the script runs, whether to backup the data files and conntrol the logging of activity. You can edit those settings and the script will pick up the changes on the fly. The script automatically creates all necessary folders and copies files to the appropriate locations. I've added a lot of logic that detects (and recovers from) oddball behaviours that I witnessed in other similar scripts.

In the Web UI, I make extensive use of localStorage strings to retain settings, preferences and other information between sessions. There are a variety of tables and graphs which [hopefully] present the info in a meaningful way.

The attached zip file includes a readme.txt which provides basic instructions for installation. In the following pages of this thread, you'll find a number of discussions that helped resolve problems and have directed the continuing development of YAMon... a big thanks to every one who has provided their input & feedback.

I'll do my best to help with any installation questions/concerns you might have (but I'm still a noob when it comes to setting up DD-WRT). Let me know if you'd like features added/adapted.

HISTORY:
YAMON has been adapted/extended from a number of other similar scripts - in particular, wrtbwmon. A big thanks to the authors of those tools. For background information about wrtbwmon, see http://www.dd-wrt.com/phpBB2/viewtopic.php?t=81432&highlight=... YAMon discussions get started somewhere around pg 36 or 37 of that thread.


Al



yamon1.0.15.gif
 Description:
 Filesize:  188.73 KB
 Viewed:  69802 Time(s)

yamon1.0.15.gif



yamon (v1.0.17B).zip
 Description:
version 1.0.17B is the current version... !

Download
 Filename:  yamon (v1.0.17B).zip
 Filesize:  55.2 KB
 Downloaded:  4288 Time(s)



Last edited by al_c on Thu Apr 17, 2014 15:27; edited 25 times in total
Sponsor
spanman
DD-WRT User


Joined: 27 Sep 2008
Posts: 446
Location: Port Of Spain

PostPosted: Thu Jul 25, 2013 6:22    Post subject: Reply with quote
Nice al_c love the changes.
_________________
Buffalo WZR-600DHP - 23838
TP Link WDR3600 - 21676
Linksys WRT54G-TM - 12548 (NEWD Eko Mega)
spirosb
DD-WRT Novice


Joined: 20 May 2013
Posts: 17

PostPosted: Thu Jul 25, 2013 10:43    Post subject: Reply with quote
Al thanks for all the work you have been putting into this script. It will hopefully be a nice addition into DD-WRT once the bugs have gone out.

In 1.0.6 the negative bug(overflow) still exists (fresh YAMon script install) in a K3x DD-WRT build on a Netgear WNDR4500v1.
Also (you might have notice and it might be intended) the new new hourly graph overlaps the daily usage pie chart.

Thanks again and keep up the good work Smile
spanman
DD-WRT User


Joined: 27 Sep 2008
Posts: 446
Location: Port Of Spain

PostPosted: Thu Jul 25, 2013 11:45    Post subject: Reply with quote
bit overflow is specific to your router not a bug with the script. My router also had the bit overflow issues but whatever changes al made I no longer get negative values.

al_c the overlap in the graph is due to the different resolution persons monitors are displaying at that is why we see the new hourly graph overlapping the daily usage pie chart while you don't. My display is set a 1366x 768 and I too have the overlapping. So it is the dynamics in resolution.

EDIT: I added three breaks to the html file and I don't get any overlap like image attached

</br> </br> </br>

By the way love how you adjusted the legend and made it simpler



sshot.jpg
 Description:
 Filesize:  73.2 KB
 Viewed:  77411 Time(s)

sshot.jpg



_________________
Buffalo WZR-600DHP - 23838
TP Link WDR3600 - 21676
Linksys WRT54G-TM - 12548 (NEWD Eko Mega)
al_c
DD-WRT Guru


Joined: 13 Apr 2013
Posts: 2134
Location: Ottawa Canada

PostPosted: Thu Jul 25, 2013 14:56    Post subject: Reply with quote
spirosb wrote:
Al thanks for all the work you have been putting into this script. It will hopefully be a nice addition into DD-WRT once the bugs have gone out.

In 1.0.6 the negative bug(overflow) still exists (fresh YAMon script install) in a K3x DD-WRT build on a Netgear WNDR4500v1.
Also (you might have notice and it might be intended) the new new hourly graph overlaps the daily usage pie chart.

Thanks again and keep up the good work Smile


Thanks to spanman for his defense of the script but, in spite of the fact that this appears to be a hardware specific issue, I agree that the integer overflow issue is still a defect. I was so hoping that I would not have to revert to the `bc` package to make this work but it appears that I will have to.

I'm thinking that I'll add a `useBC` parameter to the config.file and if this value is set to one, the script will perform the calculation using the `bc` package; otherwise it'll use the standard method.
This does not add much complexity to the script but it does mean that the script will be [at least partially] dependent upon an external package and that might make it more difficult for noobs (like me) to get it installed and configured.

Oh... the trade-offs we have to make in life.

I'll get a new version of the script to you by the weekend.

Al
al_c
DD-WRT Guru


Joined: 13 Apr 2013
Posts: 2134
Location: Ottawa Canada

PostPosted: Thu Jul 25, 2013 15:04    Post subject: Reply with quote
spanman wrote:
bit overflow is specific to your router not a bug with the script. My router also had the bit overflow issues but whatever changes al made I no longer get negative values.

Nah... its a bug! Damn those hardware issues

But I think I've come up with another [slightly more complex] way to resolve this (again)...


spanman wrote:
al_c the overlap in the graph is due to the different resolution persons monitors are displaying at that is why we see the new hourly graph overlapping the daily usage pie chart while you don't. My display is set a 1366x 768 and I too have the overlapping. So it is the dynamics in resolution.

EDIT: I added three breaks to the html file and I don't get any overlap like image attached


Damn those browser/resolutions issues

I'm hoping that I can tweak the CSS to prevent the overlap...
However, the recent legend vs pie chart fiasco makes me worried that this might be one of those situations that sound easy if you say it fast.

Thanks for the positive feedback though.

Al
Canadian Geek
DD-WRT Novice


Joined: 18 Jun 2013
Posts: 49

PostPosted: Thu Jul 25, 2013 17:14    Post subject: Reply with quote
Looking very good Al. I've already removed the old monitoring software from my router (why run two and overload the router) as this one is so much better. My two kids (kids Wink yeah, 20 & 18 ) have access to the page and like I said, usage has gone WAY down since I started monitoring. No more pointing fingers at each other. In fact, they both like it because it has helped them debug problems of programs downloading without their knowledge.

Jeff
al_c
DD-WRT Guru


Joined: 13 Apr 2013
Posts: 2134
Location: Ottawa Canada

PostPosted: Thu Jul 25, 2013 18:00    Post subject: Reply with quote
Canadian Geek wrote:
Looking very good Al. I've already removed the old monitoring software from my router (why run two and overload the router) as this one is so much better. My two kids (kids Wink yeah, 20 & 18 ) have access to the page and like I said, usage has gone WAY down since I started monitoring. No more pointing fingers at each other. In fact, they both like it because it has helped them debug problems of programs downloading without their knowledge.

Jeff


Jeff - Good to hear.
I started looking for a monitoring tool because I didn't believe the numbers that I was getting on the bills from my ISP (Rogers)... they simply did not jive with what the standard firmware on my router reported. I would get their bill but I couldn't see their meter and they couldn't tell me which devices were responsible for the overages.

My first step was to get DD-WRT installed and configured but even so, the reporting did not give me the granularity I wanted. I found `wrtbwmonitor` but I didn't like the reports so I started hacking them and then I found a variety of performance issues on the router so next I attacked the script.

I'm not sure if its because I let my ISP know that I'm measuring what they report or whether my kids have tempered their usage because I'm monitoring things but our total usage on the ISP bill is down by 20-30%! Their numbers now are only 1-2% higher than what I measure and typically my numbers differ from the router Admin-->WAN(LAN?) figures by less than 1%. So I'm *really* pleased with the results already Very Happy

I've only just implemented the hourly graph but I've already noticed some interesting 'behaviours'... apparently my kids are using their iThings *long* after bed time. We'll have to nip that in the bud.

Al
asfd64
DD-WRT User


Joined: 13 Nov 2010
Posts: 59

PostPosted: Thu Jul 25, 2013 18:08    Post subject: Reply with quote
The new hourly graph is nice, great work!

I still have the integer issue but sounds like your plan to use bc will work.

Thanks for the update Al!

Ian
al_c
DD-WRT Guru


Joined: 13 Apr 2013
Posts: 2134
Location: Ottawa Canada

PostPosted: Thu Jul 25, 2013 18:16    Post subject: Reply with quote
asfd64 wrote:
The new hourly graph is nice, great work!

I still have the integer issue but sounds like your plan to use bc will work.

Thanks for the update Al!

Ian


Ian - thanks!

I've actually coded up the revised logic - actually a very simple change to the script... less than 10 lines changed - but I want to run it at home for a while before releasing it to the wild.

I'll send you a copy ASAP

Al
asfd64
DD-WRT User


Joined: 13 Nov 2010
Posts: 59

PostPosted: Thu Jul 25, 2013 18:24    Post subject: Reply with quote
You are quick Al, thanks!
chtse53
DD-WRT Novice


Joined: 01 Jul 2013
Posts: 10

PostPosted: Thu Jul 25, 2013 18:45    Post subject: Reply with quote
I followed the instruction on the readme.txt but I always got an error about not able to read from the devices file and no data was shown on the webpage. I can see that the users.html file and the -date-mac_data.html file under /tmp/www/data. The contents of these appear to contain the right information. What have I done wrong?
al_c
DD-WRT Guru


Joined: 13 Apr 2013
Posts: 2134
Location: Ottawa Canada

PostPosted: Thu Jul 25, 2013 19:26    Post subject: Reply with quote
chtse53 wrote:
I followed the instruction on the readme.txt but I always got an error about not able to read from the devices file and no data was shown on the webpage. I can see that the users.html file and the -date-mac_data.html file under /tmp/www/data. The contents of these appear to contain the right information. What have I done wrong?


The good news is that the script seems to be running (as the date files are being generated)

Can you PM me a few screen shots?
1. the settings tab of the web ui
2. the contents of /tmp/www/data directory

I'm guessing that the values of the ISPBilling date in config.file and on the settings tab are out of whack.


Al
al_c
DD-WRT Guru


Joined: 13 Apr 2013
Posts: 2134
Location: Ottawa Canada

PostPosted: Thu Jul 25, 2013 19:57    Post subject: Reply with quote
chtse53 wrote:
Content of the /tmp/www/data:
03-07-2013-mac_data.html users.html 25-07-2013-hourly_data.html

Screenshot of the settings tab attached.


Hmmm... not what I thought it was in my previous response. Can you also send me screen shot of the contents of /tmp/www/data?

Does the file /tmp/www/data/users.html exist? Is it more than zero bytes?

Al
bpsmicro
DD-WRT User


Joined: 07 Oct 2010
Posts: 180

PostPosted: Thu Jul 25, 2013 22:05    Post subject: Reply with quote
Couple of curiosities I'm following with the hourly usage thing:

1) If I reboot the router, it seems to lose the hourly data from earlier in the day. Perhaps that's deliberate?
2) At some point, the hourly_usage.html file gets "corrupted", and the web page shows what looks like the data table contents rather than the graph.

Neither of these are particularly worrisome, I'm just keeping an eye on the situation. I'll PM the data file once I capture a good candidate.
Goto page 1, 2, 3 ... 46, 47, 48  Next Display posts from previous:    Page 1 of 48
Post new topic   This topic is locked: you cannot edit posts or make replies.    DD-WRT Forum Index -> General Questions All times are GMT

Navigation

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You can attach files in this forum
You can download files in this forum