OK after further investigation, part of the problem is me learning YAMon and part appears to be an issue with YAMon. It looks like at least one device on my network is being tallied as WAN traffic when it is clearly LAN only: it has no access to the "outside world", only the LAN. Yet YAMon shows all traffic from that device (a security camera connected to my PC via wireless). Anyone know what could cause YAMon to misread LAN traffic as WAN traffic? Also, it'd be handy if there were checkboxes to actually enable LAN traffic monitoring and not just WAN. Sometimes I'd like to see what is bogging down my LAN traffic too!
OK after further investigation, part of the problem is me learning YAMon and part appears to be an issue with YAMon. It looks like at least one device on my network is being tallied as WAN traffic when it is clearly LAN only: it has no access to the "outside world", only the LAN. Yet YAMon shows all traffic from that device (a security camera connected to my PC via wireless). Anyone know what could cause YAMon to misread LAN traffic as WAN traffic? Also, it'd be handy if there were checkboxes to actually enable LAN traffic monitoring and not just WAN. Sometimes I'd like to see what is bogging down my LAN traffic too!
Mike
Mike -
If you open Putty and enter: cat "/proc/net/arp",
does the camera have a different entry under the Device heading? If yes, there is a way to accommodate this for you.
Search in yamon.sh for
Code:
#if [ "$iface" != "$LAN_IFACE" ] ; then continue; fi
Remove the `#` at the beginning of the line
NB - you will have to do this for all 3 occurrences of this line.
Then stop and restart yamon.sh
It's hard to tell without seeing things but I think this'll give you want you want.
Sorry you had problems with the initial version(s) of 2. I've added an option within the v2.0 config.file that allows you to turn this feature on/off without having to restart the script (or edit the script file).
Mike -
If you open Putty and enter: cat "/proc/net/arp",
does the camera have a different entry under the Device heading? If yes, there is a way to accommodate this for you.
Search in yamon.sh for
Code:
#if [ "$iface" != "$LAN_IFACE" ] ; then continue; fi
Remove the `#` at the beginning of the line
NB - you will have to do this for all 3 occurrences of this line.
Then stop and restart yamon.sh
It's hard to tell without seeing things but I think this'll give you want you want.
Sorry you had problems with the initial version(s) of 2. I've added an option within the v2.0 config.file that allows you to turn this feature on/off without having to restart the script (or edit the script file).
Al
Thanks Al. My setup may be a bit atypical. I have a guest network that is open and two of my (many) security cameras are on that because they don't comply with the WPA2 security I use on my main network. The camera in question is on w10.1 (the guest network) while the others are on br0. The weird thing is I have two cameras on w10.1 and one of them shows almost no traffic while the other shows like 5GB per day. Both are transmitting constantly to the PC so I don't understand that. If I comment out that line, from what I read in that line, it'll ignore everything on the guest network which is not what I want. Correct?
Also, I'd love to be able to monitor both LAN and WAN traffic but I can't find anywhere in the script where it finds and uses only the WAN port. Can LAN traffic be monitored too? Sometimes I have so much going on WRT the LAN traffic that a DVR starts skipping, etc. I'd like to be able to see what's hogging the LAN. Still running 1.0.17b because v2 just wasn't updating properly, the device list didn't get updated properly, etc.
Love this tool. Its the most simple and easy to install one out there that gives me what I want.
I read you were working on adding hardware specs (CPU, Memory , yada yada) which would be great. Another suggestion, add a log viewer UI. Something that can parse the logs in a decent display in a webbrowser (better than the default setting up a link to temp/hmtl). Something like Logstash, but much lighter.
Love this tool. Its the most simple and easy to install one out there that gives me what I want.
I read you were working on adding hardware specs (CPU, Memory , yada yada) which would be great. Another suggestion, add a log viewer UI. Something that can parse the logs in a decent display in a webbrowser (better than the default setting up a link to temp/hmtl). Something like Logstash, but much lighter.
Thank you for all your work!
Chris
Chris - thanks for the feedback and also the feature suggestion.
I do not have the server log option enabled on my router (presumably you do) and I haven't take a look at Logstash yet. Were you looking/hoping for just a viewer for the logs or some sort of aggregation too (I expect that'd be a #lot# more work)?
Posted: Thu Apr 17, 2014 8:55 Post subject: users.file
Very useful program, thank you.
I have a couple of questions:
- my users.file keeps getting overwritten every few minutes. I was under the impression that I could make changes to it and it would stay constant. Is that not correct? I ended up changing /tmp/www/data/users.html, and get what I want in the reports in the web interface, but won't /tmp be lost upon reboot? (Update: /tmp/www/data/users/html also gets overwritten once in a while by yamon, so I really have no idea how to make my changes to users.file persistent)
- entries in users.file include dynamic DHCP IP addresses. Wouldn't this cause problems if these IP addresses change from day to day? In other words shouldn't users.file map only MAC addresses to user/name lables, and not IP addresses?
Last edited by routesforall on Thu Apr 17, 2014 15:24; edited 1 time in total
Chris - thanks for the feedback and also the feature suggestion.
I do not have the server log option enabled on my router (presumably you do) and I haven't take a look at Logstash yet. Were you looking/hoping for just a viewer for the logs or some sort of aggregation too (I expect that'd be a #lot# more work)?
Al[/quote]
I agree Logstash would be a lot of work.
Following this method below, it would be great if Ymon could have a module to formate it nicer. Right now is just is a big mess in the browser window.
Posted: Thu Apr 17, 2014 15:21 Post subject: Re: users.file
routesforall wrote:
Very useful program, thank you.
I have a couple of questions:
- my users.file keeps getting overwritten every few minutes. I was under the impression that I could make changes to it and it would stay constant. Is that not correct? I ended up changing /tmp/www/data/users.html, and the reports in the web interface are much better, but won't /tmp be lost upon reboot?
- entries in users.file include dynamic DHCP IP addresses. Wouldn't this cause problems if these IP addresses change from day to day? In other words shouldn't users.file map only MAC addresses to user/name lables, and not IP addresses?
@routesforall - you should be edit user.file (which'll be in `/opt/YAMon/data` if you've used the default installation location). Once you make the change, the script will detect that the md5 signature for the file has changed and will copy the updated file to `/tmp/www/data/users.html`.
This gives you two reason for not editing the file in `/tmp/www/data/`
As you point out, if the router reboots, the changes'll be lost and also if a new device is added to `users.file`, you changes will also get clobbered.
If your changes in `/opt/YAMon/data` are being overwritten, I'll need more info...
Do you have a network range extender or bridge in your network?
Do you have logging turned on in your config.file?
If no, please open that file and set `_log2file=1` & `_loglevel=0` (don't forget to save the changes). You do not have to restart the script, it will automatically pick up those changes on the fly. Let things run for 5-10min; then edit your users.file and let things again (at least until the changes get over-written). Once that is done, PM the log file to me and I'll hopefully be able to see what is going on).
WRT DHCP - because the IP addresses can change, I use MAC addresses as the unique identifier throughout the script, data files and the reports. I have to keep the associated IP address in users.file because the uploads and downloads are actually tracked by IP address (from the `iptables` function call). I also detect whether the IP address changes, and update user.file as appropriate.
BTW - in YAMon v2, the user/device info is stored in JavaScript structures (in users.js) and I now also capture the date the device was added and also the date of the last change.
Posted: Thu Apr 17, 2014 15:33 Post subject: Re: users.file
al_c wrote:
If your changes in `/opt/YAMon/data` are being overwritten, I'll need more info...
Do you have a network range extender or bridge in your network?
Yes, I do have an extender in my network. The main router is the one running dd-wrt. Would this affect things? The extender runs its own proprietary firmware.
al_c wrote:
Do you have logging turned on in your config.file?
If no, please open that file and set `_log2file=1` & `_loglevel=0` (don't forget to save the changes). You do not have to restart the script, it will automatically pick up those changes on the fly. Let things run for 5-10min; then edit your users.file and let things again (at least until the changes get over-written).
Okay, will do and PM you in a bit.
al_c wrote:
BTW - in YAMon v2, the user/device info is stored in JavaScript structures (in users.js) and I now also capture the date the device was added and also the date of the last change. Al
That's great. Is v2 in development still or can I download it from somewhere? I still can't figure out how to search for posts in a single thread here at the dd-wrt.com forum!
Posted: Thu Apr 17, 2014 15:51 Post subject: Re: users.file
routesforall wrote:
al_c wrote:
If your changes in `/opt/YAMon/data` are being overwritten, I'll need more info...
Do you have a network range extender or bridge in your network?
Yes, I do have an extender in my network. The main router is the one running dd-wrt. Would this affect things? The extender runs its own proprietary firmware.
al_c wrote:
Do you have logging turned on in your config.file?
If no, please open that file and set `_log2file=1` & `_loglevel=0` (don't forget to save the changes). You do not have to restart the script, it will automatically pick up those changes on the fly. Let things run for 5-10min; then edit your users.file and let things again (at least until the changes get over-written).
Okay, will do and PM you in a bit.
al_c wrote:
BTW - in YAMon v2, the user/device info is stored in JavaScript structures (in users.js) and I now also capture the date the device was added and also the date of the last change. Al
That's great. Is v2 in development still or can I download it from somewhere? I still can't figure out how to search for posts in a single thread here at the dd-wrt.com forum!
Much like this one, V2 is a work in progress... it will evolve as folks like you ask for new features.
I (and several willing guinea pigs) have been running V2 for sometime now. If I've learned anything over the past year or so, it is that there are a lot of subtle differences between different DD-WRT firmware versions and things don't always work on all routers (like they do on mine). I believe we've shaken out all of the major issues now. I've set things up so that V1 & V2 can run in parallel with each other (this'll add somewhat to the load on your router but provides a good comparison of the two).
You can find new download and lots of discussion here: http://www.dd-wrt.com/phpBB2/viewtopic.php?t=259806&highlight= (I've also updated the initial message in this thread to include a link to the new version too)... thanks for bringing this to my attention
Given that you've gone through the V1 process, hopefully it'll be easier to get going with V2. I have included scripts that will convert you V1 data files to the new format used by V2 so you don't have to start completely from scratch.
WRT to the extender - I have a vague recollection about a posting somewhere in this thread... someone noting that their extender was causing multiple IP addresses changes which caused problems in users.file... I do not recall what (if anything) I was able to do about that...
I've made significant numbers of changes to both the structure of the data files and the way in which they're updated so maybe (hopefully?) it won't be a problem in V2.
Posted: Wed Apr 23, 2014 19:46 Post subject: Re: users.file
[quote="al_c"]
routesforall wrote:
WRT to the extender - I have a vague recollection about a posting somewhere in this thread... someone noting that their extender was causing multiple IP addresses changes which caused problems in users.file... I do not recall what (if anything) I was able to do about that...Al
Would this cause the users.file to keep getting overwritten by YAMon frequently?
Posted: Wed Apr 23, 2014 20:40 Post subject: Re: users.file
[quote="routesforall"]
al_c wrote:
routesforall wrote:
WRT to the extender - I have a vague recollection about a posting somewhere in this thread... someone noting that their extender was causing multiple IP addresses changes which caused problems in users.file... I do not recall what (if anything) I was able to do about that...Al
Would this cause the users.file to keep getting overwritten by YAMon frequently?
Here is the text from a message back in Feb:
PeterS-ZH wrote:
Yes, I was editing the one in /opt/YAMon/data and the changes were not in there long enough to see them with 'less users.file' after quitting the editor.
NOW, in the meantime I have shut down the script and manually adjusted all the IP addresses that were being reported in the log file in the lines
Code:
Updated IP address for: 74:e2:xx:xx:xx:xx -> from: 192.168.1.119 - to: 192.168.102.139
and then I restarted the script.
Now I'm able to edit the file users.file while the script is running and the changes are not overwritten. It seems that if one (or many) of the IP addresses in the file users.file does not correspond to the actual address of the device, then the script does something to the file that inhibits any other editing activities and reports the discrepancy of IP addresses in the log file. Unfortunately, the IP addresses in the file users.file are not updated and therefore the situation of a discrepancy within the addresses is still the same the next tine the script checks. And naturally the editing is then still not possible.
Does Peter's description help at all?
BTW - Do you mean users.js (V2) or the users.file from V1?
I ask because the technique I'm using in the V2 script is a safer way to do the updates (I think).
In yamon2.sh (V2), I made a number of changes to the function that updates `users.js`... in particular, I currently use a `sed` call to do an inline update of [just] that particular record when its IP address changes. In the V1 script, when an IP address changed, I read the entire file, updated the record and then wrote all of the infor back to disk. If a number of other IP addresses had changed, this (conceivably) could result in subsequent updates reading an `old` copy of the file before the previous update had finished writing (and this could/would clobber the previous changes)... of course, I've not tested or been able to prove this.
NB - I actually use a different approach for the hourly usage files. For those, I read the file once (before entering the loop) and then make the updates in memory before writing all of the info back to disk at once (after the loop) - i.e., I am trying to reduce the number of times that I read from and write to the disk. I did not do this for users.js because (in my experience) once the file has been created, there are relatively few changes. However, based upon your feedback, I may have to change this...
I don't know if there is not a pb en the calcul , for daily all it's ok but for monthly i have a gap .
yesterday i have 13gb in download, but when i goes monthly i haven't them.
I don't know if there is not a pb en the calcul , for daily all it's ok but for monthly i have a gap .
yesterday i have 13gb in download, but when i goes monthly i haven't them.
Can you PM me your hourly data file from yesterday and also the monthly file?
Is your time zone set properly on your router? The daily usage results are added to the monthly file just after midnight, if the timezone is wrong then that could be the issue.
Did your router reboot last night? Perhaps at about midnight? If yes, that would have interfered with the calculations.
Regardless, there is a file called h2m.sh in the Setup archive. If you give it execute permission and then run it, it will parse through the hourly data files and make a new monthly file... the new file does *not* overwrite your existing file so it is safe to run.
To run this file
- open Putty (or equivalent)
- (again, make sure it has execute permissions)
- enter /opt/YAMon2/Setup/h2m.sh [_ispBillingDay] [month] - e.g.,
> /opt/YAMon2/Setup/h2m.sh 5 10
- then wait.
It will take several minutes as the script reads through the data files from Oct 5th through to Nov 4th.
A new file named 2014-10-05-mac_data2.js will be created in your data directory
If you are not using the default data directory, you will have to update line #21 to point to your location.
_baseDir="/opt/YAMon2/"
ok many thanks for your answer and your job i'll try your script tonight . my router never reboot form cron task or other .
I'l verify the time zone it's perhaps the promblem .
and i'll use the other post.
