dd-wrt - Bandwidth Usage Monitor

Post new topic   This topic is locked: you cannot edit posts or make replies.    DD-WRT Forum Index -> Broadcom SoC based Hardware
Goto page Previous  1, 2, 3, 4, 5 ... 40, 41, 42  Next
Author Message
FezJay
DD-WRT User


Joined: 11 Sep 2010
Posts: 178

PostPosted: Sat Oct 30, 2010 3:21    Post subject: Reply with quote
Theliel wrote:
yes, awk work fine, im using this on some scripts

add other clients to the list its fine, the problem are some client that already exist on list are added too with MAC 00:00:00:00:00:00, but with a correct and existing IP.


I get that MAC added as well.

I have always though it was some kind of system MAC that the router uses.

It does use internet usage, hence why I record it.

It seems to be some kind of sys default.

This is not an issue. Its not going to create multiple entries for 00:00..., every time the update encounters it, the IP will be updated.

We will call this MAC, the overhead MAC okay lol. I dont know where it comes from, but it seems to only get used when a new client connects up for the first time.

_________________
dd-wrt eko/V24-K26 15508
Belkin F7D4301
8MB Flash
64MB RAM

Optware (jffs/mnt) 4GB Flash Drive
Sponsor
FezJay
DD-WRT User


Joined: 11 Sep 2010
Posts: 178

PostPosted: Sat Oct 30, 2010 6:20    Post subject: Reply with quote
Hey

Just made some changes on the trunk version of the script.

I have added the ability to control if you would like the system to auto add MAC addresses if they dont exist in your users file.

I have also added some code to decide for specefic MAC addresses if you would like to see their usage data displayed on the monitor.html page.

This is controlled via the users file by the addition of an extra field being added.

e.g. MAC,IP,USER,TYPER,DISPLAY

A Display value of no value or 1 will allow the JS to create a JS entry for the user, but a value of 0 will cause NO JS objects created for that MAC (hence preventing the data from being displayed on the web page).

When I have completed my testing (made sure it works), I will branch to 0.9.

_________________
dd-wrt eko/V24-K26 15508
Belkin F7D4301
8MB Flash
64MB RAM

Optware (jffs/mnt) 4GB Flash Drive
Theliel
DD-WRT User


Joined: 08 Jun 2008
Posts: 81

PostPosted: Sat Oct 30, 2010 10:32    Post subject: Reply with quote
About 00:00:00:00:00:00, Yes, I thought the same, but in my case always have the same IP

I have the perception too that monitor dont register absolutely all packets, i have some discrepancies on about 30-40MB on 1.10GB. Im going to make some test and monitor manually iptables too

_________________
My Blog
wrt54GL 1.1 -> Resurrected eko 14471/MMC Mod
E2000 (wrt320n converted) 1.0 -> BS 15962M -> Resurrected WAN<Serial> BS r23919
RT-AC56U -> Merlin 374.42_2
FezJay
DD-WRT User


Joined: 11 Sep 2010
Posts: 178

PostPosted: Sat Oct 30, 2010 11:21    Post subject: Reply with quote
Theliel wrote:
About 00:00:00:00:00:00, Yes, I thought the same, but in my case always have the same IP

I have the perception too that monitor dont register absolutely all packets, i have some discrepancies on about 30-40MB on 1.10GB. Im going to make some test and monitor manually iptables too


Im going to assume that you are using the in built WAN monitor of the DD-WRT that records overall usage?

This is what I have been using as well.

I think that it is using base 1000 to display its stats.

Is your page using 1024 or 1000?

Try changing it to 1000 (in the monitor.html page).

The best way to test this, is to let the script go past Midnight (creates a new day record), then let it run for a few hours.

Unplug the WAN (disconnect router from internet) and wait until the next refresh occurs.

Then compare the values. In my own experience, these values are very close. 1-2 MB differences.

This can be accounted for as every now and then, the BWMON rule must be removed and replaced back at the top of the iptables FORWARD rules (takes milliseconds).

In the time that this takes, you may have lost a few KB's of data (across all clients).

_________________
dd-wrt eko/V24-K26 15508
Belkin F7D4301
8MB Flash
64MB RAM

Optware (jffs/mnt) 4GB Flash Drive
Theliel
DD-WRT User


Joined: 08 Jun 2008
Posts: 81

PostPosted: Sat Oct 30, 2010 11:50    Post subject: Reply with quote
interface.html (monitor) use 1000 by default, i will test with 1024 too.
_________________
My Blog
wrt54GL 1.1 -> Resurrected eko 14471/MMC Mod
E2000 (wrt320n converted) 1.0 -> BS 15962M -> Resurrected WAN<Serial> BS r23919
RT-AC56U -> Merlin 374.42_2
FezJay
DD-WRT User


Joined: 11 Sep 2010
Posts: 178

PostPosted: Sat Oct 30, 2010 12:13    Post subject: Reply with quote
Theliel wrote:
interface.html (monitor) use 1000 by default, i will test with 1024 too.


This is the improved startup script. It allows you to change the script path in the html file from here, instead of having to change it manually.

#!/bin/sh
while [ ! -f /tmp/monitor-started.lock ] && [ ! -f /tmp/monitor-stop ]; do
if [ ! -f /tmp/www/monitor.html ]; then
sed 's/src=.*>/src="http:\/\/192.168.1.1\/user\/user_details.js"><\/script>/' /mnt/monitor/setup/monitor.html > /tmp/www/monitor.html
fi
/mnt/monitor/setup/bw_monitor.sh 30 3 30 4 /mnt/monitor/setup/users.file /mnt/monitor/mac_usage.backup /mnt/monitor/history/ 1 '' 1 1 1 1 1 1
if [ ! -f /tmp/monitor-started.lock ]; then
sleep 10
fi
done

_________________
dd-wrt eko/V24-K26 15508
Belkin F7D4301
8MB Flash
64MB RAM

Optware (jffs/mnt) 4GB Flash Drive
FezJay
DD-WRT User


Joined: 11 Sep 2010
Posts: 178

PostPosted: Sat Oct 30, 2010 12:36    Post subject: Reply with quote
If any one is interested.

You can download the latest monitor.html file from the trunk.

It has a few little things that are useful.

1. Allows you to stop start the refresh countdown
2. Allows you to change the byte base value (between 1000 and 1024). This is done via an HTML query parameters.

You will see it when you click 'refresh' in the url bar.

3. I keep track of what day you were looking at before the page refreshed. This is also done using a html query parameter.

_________________
dd-wrt eko/V24-K26 15508
Belkin F7D4301
8MB Flash
64MB RAM

Optware (jffs/mnt) 4GB Flash Drive
FezJay
DD-WRT User


Joined: 11 Sep 2010
Posts: 178

PostPosted: Sat Oct 30, 2010 14:41    Post subject: Reply with quote
I have had a query in regards to helping someone set this up. (musafir)

My response is below, but I think this person will need a little more direction.

Heres the startup script you should use:

You will need to update the file paths to match your current setup.

I have done as much as I can in regards to the actual setup script.

The following assumes you have NO persistant storage

Example

#!/bin/sh
wget http://code.google.com/p/lal-projects/source/browse/tags/V0.8_wrt-bandwidth-statistics/bw_monitor.sh -O /tmp/bw_monitor.sh && chmod +x /tmp/bw_monitor.sh
wget http://code.google.com/p/lal-projects/source/browse/tags/V0.8_wrt-bandwidth-statistics/monitor.html -O /tmp/monitor.html

#INSERT CODE TO RESTORE BACKED UP USAGE FILE. SAVED AS /tmp/mac_usage.backup

#If not using dnsmasq file
echo 'MAC,IP,USER,TYPE' > /tmp/users.file
echo 'MAC,IP,USER,TYPE' >> /tmp/users.file
echo 'MAC,IP,USER,TYPE' >> /tmp/users.file

while [ ! -f /tmp/monitor-started.lock ] && [ ! -f /tmp/monitor-stop ]; do
if [ ! -f /tmp/www/monitor.html ]; then
sed 's/src=.*>/src="http:\/\/192.168.1.1\/user\/user_details.js"><\/script>/' /tmp/monitor.html > /tmp/www/monitor.html
fi
/tmp/bw_monitor.sh 30 3 30 4 /tmp/users.file /tmp/mac_usage.backup /tmp/ /tmp/mac_usage.db '' 1 1 1 1 1 1
if [ ! -f /tmp/monitor-started.lock ]; then
sleep 10
fi
done

rm /tmp/monitor.html

Ideally, you would have some sort of cron job that backups the mac_usage.db file (maybe an FTP site etc) and an extra call to place the backed up file back into temp (before the script is started) so it can be restored after subsequent restarts.

After you have upadated the above script, you will need to get this to run on router startup (save it).

if you have jffs enabled, you can save the script as /jffs/etc/config/bw_monitor.startup.

You can also save a startup script from the UI. http://192.168.1.1/Diagnostics.asp (change the IP to your routers)

Copy and paste the above script in (with your changes) and then click 'Save Startup'

_________________
dd-wrt eko/V24-K26 15508
Belkin F7D4301
8MB Flash
64MB RAM

Optware (jffs/mnt) 4GB Flash Drive
musafir
DD-WRT Novice


Joined: 06 Jun 2010
Posts: 10

PostPosted: Sat Oct 30, 2010 15:24    Post subject: Step-by-step: How to setup Bandwidth Monitor Reply with quote
How to set up this, I am a total beginner, if someone could explain how to set up "step by step".

Router is Linksys WRT54G with firmware dd-wrt.

Memory
Total Available
12.7 MB / 16.0 MB
Free
3.6 MB / 12.7 MB
Used
9.1 MB / 12.7 MB
Buffers
1.3 MB / 9.1 MB
Cached
3.8 MB / 9.1 MB
Active
3.4 MB / 9.1 MB
Inactive
1.8 MB / 9.1 MB
Theliel
DD-WRT User


Joined: 08 Jun 2008
Posts: 81

PostPosted: Sat Oct 30, 2010 18:26    Post subject: Reply with quote
Other bug:

All calculation are in bytes, and a 32bit unsigned integer can hold only up 4GB (About 2GB for signed). On actual builds, when any client (or summary) reaches this limit, monitor will show 0 Bytes, and files db a negative number:

Last working check: PC1 -> 1.9GB (on fb files and html)
Next Update: PC 1 -> -1.7GB on db files, 0 on html
Next Update: PC 1 -> -1.5GB...
...
......

a 32bits depth its insufficient, we need work with 64bits integer


currentUsageIn=`expr $currentUsageIn + $newUsageIn`
currentUsageOut=`expr $currentUsageOut + $newUsageOut

But i dont understand, i believed that ddwrt ash worked fine with 64 bits integer


maybe the problem are completlay different?

_________________
My Blog
wrt54GL 1.1 -> Resurrected eko 14471/MMC Mod
E2000 (wrt320n converted) 1.0 -> BS 15962M -> Resurrected WAN<Serial> BS r23919
RT-AC56U -> Merlin 374.42_2
FezJay
DD-WRT User


Joined: 11 Sep 2010
Posts: 178

PostPosted: Sun Oct 31, 2010 0:07    Post subject: Reply with quote
Theliel wrote:
Other bug:

All calculation are in bytes, and a 32bit unsigned integer can hold only up 4GB (About 2GB for signed). On actual builds, when any client (or summary) reaches this limit, monitor will show 0 Bytes, and files db a negative number:

Last working check: PC1 -> 1.9GB (on fb files and html)
Next Update: PC 1 -> -1.7GB on db files, 0 on html
Next Update: PC 1 -> -1.5GB...
...
......

a 32bits depth its insufficient, we need work with 64bits integer


currentUsageIn=`expr $currentUsageIn + $newUsageIn`
currentUsageOut=`expr $currentUsageOut + $newUsageOut

But i dont understand, i believed that ddwrt ash worked fine with 64 bits integer

maybe the problem are completlay different?



I had this problem with the [ $currentUsageIn -gt 0 ] check. If the value was bigger than 2GB then it would fail the check.

All I did was remove checks like this.

Now you are saying that the exp function is also doing somthing strange. This was not seen in my testing, I have values for a day that hit around the 5GB mark.

How do you say that a value is 64 or 32 bit in sh?

Its a non-declarative language, meaning I cannot say int usage = 32323 or long usage = 3434 or decimal usage = 2222.

So, whats the best way around this?

Im not too sure, I will need to do some research.

There must be away, because if you run iptables -L FORWARD -vnx, the numbers assigned to the rules are HUGE. Ive seen them get well into the tens og GB's.

_________________
dd-wrt eko/V24-K26 15508
Belkin F7D4301
8MB Flash
64MB RAM

Optware (jffs/mnt) 4GB Flash Drive
FezJay
DD-WRT User


Joined: 11 Sep 2010
Posts: 178

PostPosted: Sun Oct 31, 2010 0:30    Post subject: Reply with quote
I have just run the following

root@DD-WRT:~# expr 999999999999999999 + 999999999999999999
1999999999999999998

As you can see, expr can handle 2 18 digit values being added togther and its fine with it.

Mmmm, im thinking that those stupid [ value -ge 0 ] checks are somehow converting the type of the variable to 32 bit.

so when we call expr usage + newUsage it thinks its adding 2 32 bit integers together.

Thats my theory Wink, I will run this is a sh script and see if I get similar behaviour.

A

_________________
dd-wrt eko/V24-K26 15508
Belkin F7D4301
8MB Flash
64MB RAM

Optware (jffs/mnt) 4GB Flash Drive
FezJay
DD-WRT User


Joined: 11 Sep 2010
Posts: 178

PostPosted: Sun Oct 31, 2010 0:49    Post subject: Reply with quote
FezJay wrote:
I have just run the following

root@DD-WRT:~# expr 999999999999999999 + 999999999999999999
1999999999999999998

As you can see, expr can handle 2 18 digit values being added togther and its fine with it.

Mmmm, im thinking that those stupid [ value -ge 0 ] checks are somehow converting the type of the variable to 32 bit.

so when we call expr usage + newUsage it thinks its adding 2 32 bit integers together.

Thats my theory Wink, I will run this is a sh script and see if I get similar behaviour.

A



LOL nope, thats not the problem

I have created and run a script with that calculation in it, and it has worked out the 2 large number addition very well.

If you are interested, that value 1999999999999999998 bytes, comes out to be around 1862645149GB's

_________________
dd-wrt eko/V24-K26 15508
Belkin F7D4301
8MB Flash
64MB RAM

Optware (jffs/mnt) 4GB Flash Drive
FezJay
DD-WRT User


Joined: 11 Sep 2010
Posts: 178

PostPosted: Sun Oct 31, 2010 1:22    Post subject: Reply with quote
Hey man

I dont know if it was my change or if it was already working, but I hacked my backup file, and set a HUGE number in there, then I restarted the script, and downloaded a file.

The addition worked fine, my display shows 77.49 TB.

Thats how large the value was.

All I did wad remove a few of the -ge checks and instead performed a not empty check instead, and I set the defaukts to string 0

e.g.

newUsageIn=0 TO newUsageIn='0'

Im not sure if this forces the expr to first work out the length of the string and then forces it to determine that it is not a 32 bit integer.

Its on the trunk if you are keen on testing if it works for you.

A

_________________
dd-wrt eko/V24-K26 15508
Belkin F7D4301
8MB Flash
64MB RAM

Optware (jffs/mnt) 4GB Flash Drive
Theliel
DD-WRT User


Joined: 08 Jun 2008
Posts: 81

PostPosted: Sun Oct 31, 2010 15:34    Post subject: Reply with quote
yep, the new version work fine Wink
_________________
My Blog
wrt54GL 1.1 -> Resurrected eko 14471/MMC Mod
E2000 (wrt320n converted) 1.0 -> BS 15962M -> Resurrected WAN<Serial> BS r23919
RT-AC56U -> Merlin 374.42_2
Goto page Previous  1, 2, 3, 4, 5 ... 40, 41, 42  Next Display posts from previous:    Page 4 of 42
Post new topic   This topic is locked: you cannot edit posts or make replies.    DD-WRT Forum Index -> Broadcom SoC based Hardware All times are GMT

Navigation

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You can attach files in this forum
You can download files in this forum