Country Blocking

Post new topic   Reply to topic    DD-WRT Forum Index -> Broadcom SoC based Hardware
Goto page Previous  1, 2, 3, 4, 5, 6, 7, 8, 9, 10  Next
Author Message
SirSilentBob
DD-WRT User


Joined: 09 Oct 2007
Posts: 258

PostPosted: Thu Dec 10, 2015 15:07    Post subject: Thanks! Reply with quote
Thanks everyone who worked on, tested, and improved this script! Working on an R7000.

(Side note, now I'll just have to find a file that blocks all those microsoft IP ranges that people like to block to minimize all the spying that windows 10 does. Razz )

_________________
Netgear R7000 w/r31780M <KONG> build

Netgear R6700 (Un-opened with stock. My backup/emergency router if the R7000 takes a dump...)

2x Buffalo WHR-HP-GN 28493 (Used for 2.4 Ghz bridge when needed.)

Asus WL-500g Premium (1x v1 & 1x v2) (Still have, but retired for now.)

1x Linksys WRT54G v8 >>DD-WRT v24SP1 (The other routers needed something to point at and make fun of.)
Sponsor
SirSilentBob
DD-WRT User


Joined: 09 Oct 2007
Posts: 258

PostPosted: Mon Dec 14, 2015 5:14    Post subject: Modified the main shell script a little... Reply with quote
To Badmoon & JAMESMTL:

I made a small modification to your main script file that allows a user to put a custom.zone file in the "zones" directory. That way a user can manually add custom ip ranges to a file, and have it processed and handled the same way as the online-hosted zone files.

If there is a "better" or "more efficient" way to do what I did, please feel free to laugh at me and make changes.

I'll paste the modified file below. Do you want to edit your original post, and include this modified version as well as the diagnostic shell file also, so users can choose to use just the hosted zone files, or use hosted as well as custom.zone, and can easily test the results?

**ALSO: I made a simple small shell file that runs all of the diagnostic commands that you have suggested people run, JAMESMTL. That will also be posted below. (It simplified my process of testing/diagnosing/verifying since it does the 5 or 6 commands back to back, and therefore all the info is clustered together.)


ipblock.sh (modified for custom.zone):

Code:

#!/bin/sh

#set -x

### Block all traffic from listed. Use ISO code ###
ISO="cn-aggregated tw-aggregated kp-aggregated ru-aggregated ir-aggregated"
CLOCAL="custom"

#Testing
#ISO="kr-aggregated"

### Set PATH ###
IPT=/usr/sbin/iptables
WGET=/usr/bin/wget
EGREP=/bin/egrep
LOCKFILE=/tmp/ipblocklock.txt

### No editing below ###
inSPAMLIST="countrydropin"
outSPAMLIST="countrydropout"
ZONEROOT="/tmp/mnt/sda1/ipblock/zones"
DLROOT="http://www.ipdeny.com/ipblocks/data/aggregated"
iBL="/tmp/mnt/sda1/ipblock/zones/ipblockin.sh"
oBL="/tmp/mnt/sda1/ipblock/zones/ipblockout.sh"

if [ -e ${LOCKFILE} ] && kill -0 `cat ${LOCKFILE}`; then
    echo "Lock file exist.. exiting"
    exit
fi

# make sure the lockfile is removed when we exit and then claim it
trap "rm -f ${LOCKFILE}; exit" INT TERM EXIT
echo $$ > ${LOCKFILE}

cleanOldRules(){
$IPT -F countrydropin
$IPT -F countrydropout
}

# create a dir
[ ! -d $ZONEROOT ] && /bin/mkdir -p $ZONEROOT

# clean old rules
cleanOldRules
rm -f $iBL
rm -f $oBL

echo '*filter' > $iBL
echo '*filter' > $oBL

for c in $ISO
do
        # local zone file
        tDB=$ZONEROOT/$c.zone

        # get fresh zone file
        $WGET -T 30 -O $tDB $DLROOT/$c.zone

        awk -v inSPAMLIST=$inSPAMLIST '{print "-A "inSPAMLIST" -s "$1" -j DROP"}' $tDB >> $iBL
        awk -v outSPAMLIST=$outSPAMLIST '{print "-A "outSPAMLIST" -d "$1" -j REJECT"}' $tDB >> $oBL
done

for c in $CLOCAL
do
        # local custom zone file
        tDB=$ZONEROOT/$c.zone

        awk -v inSPAMLIST=$inSPAMLIST '{print "-A "inSPAMLIST" -s "$1" -j DROP"}' $tDB >> $iBL
        awk -v outSPAMLIST=$outSPAMLIST '{print "-A "outSPAMLIST" -d "$1" -j REJECT"}' $tDB >> $oBL
done

echo 'COMMIT' >> $iBL
echo 'COMMIT' >> $oBL

iptables-restore -n < $iBL
iptables-restore -n < $oBL

rm -f ${LOCKFILE}




ipbtest.sh (for testing/diagnostic purposes):

Code:
#!/bin/sh

iptables -vnL INPUT
iptables -vnL FORWARD
iptables -vnL countrydropin | tail -n 5
iptables -vnL countrydropout | tail -n 5
iptables -vnL countrydropin | awk '{ if ($1 > 0) print $0 }'
iptables -vnL countrydropout | awk '{ if ($1 > 0) print $0 }'

_________________
Netgear R7000 w/r31780M <KONG> build

Netgear R6700 (Un-opened with stock. My backup/emergency router if the R7000 takes a dump...)

2x Buffalo WHR-HP-GN 28493 (Used for 2.4 Ghz bridge when needed.)

Asus WL-500g Premium (1x v1 & 1x v2) (Still have, but retired for now.)

1x Linksys WRT54G v8 >>DD-WRT v24SP1 (The other routers needed something to point at and make fun of.)
badmoon
DD-WRT Novice


Joined: 22 Jul 2014
Posts: 41

PostPosted: Mon Dec 14, 2015 13:24    Post subject: Reply with quote
Cool update. I switched mine over to ripe.net some time ago because it looked like ipdeny let their domain subscription lapse.

Code:

#!/bin/sh

#set -x

### Block all traffic from listed. Use ISO code ###
ISO="cn tw kp kr ru ir"
#ISO="kp kr"

#Testing
#ISO="kr"

### Set PATH ###
IPT=/usr/sbin/iptables
WGET=/usr/bin/wget
EGREP=/bin/egrep
LOCKFILE=/tmp/ipblocklock.txt

### No editing below ###
inLIST="countrydropin"
outLIST="countrydropout"
ZONEROOT="/opt/ipblock/zones"
DLROOT="https://stat.ripe.net/data/country-resource-list/data.json?resource="
iBL="/opt/ipblock/zones/ipblockin.sh"
oBL="/opt/ipblock/zones/ipblockout.sh"

if [ -e ${LOCKFILE} ] && kill -0 `cat ${LOCKFILE}`; then
    echo "Lock file exist.. exiting"
    exit
fi

# make sure the lockfile is removed when we exit and then claim it
trap "rm -f ${LOCKFILE}; exit" INT TERM EXIT
echo $$ > ${LOCKFILE}

cleanOldRules(){
$IPT -F countrydropin
$IPT -F countrydropout
$IPT -F countrylogin
$IPT -F countrylogout
}

setuplogging(){
#$IPT -X countrylogin
$IPT -N countrylogin
#$IPT -A countrylogin -j LOG --log-prefix "COUNTRY IN DROP "
$IPT -A countrylogin -j DROP
#$IPT -X countrylogout
$IPT -N countrylogout
#$IPT -A countrylogout -j LOG --log-prefix "COUNTRY OUT DROP "
$IPT -A countrylogout -j REJECT
}

# create a dir
[ ! -d $ZONEROOT ] && /bin/mkdir -p $ZONEROOT

# clean old rules
cleanOldRules
rm -f $iBL
rm -f $oBL

setuplogging

echo '*filter' > $iBL
echo '*filter' > $oBL

for c in $ISO
do
        # local zone file
        tDB=$ZONEROOT/$c.zone
        tDBt=$ZONEROOT/$c.zone.temp

        # get fresh zone file
        $WGET -T 30 -O $tDBt $DLROOT$c

        #extract data
        grep -E -o '(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\/([0-9]|[1-3][0-9])' $tDBt > $tDB

        awk -v inLIST=$inLIST '{print "-A "inLIST" -s "$1" -j countrylogin"}' $tDB >> $iBL
        awk -v outLIST=$outLIST '{print "-A "outLIST" -d "$1" -j countrylogout"}' $tDB >> $oBL

done

echo 'COMMIT' >> $iBL
echo 'COMMIT' >> $oBL

iptables-restore -n < $iBL
iptables-restore -n < $oBL

rm -f ${LOCKFILE}
JAMESMTL
DD-WRT Guru


Joined: 13 Mar 2014
Posts: 856
Location: Montreal, QC

PostPosted: Mon Dec 14, 2015 17:23    Post subject: Reply with quote
@SSB

My scripts also make use of whitelists & blacklists so I agree with having the ability for a custom zone. As an example I have a serious distaste for the scans by shodan.io as they make it easy to identify all IPs running any given service such as dropbear etc. so I block IP ranges I know that host their census servers.


@Badmoon

It was a PITA when ipdeny allowed their domain to expire. Internally I have moved to parsing the individual raw RIR zone files and creating aggregated cidr blocks served by an SQL server via HTTP to my various routers. The issue I have with the ripe json query is that occasionally they include ipranges instead of cidr blocks see ru,ir,us zones as examples.
blaser
DD-WRT Guru


Joined: 16 Jul 2006
Posts: 525

PostPosted: Fri Dec 25, 2015 0:20    Post subject: Reply with quote
just upgraded my router to 28598, latest version
wget doesn't accept https from stat.ripe.net
I had to change it to http.
Anyone else?

_________________
Netgear R9000 main router
RAX80 as AP
badmoon
DD-WRT Novice


Joined: 22 Jul 2014
Posts: 41

PostPosted: Sat Dec 26, 2015 15:26    Post subject: Reply with quote
@JAMESMTL

Strange, I haven't seen that issue. Will keep a look out. Thanks!
HalfBit
DD-WRT Guru


Joined: 04 Sep 2009
Posts: 776
Location: AR, USA

PostPosted: Sat Dec 26, 2015 23:07    Post subject: Reply with quote
blaser wrote:
just upgraded my router to 28598, latest version
wget doesn't accept https from stat.ripe.net
I had to change it to http.
Anyone else?

I can confirm this behavior. I had to switch to http/port 80. I am running build 28000M.

_________________
R7000 Nighthawk - DD-WRT v3.0-r50308
R7000 Nighthawk - DD-WRT v3.0-r50308
~~~~~~~~~~Dismantled for learning opportunities~~~~~~~~~~
WRT54Gv2
WRT54Gv8.2
~~~~~~~~~~Other Settings~~~~~~~~~
https://nextdns.io/?from=2d3sq39x
https://pi-hole.net/
https://github.com/DNSCrypt/dnscrypt-proxy
SirSilentBob
DD-WRT User


Joined: 09 Oct 2007
Posts: 258

PostPosted: Sun Dec 27, 2015 8:00    Post subject: Reply with quote
HalfBit wrote:
blaser wrote:
just upgraded my router to 28598, latest version
wget doesn't accept https from stat.ripe.net
I had to change it to http.
Anyone else?

I can confirm this behavior. I had to switch to http/port 80. I am running build 28000M.


I believe that the issue is due to wget not being compiled with HTTPS support. According to: https://www.gnu.org/software/wget/manual/html_node/HTTPS-_0028SSL_002fTLS_0029-Options.html

"Wget must be compiled with an external SSL library."

Do any of you think that opening a ticket and requesting wget being compiled with HTTPS support would go anywhere? I did notice on the RIPE site https://stat.ripe.net/docs/data_api this following text:

Code:
Note on SSL

Although it's still possible to use the RIPEstat Data API on a non-secure connection (ordinary HTTP) we strongly encourage using https. If there's a reason for you why you can't use HTTPS at all, please inform us since we will disable HTTP in the near future.


So it would appear that we will not be able to use just plain old HTTP forever...

Thoughts?

_________________
Netgear R7000 w/r31780M <KONG> build

Netgear R6700 (Un-opened with stock. My backup/emergency router if the R7000 takes a dump...)

2x Buffalo WHR-HP-GN 28493 (Used for 2.4 Ghz bridge when needed.)

Asus WL-500g Premium (1x v1 & 1x v2) (Still have, but retired for now.)

1x Linksys WRT54G v8 >>DD-WRT v24SP1 (The other routers needed something to point at and make fun of.)
HalfBit
DD-WRT Guru


Joined: 04 Sep 2009
Posts: 776
Location: AR, USA

PostPosted: Sun Dec 27, 2015 13:15    Post subject: Reply with quote
SirSilentBob wrote:
I believe that the issue is due to wget not being compiled with HTTPS support. According to: https://www.gnu.org/software/wget/manual/html_node/HTTPS-_0028SSL_002fTLS_0029-Options.html

"Wget must be compiled with an external SSL library."

Do any of you think that opening a ticket and requesting wget being compiled with HTTPS support would go anywhere? I did notice on the RIPE site https://stat.ripe.net/docs/data_api this following text:

Code:
Note on SSL

Although it's still possible to use the RIPEstat Data API on a non-secure connection (ordinary HTTP) we strongly encourage using https. If there's a reason for you why you can't use HTTPS at all, please inform us since we will disable HTTP in the near future.


So it would appear that we will not be able to use just plain old HTTP forever...

Thoughts?

I think it would be worthwhile to start a discussion on and/or create a ticket on TRAC to get SSL with WGET compiled in. More and more sites are switching to HTTPS. I've never been able to create an account on TRAC, though.

@badmoon and/or @JAMESMTL--Any suggestions or thoughts?

_________________
R7000 Nighthawk - DD-WRT v3.0-r50308
R7000 Nighthawk - DD-WRT v3.0-r50308
~~~~~~~~~~Dismantled for learning opportunities~~~~~~~~~~
WRT54Gv2
WRT54Gv8.2
~~~~~~~~~~Other Settings~~~~~~~~~
https://nextdns.io/?from=2d3sq39x
https://pi-hole.net/
https://github.com/DNSCrypt/dnscrypt-proxy
blaser
DD-WRT Guru


Joined: 16 Jul 2006
Posts: 525

PostPosted: Sun Dec 27, 2015 14:37    Post subject: Reply with quote
Agree, https is a must
_________________
Netgear R9000 main router
RAX80 as AP
JAMESMTL
DD-WRT Guru


Joined: 13 Mar 2014
Posts: 856
Location: Montreal, QC

PostPosted: Sun Dec 27, 2015 18:17    Post subject: Reply with quote
use curl instead of wget
HalfBit
DD-WRT Guru


Joined: 04 Sep 2009
Posts: 776
Location: AR, USA

PostPosted: Sun Dec 27, 2015 22:50    Post subject: Reply with quote
JAMESMTL wrote:
use curl instead of wget

Anyone know how to do this? I've tried a few things among the following and always get the 77 error:
Code:
root@R7000:/opt/ipblock# curl --capath /opt/usr/bin --cacert ca-bundle.crt -o opt/ipblock/zones/test.zone.temp https://stat.ripe.net/data/country-resource-list/data.json?resource=cn
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
curl: (77) error setting certificate verify locations:
  CAfile: ca-bundle.crt
  CApath: /opt/usr/bin

I've Googled some, and have made some progress. I'll Google more as time permits.

_________________
R7000 Nighthawk - DD-WRT v3.0-r50308
R7000 Nighthawk - DD-WRT v3.0-r50308
~~~~~~~~~~Dismantled for learning opportunities~~~~~~~~~~
WRT54Gv2
WRT54Gv8.2
~~~~~~~~~~Other Settings~~~~~~~~~
https://nextdns.io/?from=2d3sq39x
https://pi-hole.net/
https://github.com/DNSCrypt/dnscrypt-proxy
JAMESMTL
DD-WRT Guru


Joined: 13 Mar 2014
Posts: 856
Location: Montreal, QC

PostPosted: Sun Dec 27, 2015 23:15    Post subject: Reply with quote
for the purpose of simply getting ripe stats you can use the -k flag instead of setting capath / cacert. When I need cert verification I just make a copy of one of my linux openssl cert folders and set capath to a copy of that dir
HalfBit
DD-WRT Guru


Joined: 04 Sep 2009
Posts: 776
Location: AR, USA

PostPosted: Mon Dec 28, 2015 2:16    Post subject: Reply with quote
JAMESMTL wrote:
for the purpose of simply getting ripe stats you can use the -k flag instead of setting capath / cacert. When I need cert verification I just make a copy of one of my linux openssl cert folders and set capath to a copy of that dir

OK, that worked--thank you.

For my learning, do you or anyone else know how to get the cacert directory, and file working with curl? Links would be great as well, or as a substitute. Smile

_________________
R7000 Nighthawk - DD-WRT v3.0-r50308
R7000 Nighthawk - DD-WRT v3.0-r50308
~~~~~~~~~~Dismantled for learning opportunities~~~~~~~~~~
WRT54Gv2
WRT54Gv8.2
~~~~~~~~~~Other Settings~~~~~~~~~
https://nextdns.io/?from=2d3sq39x
https://pi-hole.net/
https://github.com/DNSCrypt/dnscrypt-proxy
JAMESMTL
DD-WRT Guru


Joined: 13 Mar 2014
Posts: 856
Location: Montreal, QC

PostPosted: Mon Dec 28, 2015 2:53    Post subject: Reply with quote
HalfBit wrote:
JAMESMTL wrote:
for the purpose of simply getting ripe stats you can use the -k flag instead of setting capath / cacert. When I need cert verification I just make a copy of one of my linux openssl cert folders and set capath to a copy of that dir

OK, that worked--thank you.

For my learning, do you or anyone else know how to get the cacert directory, and file working with curl? Links would be great as well, or as a substitute. Smile


set capath & cacert

make sure to use full path for cacert

ex. curl --capath /opt/usr/bin --cacert /opt/usr/bin/ca-bundle.crt .....
Goto page Previous  1, 2, 3, 4, 5, 6, 7, 8, 9, 10  Next Display posts from previous:    Page 8 of 10
Post new topic   Reply to topic    DD-WRT Forum Index -> Broadcom SoC based Hardware All times are GMT

Navigation

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You can attach files in this forum
You can download files in this forum