Backup settings and restore them to even different hardware.

Post new topic   Reply to topic    DD-WRT Forum Forum Index -> Broadcom SoC based Hardware
Goto page 1, 2, 3 ... 20, 21, 22  Next
Author Message
frater
DD-WRT Guru


Joined: 07 Jun 2006
Posts: 2777

PostPosted: Sun Dec 21, 2008 11:35    Post subject: Backup settings and restore them to even different hardware. Reply with quote
When DD-WRT does a backup of the settings this is done in a binary way. Even if no variables have changed and the hardware has not changed, it is not totally safe to restore this binary backup on another version.

I'm saving these variables to a script. This was not my idea, but the other implementation on this forum was saving more than it should.

This script also creates some variables that don't really exist. I previously wrote a script that was able to avoid this, but it had a terrible speed-impact. The variables it creates (makes up) comes from data containing a "=". This really is not a problem.

I'm deliberately skipping some variables that contain hardware-specific data. This way it is safe to restore these settings on different hardware.
After an upgrade you can run this restore-script and be up and running in no time.

Code:
#!/bin/sh
#
# This shell script creates a shell file with lines of the form
# nvram set x="y"
# for every nvram variable found from
# nvram show
#
DATE=`date +%m%d%Y`
MAC=`nvram get lan_hwaddr | tr -d ":"`
FILE=${MAC}.${DATE}
CUR_DIR=`dirname $0`
FOLDER=/opt/var/backups
TO_ALL=${FOLDER}/${MAC}.${DATE}.all.sh
TO_INCLUDE=${FOLDER}/${MAC}.${DATE}.essential.sh
TO_EXCLUDE=${FOLDER}/${MAC}.${DATE}.dangerous.sh
FTPS=ftp://192.168.10.210/backups
USERPASS=user:pass

nvram show 2>/dev/null | egrep '^[a-zA-Z].*=' | awk -F= '{print $1}' | grep -v "[ /+<>,:;]" | sort -u >/tmp/all_vars

#
echo -e "#!/bin/sh\n#\necho \"Write variables\"\n" | tee -i ${TO_EXCLUDE} | tee -i  ${TO_ALL} > ${TO_INCLUDE}

cat /tmp/all_vars | while read var
do
  if echo ${var} | grep -q -f "${CUR_DIR}/vars_to_skip" ; then
    bfile=$TO_EXCLUDE
  else
    bfile=$TO_INCLUDE
  fi

  # get the data out of the variable
  data=`nvram get ${var}`
  if [ "${data}" == "" ] ; then
    echo -e "nvram set ${var}="  | tee -ia  ${TO_ALL} >> ${bfile}
  else
    # write the var to the file and use \ for special chars: (\$`")
    echo -en "nvram set ${var}=\"" | tee -ia ${TO_ALL} >> ${bfile}
    echo -n "${data}" | sed 's/\\/\\\\/g' | sed 's/`/\\`/g' | sed 's/\$/\\\$/g' | sed 's/\"/\\"/g' | tee -ia  ${TO_ALL} >> ${bfile}
    echo -e "\"" | tee -ia  ${TO_ALL} >> ${bfile}
  fi

done

rm /tmp/all_vars

echo -e "\n# Commit variables\necho \"Save variables to nvram\"\nnvram commit"  | tee -ia  ${TO_ALL} | tee -ia  ${TO_EXCLUDE} >> ${TO_INCLUDE}
chmod +x ${TO_ALL}
chmod +x ${TO_INCLUDE}
chmod +x ${TO_EXCLUDE}

tar cpf - -C / "${TO_INCLUDE}" 2>/dev/null | /opt/bin/gzip -c |  /opt/bin/curl -s -u ${USERPASS} "${FTPS}/${FILE}.essential.sh.tgz" -T -
tar cpf - -C / "${TO_EXCLUDE}" 2>/dev/null | /opt/bin/gzip -c |  /opt/bin/curl -s -u ${USERPASS} "${FTPS}/${FILE}.dangerous.sh.tgz" -T -
tar cpf - -C / "${TO_ALL}" 2>/dev/null | /opt/bin/gzip -c |  /opt/bin/curl -s -u ${USERPASS} "${FTPS}/${FILE}.all.sh.tgz" -T -

vars_to_skip
Code:
DD_BOARD
^board
browser_method
^cfe
ct_modules
custom_shutdown_command
^def_
^default_
dist_type
dl_ram_addr
early_startup_command
^et0
^et1
^ezc
generate_key
gozila_action
gpio
^hardware
^is_
^kernel_
lan_default
^lan_hw
^lan_ifname
landevs
manual_boot_nv
misc_io_mode
need_commit
^os_
overclocking
pa0maxpwr
phyid_num
pmon_ver
pppd_pppifname
pppoe_ifname
pppoe_wan_ifname
primary_ifname
probe_blacklist
regulation_domain
rescue
reset_
scratch
sdram
^sh_
^skip
sshd_dss_host_key
sshd_rsa_host_key
startup_command
^wan_default
^wan_hw
^wan_if
^wan_vport
^wandevs
web_hook_libraries
^wifi_
wl0.1_hwaddr
wl0.2_hwaddr
wl0.3_hwaddr
wl0_hwaddr
wl0_ifname
wl0_radioids
wl_hwaddr
wl_ifname
^wlan_

_________________
Asus RT16N + OTRW
Kingston 4GB USB-disk 128 MB swap + 1.4GB ext3 on /opt + 2 GB ext3 on /mnt
Copperjet 1616 modem in ZipB-config
Asterisk, pixelserv & Pound running on router
Another Asus RT16N as WDS-bridge

DD-WRT v24-sp2 vpn (c) 2010 NewMedia-NET GmbH
Release: 12/16/10 (SVN revision: 15758M)


Last edited by frater on Wed Dec 31, 2008 14:47; edited 4 times in total
Sponsor
Yhoni
DD-WRT User


Joined: 08 Mar 2008
Posts: 85

PostPosted: Tue Dec 30, 2008 15:47    Post subject: Reply with quote
Ok mate, thanks for your work.
Donny
DD-WRT Guru


Joined: 13 Nov 2008
Posts: 5266
Location: CENTRAL Midnowhere

PostPosted: Tue Dec 30, 2008 18:32    Post subject: Re: Backup settings and restore them to even different hardw Reply with quote
frater wrote:
I previously wrote a script that was able to avoid this, but it had a terrible speed-impact.


As a non programmer Crying or Very sad I could use a brief explanation of what this does...eg. where does it store the data? Does it backup access restrictions?

Also, could you explain what that speed impact was....throughput speed?

I appreciate your persistant efforts to resolve this shortcoming in dd-wrt. Thanks.

_________________
Warning: I'm "out of my element!"
http://www.youtube.com/watch?v=MjYJ7zZ9BRw&NR=1

Peacock Thread Sticky- Just read it! (Anyone using SP1 will be taken out back and shot)
http://www.dd-wrt.com/phpBB2/viewtopic.php?t=51486
frater
DD-WRT Guru


Joined: 07 Jun 2006
Posts: 2777

PostPosted: Tue Dec 30, 2008 20:52    Post subject: Reply with quote
Here it will fill /tmp/all_vars with all the variables being used (sorted)...
Code:
nvram show 2>/dev/null | grep  "=" | awk -F= '{print $1}' | grep -v "[ /+<>,:;]" | sort -u >/tmp/all_vars

It will now parse that file and make 3 shell scripts. One will contain every variable. If a variable is inside the file "vars_to_skip" it will not go into the "essential" file.
Code:
cat /tmp/all_vars | while read var
do
...
done

After parsing it will use curl to upload it to an FTP-server

_________________
Asus RT16N + OTRW
Kingston 4GB USB-disk 128 MB swap + 1.4GB ext3 on /opt + 2 GB ext3 on /mnt
Copperjet 1616 modem in ZipB-config
Asterisk, pixelserv & Pound running on router
Another Asus RT16N as WDS-bridge

DD-WRT v24-sp2 vpn (c) 2010 NewMedia-NET GmbH
Release: 12/16/10 (SVN revision: 15758M)
frater
DD-WRT Guru


Joined: 07 Jun 2006
Posts: 2777

PostPosted: Tue Dec 30, 2008 20:57    Post subject: Reply with quote
Code:
nvram show 2>/dev/null | grep  "=" | awk -F= '{print $1}' | grep -v "[ /+<>,:;]" | sort -u >/tmp/all_vars
This one-liner was a while loop in which it also checked if the variable found wasn't by chance part of the data of the previously found variable containing an "=".

It was taking 2~3 minutes.

_________________
Asus RT16N + OTRW
Kingston 4GB USB-disk 128 MB swap + 1.4GB ext3 on /opt + 2 GB ext3 on /mnt
Copperjet 1616 modem in ZipB-config
Asterisk, pixelserv & Pound running on router
Another Asus RT16N as WDS-bridge

DD-WRT v24-sp2 vpn (c) 2010 NewMedia-NET GmbH
Release: 12/16/10 (SVN revision: 15758M)
eXisor
DD-WRT User


Joined: 08 Jun 2006
Posts: 95

PostPosted: Tue Jan 20, 2009 7:00    Post subject: Reply with quote
@frater:

Thank you for sharing the script with us, but I think many people are daunted by it since there are no installation or usage instructions, and because the curl and FTP part is so specific to your setup.

For those of us who don't know bash scripting (me), it'd be really helpful to have an idiots guide to using it in the various dd-wrt storage scenarios.

In particular, (I've worked out most of it), but I'd have liked to know:

1) Where do you suggest we put the backup_nvram.sh script and vars file (I'm using /opt/var/backups). The script looks for the vars file in CURRENT_DIR, so if one puts them in the same folder, one has to navigate to that folder before running the script.
2) What else does the script assume. FI, that we have /opt mounted and have created a /opt/var/backups folder. (Or have an FTP server at 192.168.1.*** etc)
3) If we mount /opt to an SD/MMC/USB HDD we can/should delete the FTP part. (Perhaps a START FTP CUT HERE and END FTP CUT HERE comment?)
4) To backup - we navigate to the folder and run the backup_nvram.sh script (which creates three dated files in in /opt/var/backups).
5) To restore - we navigate to /opt/var/backups and run the latest essentials script.
6) The other files are for...? Completeness?

Instructions along these lines would have saved bash scripting noob me, in particular, a lot have time and made using the script a synch.

Thank you.

_________________
WRT54GL v1.1 - dd-wrt V24 pre-sp2
E3000 - build 14929
strfr
DD-WRT User


Joined: 21 Jan 2008
Posts: 192

PostPosted: Tue Jan 20, 2009 12:32    Post subject: Reply with quote
Hi frater, thanks for the script but as eXisor has said I think it would be great if you will post kind of newie guide, without FTP uploading part so everyone can run it and copy the right file from the /var/tmp folder

anyway thanks for sharing buddy
zikronix
DD-WRT User


Joined: 21 Feb 2008
Posts: 118

PostPosted: Sat Jan 31, 2009 15:01    Post subject: Reply with quote
so i can use this after I upgrade my firmware, then do a facrtory reset to restore all my settings?

where does it save the file...I dont see any calls to ftp like it was discussed.
freonchill
DD-WRT Guru


Joined: 17 Jul 2006
Posts: 2056

PostPosted: Sat Jan 31, 2009 17:16    Post subject: Reply with quote
but i thought saving/restoring backups from different firmwares (let alone different hardware) would possibly cause a bricked router?

is this safe?

_________________
2x WRT54G v5, 2x WRT54G v2
1x WRT54G-TM
1x WRT54GL
2x BUFFALO WHR-G54S
1x La Fonera
FON Client Bridge tutorial
frater
DD-WRT Guru


Joined: 07 Jun 2006
Posts: 2777

PostPosted: Sat Jan 31, 2009 17:53    Post subject: Reply with quote
zikronix wrote:
so i can use this after I upgrade my firmware, then do a facrtory reset to restore all my settings?

where does it save the file...I dont see any calls to ftp like it was discussed.


It's called efficient programming:
Code:
tar cpf - -C / "${TO_INCLUDE}" 2>/dev/null | /opt/bin/gzip -c |  /opt/bin/curl -s -u ${USERPASS} "${FTPS}/${FILE}.essential.sh.tgz" -T -

_________________
Asus RT16N + OTRW
Kingston 4GB USB-disk 128 MB swap + 1.4GB ext3 on /opt + 2 GB ext3 on /mnt
Copperjet 1616 modem in ZipB-config
Asterisk, pixelserv & Pound running on router
Another Asus RT16N as WDS-bridge

DD-WRT v24-sp2 vpn (c) 2010 NewMedia-NET GmbH
Release: 12/16/10 (SVN revision: 15758M)
frater
DD-WRT Guru


Joined: 07 Jun 2006
Posts: 2777

PostPosted: Sat Jan 31, 2009 17:55    Post subject: Reply with quote
freonchill wrote:
but i thought saving/restoring backups from different firmwares (let alone different hardware) would possibly cause a bricked router?

is this safe?


I'm not restoring a backup-file.
I'm setting all the parameters one by one, but skip those that are hardware dependent....

_________________
Asus RT16N + OTRW
Kingston 4GB USB-disk 128 MB swap + 1.4GB ext3 on /opt + 2 GB ext3 on /mnt
Copperjet 1616 modem in ZipB-config
Asterisk, pixelserv & Pound running on router
Another Asus RT16N as WDS-bridge

DD-WRT v24-sp2 vpn (c) 2010 NewMedia-NET GmbH
Release: 12/16/10 (SVN revision: 15758M)
freonchill
DD-WRT Guru


Joined: 17 Jul 2006
Posts: 2056

PostPosted: Sat Jan 31, 2009 18:58    Post subject: Reply with quote
oh, sorry - i missunderstood.
_________________
2x WRT54G v5, 2x WRT54G v2
1x WRT54G-TM
1x WRT54GL
2x BUFFALO WHR-G54S
1x La Fonera
FON Client Bridge tutorial
frater
DD-WRT Guru


Joined: 07 Jun 2006
Posts: 2777

PostPosted: Sat Jan 31, 2009 19:16    Post subject: Reply with quote
one by one.... in 20 seconds Wink
_________________
Asus RT16N + OTRW
Kingston 4GB USB-disk 128 MB swap + 1.4GB ext3 on /opt + 2 GB ext3 on /mnt
Copperjet 1616 modem in ZipB-config
Asterisk, pixelserv & Pound running on router
Another Asus RT16N as WDS-bridge

DD-WRT v24-sp2 vpn (c) 2010 NewMedia-NET GmbH
Release: 12/16/10 (SVN revision: 15758M)
zikronix
DD-WRT User


Joined: 21 Feb 2008
Posts: 118

PostPosted: Sun Feb 01, 2009 4:54    Post subject: Reply with quote
so this will back up the settings too?

do i run this from the shell or from the gui because it never prompted me to go to any ftp despite the coding.

and then i shelled in anc couldnt find said file
frater
DD-WRT Guru


Joined: 07 Jun 2006
Posts: 2777

PostPosted: Sun Feb 01, 2009 7:33    Post subject: Reply with quote
It's a script which should be stored on your router. Maybe jffs or opt.
Giving your questions I don't think you have the basic Linux skills to pull it off.

It's a script which writes 3 scripts that can recover your settings. These 3 scripts are FTP'd to an FTP-server for which you have the credentials.
This needs the package "curl" to be installed on your router.

I don't think you know how to install curl (optware), but that's not really a problem. It will just not FTP those 3 scripts. These scripts will then be on your router and you can get them with winSCP.

_________________
Asus RT16N + OTRW
Kingston 4GB USB-disk 128 MB swap + 1.4GB ext3 on /opt + 2 GB ext3 on /mnt
Copperjet 1616 modem in ZipB-config
Asterisk, pixelserv & Pound running on router
Another Asus RT16N as WDS-bridge

DD-WRT v24-sp2 vpn (c) 2010 NewMedia-NET GmbH
Release: 12/16/10 (SVN revision: 15758M)
Goto page 1, 2, 3 ... 20, 21, 22  Next Display posts from previous:    Page 1 of 22
Post new topic   Reply to topic    DD-WRT Forum Forum Index -> Broadcom SoC based Hardware All times are GMT

Navigation

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You can attach files in this forum
You can download files in this forum