NEW: Firmware Modification Kit (beta testing)

Post new topic   Reply to topic    DD-WRT Forum Index -> Broadcom SoC based Hardware
Goto page 1, 2, 3 ... 12, 13, 14  Next
Author Message




Joined: 01 Jan 1970
Posts:

PostPosted: Wed Aug 09, 2006 5:37    Post subject: NEW: Firmware Modification Kit (beta testing) Reply with quote
The other thread is a mess, thanks to me releasing things before I should, so I'm starting this one. Currently this kit is in beta testing.

Description

This kit gives the user the ability to make changes to a firmware image without recompiling the firmware sources. It works by extracting the firmware into its component parts, then extracting the squashfs-lzma based file system (other file systems can be supported with tweaks). The user can then make modifications to the extracted file system, then rebuild the firmware image.

Why is this useful?

Some of the many possible uses of this kit:

- add initialization scripts
- change the web UI pages
- install new packages (packages pre-installed are compressed much better)
- remove un-needed packages
- mix-and-match packages from various DD-WRT variants

Documentation/description: http://www.bitsum.com/firmware_mod_kit.htm

Please let me know of any problems. I'll keep updating it as I get the urge, there is a bit more polishing and documentation to write.

Even though I've simplified the process to a single step for extraction and a single step for rebuilding, I don't recommend the totally technically inept use this tool. Use at your own risk.

Special thanks to josh22_22 and others in the original thread for putting up with the early releases and reporting problems.


Last edited by on Wed Aug 09, 2006 14:34; edited 4 times in total
Sponsor




Joined: 01 Jan 1970
Posts:

PostPosted: Wed Aug 09, 2006 13:13    Post subject: New version: v0.151 beta Reply with quote
v0.151 beta released

This version cleans up the script outputs and adds better sanity checking. I've also used this toolkit a few times now to creat custom micro builds, so it's been through more testing.


  • Reworked scripts so they have cleaner, clearer output and use a log file (build.log and extract.log)
  • Added more sanity checks in the scripts
  • Documentation additions
  • Misc. polishing
SBeaver
DD-WRT Novice


Joined: 17 Jun 2006
Posts: 6

PostPosted: Wed Aug 09, 2006 14:12    Post subject: Reply with quote
How much memory do you think this would save? let's say i use dd-wrt standard with QoS, UPNP, telnet and dnsmasq. is there really so much space to save or is this more for v5 routers with smaller memory?




Joined: 01 Jan 1970
Posts:

PostPosted: Wed Aug 09, 2006 14:23    Post subject: Reply with quote
SBeaver wrote:
How much memory do you think this would save? let's say i use dd-wrt standard with QoS, UPNP, telnet and dnsmasq. is there really so much space to save or is this more for v5 routers with smaller memory?


This technique isn't useful for just micro builds. It is helpful for almost anyone not wanting to run a default build of DD-WRT. You can use it to reduce the number of packages, increase the number of packages, or just change things around a bit (scripts, web UI, whatever..). Even for routers where you have extra ROM space already,, by pre-installing things you save ROM space since they are MUCH better compressed than if installed after you flash the firmware.

How much ROM space to save by chopping unused services? Well, if you JUST chopped PPP, PPPOE, and PPTP you'll save around 200KB. Removing specialized stuff like kaid would save a shitload more from the standard build. As far as RAM use, it all depends on what you have actually running more than what you have installed.

Once this kit gets known it will be a big deal to many people. It's so easy to use, works so well, and is so powerful.


Last edited by on Wed Aug 09, 2006 15:03; edited 5 times in total
SBeaver
DD-WRT Novice


Joined: 17 Jun 2006
Posts: 6

PostPosted: Wed Aug 09, 2006 14:48    Post subject: Reply with quote
db90h wrote:
SBeaver wrote:
How much memory do you think this would save? let's say i use dd-wrt standard with QoS, UPNP, telnet and dnsmasq. is there really so much space to save or is this more for v5 routers with smaller memory?


This technique isn't useful for just micro builds. It is helpful for almost anyone not wanting to run a default build of DD-WRT. You can use it to reduce the number of packages, increase the number of packages, or just change things around a bit (scripts, web UI, whatever..). Even for routers where you have extra ROM space already,, by pre-installing things you save ROM space since they are MUCH better compressed than if installed after you flash the firmware.

How much space to save by chopping unused services? Well, if you JUST chopped PPP, PPPOE, and PPTP you'll save around 200KB. As far as RAM use, it all depends on what you have actually running more than what you have installed.

Once this kit gets known it will be a big deal to many people. It's so easy to use, works so well, and is so powerful.


Well that sounds very good, but if it doesnt save ram I wont need it myself
Darkblue
DD-WRT Novice


Joined: 08 Jun 2006
Posts: 19

PostPosted: Wed Aug 09, 2006 16:19    Post subject: Reply with quote
Hi, great work. Do you know were is the WEB GUI images in the descompress image??
I am interesting in create, personalized GUI´s.
Thanks.
.D0T
DD-WRT Novice


Joined: 07 Jun 2006
Posts: 19

PostPosted: Wed Aug 09, 2006 16:52    Post subject: Reply with quote
Sounds fantastic! Exactly what I was waiting for Wink Recompiling isn't that funny if you just want to change a config file (hehe, finally I will have /mmc/ as standard value für ipkg^^)!

I'll try it once I've made sure my wrt will survive the "surgery". It shouldn't hurt the bootloader but you never know Wink Attaching the jtag isn't that hard after all and while doing so I can take care of the serial port(s) I wanted to add too ;)

Thanks for this great tool!
.D0T




Joined: 01 Jan 1970
Posts:

PostPosted: Wed Aug 09, 2006 17:46    Post subject: Reply with quote
Wow, many thanks for this tool. It's much faster than rebuilding the full sources. Now, OpenWRT is obsolet for me.

Again. Many, many thanks db90h!
solarflare
DD-WRT Novice


Joined: 26 Jun 2006
Posts: 14
Location: Helsinki, Finland

PostPosted: Wed Aug 09, 2006 19:35    Post subject: Doesn't compile on Mac OS X without tweaks (uname: Darwin) Reply with quote
The make command halts with these lines:

Code:
g++ -O3 -Wall -c  -I ../../../ ../../../Common/String.cpp
g++ -O3 -Wall -c  -I ../../../ ../../../Common/StringConvert.cpp
../../../Common/MyWindows.h:86: warning: ‘struct IUnknown’ has virtual functions but non-virtual destructor
g++ -O3 -Wall -c  -I ../../../ ../../../Common/StringToInt.cpp
g++ -O3 -Wall -c  -I ../../../ ../../../Common/Vector.cpp
ar r liblzma.a ZLib.o LZMADecoder.o LZMAEncoder.o LZInWindow.o LZOutWindow.o RangeCoderBit.o InBuffer.o OutBuffer.o FileStreams
.o Alloc.o C_FileIO.o CommandLineParser.o CRC.o StreamUtils.o String.o StringConvert.o StringToInt.o Vector.o
ar: creating archive liblzma.a
g++ mksquashfs.o read_fs.o sort.o -L./lzma/C/7zip/Compress/LZMA_Lib -llzma -o mksquashfs-lzma
cc mksquashfs.o read_fs.o sort.o -lz -o mksquashfs
cc -I. -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -O2   -c -o unsquashfs.o unsquashfs.c
make -C ./lzma/C/7zip/Compress/LZMA_Lib
make[1]: Nothing to be done for `all'.
g++ unsquashfs.o -L./lzma/C/7zip/Compress/LZMA_Lib -llzma -o unsquashfs-lzma
cc unsquashfs.o -lz -o unsquashfs
cc -I. -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -O2   -c -o extfirmware.o extfirmware.c
extfirmware.c:17:20: error: endian.h: No such file or directory
extfirmware.c:18:22: error: byteswap.h: No such file or directory
make: *** [extfirmware.o] Error 1

So it's some of the includes that the script doesn't found. However, I can find them in many different placess, such as:

/System/Library/Frameworks/Kernel.framework/Versions/A/Headers/i386/endian.h
/System/Library/Frameworks/Kernel.framework/Versions/A/Headers/machine/endian.h
/System/Library/Frameworks/Kernel.framework/Versions/A/Headers/ppc/endian.h
/System/Library/Frameworks/Kernel.framework/Versions/A/Headers/sys/_endian.h
/usr/include/i386/endian.h
/usr/include/machine/endian.h
/usr/include/ppc/endian.h
/usr/include/sys/_endian.h

The script could examine if the uname returns value Darwin, and act accordingly. That would be great. On the other hand, byteswap.h seems to be missing on my computer. I wonder if that inhibits the use.




Joined: 01 Jan 1970
Posts:

PostPosted: Wed Aug 09, 2006 19:47    Post subject: Re: Doesn't compile on Mac OS X without tweaks (uname: Darwi Reply with quote
solarflare wrote:
The make command halts with these lines:

I don't see such errors. Works perfectly on Ubuntu 6.06 LTS.




Joined: 01 Jan 1970
Posts:

PostPosted: Wed Aug 09, 2006 20:31    Post subject: Re: Doesn't compile on Mac OS X without tweaks (uname: Darwi Reply with quote
solarflare wrote:
The make command halts with these lines:


I've never tried to install under MAC OS X(I almost didn't notice your subject line). I think to missing Linux Header Files. Maybe I can tweak it so that it works ok, but I don't have a MAC OS X system to test on. If you find some packages that allow it to work, please let me know. If you are a coder, you may be able to change extfirmware.c and trx.c to remove any dependencies on the problematic header files (there might not even be any, I need to look.. if there are, should be easy to take care of).

If you can give me a shell on a MAC OS X box, I'm happy to with with it. On MAC OS X, uname==Darwin then ? So this is a reliable way to detect if the platform is MAC OS X?

UPDATE: I've just completed a series of changes intended to increase compatibility with non-linux systems. After testing, I'll release this new build as v0.20.


Last edited by on Wed Aug 09, 2006 22:06; edited 6 times in total




Joined: 01 Jan 1970
Posts:

PostPosted: Wed Aug 09, 2006 20:36    Post subject: Reply with quote
Darkblue wrote:
Hi, great work. Do you know were is the WEB GUI images in the descompress image??
I am interesting in create, personalized GUI´s.
Thanks.


In the working directory they are in rootfs/www ... Some of the web UI is implemented in C directly in the httpd source though.




Joined: 01 Jan 1970
Posts:

PostPosted: Wed Aug 09, 2006 20:40    Post subject: Reply with quote
.D0T wrote:
I'll try it once I've made sure my wrt will survive the "surgery". It shouldn't hurt the bootloader but you never know Wink Attaching the jtag isn't that hard after all and while doing so I can take care of the serial port(s) I wanted to add too Wink


Yes, this is something I need to document. You can definitely screw up the firmware so your unit doesn't boot successfully, but the worst case scenario is usually just that you have to TFTP a new firmware in the boot_wait period. You're right, the CFE should not be affected. You won't regret adding a serial port, I can't live without one now that I've had one.




Joined: 01 Jan 1970
Posts:

PostPosted: Wed Aug 09, 2006 20:44    Post subject: Reply with quote
sagichnicht04 wrote:
Wow, many thanks for this tool. It's much faster than rebuilding the full sources. Now, OpenWRT is obsolet for me.

Again. Many, many thanks db90h!


I'm glad you have found this useful. I've been very excited about this development, and kind of waiting for people to take notice. It will change entire landscape of DD-WRT I think as now ANYBODY can make their own custom firmwares. I played around a bit with mixing n' matching packages/binaries from different DD-WRT versions and it works great.

The next thing I might do is distribute a build environment that allows people to easily port new packages to DD-WRT. It will basically just build DD-WRT ready compatible executables for whatever platform (MIPSL for example). A few changes to the OpenWrt build root and you can use it, including all the packages that have been ported to it. Of course, if you want to do static links you can for sure use any OpenWrt pacakge, and many work already despite the sometimes different in kernel version.
josh22_22
DD-WRT Novice


Joined: 15 Jul 2006
Posts: 24

PostPosted: Wed Aug 09, 2006 21:57    Post subject: Reply with quote
no problem with the beta testing....
AND YES THANK YOU IT WORKED!!! Very Happy Very Happy

But now i have another question.Laughing How can I make the router run CGI and .PL files. The CGI file is 244 Kb. I know the router runs .cgi files becuase ther is a file 192.168.1.1/apply.cgi I tried uploading the change firmware with the the but it won't work. Well the firmaware works but not the file. Do you know how??
Thx
Goto page 1, 2, 3 ... 12, 13, 14  Next Display posts from previous:    Page 1 of 14
Post new topic   Reply to topic    DD-WRT Forum Index -> Broadcom SoC based Hardware All times are GMT

Navigation

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You can attach files in this forum
You can download files in this forum