Compiling DD-WRT

Post new topic   Reply to topic    DD-WRT Forum Forum Index -> Broadcom SoC based Hardware
Goto page Previous  1, 2, 3 ... 12, 13, 14
Author Message
hanskloss
DD-WRT User


Joined: 30 May 2010
Posts: 97

PostPosted: Mon Nov 27, 2017 19:10    Post subject: Reply with quote
Little background first. I found the following binaries/libraries responsible for serving http pages.

/etc/www (size 2.5MB)
/usr/lib/validate.so
/usr/lib/visuals.so
/usr/sbin/httpd

Using tsynik's patches for Firewall.asp and Wireless_Basic.asp and removing optimizefiles from the kromo Makefile I was able to compile more or less working web interface.
Inability to get fully working GUI may have something to do with missing repugly command, referred by the above Makefile. Thanks Broly for this and other tickets
http://svn.dd-wrt.com/ticket/6057

quarkysg wrote:

Firmware GUI issues

The source from the SVN repo apparently could not be used to successfully build a working GUI for the firmware image. The Javascript optimisation part always messes it up, resulting in corrupted Javascripts being generated and built into the firmware. Instead of troubleshooting, I took the shortcut of extracting the Javascript files in the released binary (downloaded from the dd-wrt website) and including them in my build. I replaced the Javascripts I extracted from the working binary located in this location: /src/router/kromo/.



Can you please elaborate little bit more what javascript files you have extracted and what method was used?

_________________
Asus RT-AC66R Target:brcm47xx SubTarget:mips74k Packages:mipsel_74kc
Netgear R7000 Target:bcm53xx Packages:arm_cortex-a9
Sponsor
quarkysg
DD-WRT User


Joined: 03 May 2015
Posts: 227

PostPosted: Mon Nov 27, 2017 22:34    Post subject: Reply with quote
hanskloss wrote:

Can you please elaborate little bit more what javascript files you have extracted and what method was used?


All I did was extract all Javascripts files from working firmware released in the official BrainSlayer build and replaced my build’s Javascript files. IIRC the minification of JS files during the image build process messes it up. So I just extracted the monified JS files from released builds.
hanskloss
DD-WRT User


Joined: 30 May 2010
Posts: 97

PostPosted: Sat Dec 02, 2017 17:02    Post subject: Reply with quote
Here's my experience compiling dd-wrt from sources for the ARM platform. Previous posts were very helpful in setting up the build platform. My post will focus more on steps and errors I have encountered rather than providing "do-it-all" scripts.
In the example below I have compiled a mini image for R7000 SVN revision: 33941 Full image is not recommended at first as too many changes are required, to make it work.
After taking the notes, I have re-installed the OS and pulled fresh dd-wrt sources to make sure these steps are reproducible.


Serial recovery kit is required before attempting to load the final image!!!

DD-WRT source directory structure:

    ~/DD-WRT/tools It contains binaries needed to optimize and compile WebGUI
    ~/DD-WRT/src/router Our working directory. Here we will execute the make command. Main northstar .config and Makefile.northstar are located. Also it contains all sources for packages and libraries.
    ~/DD-WRT/src/router/emf_bin/arm prioprietary emf and igs binaries
    ~/DD-WRT/src/router/rules set of make rules for each package, used during the configure and install step.
    ~/DD-WRT/src/router/arm-uclibc Final image will be located here, also INSTALLDIR and TARGETDIR which be compressed into squash filesystem.

    ~/DD-WRT/src/linux/universal/linux-4.4 Kernel northstar SMP .config will be copied here and sources for all kernel modules.
    ~/DD-WRT/src/linux/universal/linux-4.4/brcm/arm Prioprietary kernel objects needed for compiling wl.ko module.
    ~/DD-WRT/src/linux/universal/linux-4.4/arch/arm/boot Location of zImage (compressed Linux image)




PREPARATION


What toolchain?
I downloaded BS and Kong's firmwares with the same kernel and grep'ed files to see what they used.
Code:
strings /lib/libstdc++.so.6 | grep toolchain

BS result: toolchain-arm_cortex-a9_gcc-6.3.0_musl-1.1.16_eabi
Kong result: toolchain-arm_cortex-a9_gcc-6.3.0_musl_eabi

toolchain-arm_cortex-a9_gcc-6.3.0_musl-1.1.16_eabi is available in the dd-wrt.com downloads section
http://www.dd-wrt.com/site/support/other-downloads


Linux distribution used:
Debian Jessie amd64 8.9.0 DVD-1
Additional developer packages are needed to compile the mini version:
Code:
apt-get install subversion  build-essential ccache autoconf libtool cmake pkg-config bison flex gengetopt squashfs-tools lib32stdc++6


Pull the sources:
Code:
svn checkout svn://svn.dd-wrt.com/DD-WRT -r33941


Export toolchain's bin into PATH

Code:
GCCARM=/home/user/toolchain-arm_cortex-a9_gcc-6.3.0_musl-1.1.16_eabi/bin
export PATH=$GCCARM:$PATH


During compilation, some packages complain about missing revision.h file. Create the files as below.

Code:
cd ~/DD-WRT/src/router/libutils
echo -n '#define SVN_REVISION "' > revision.h
svnversion -n . >> revision.h
echo '"' >> revision.h

cd ~/DD-WRT/src/router/httpd/visuals
echo -n '#define SVN_REVISION "' > revision.h
svnversion -n . >> revision.h
echo '"' >> revision.h

cd ~/DD-WRT/src/router/httpd
echo -n '#define SVN_REVISION "' > revision.h
svnversion -n . >> revision.h
echo '"' >> revision.h

cd ~/DD-WRT/src/router/shared
echo -n '#define SVN_REVISION "' > revision.h
svnversion -n . >> revision.h
echo '"' >> revision.h

cd ~/DD-WRT/src/router/services
echo -n '#define SVN_REVISION "' > revision.h
svnversion -n . >> revision.h
echo '"' >> revision.h

cd ~/DD-WRT/src/router/rc
echo -n '#define SVN_REVISION "' > revision.h
svnversion -n . >> revision.h
echo '"' >> revision.h


Copy the mini config
Code:
cd ~/DD-WRT/src/router
cp configs/northstar/.config_northstar_mini .config


Comment out CONFIG_SUPERCHANNEL SPUTNIK_APD CONFIG_HOSTAPD2 in the .config file as they are not used for Broadcom.

Cosmetic change. Comment out rules for non existant packages in the file ~/DD-WRT/src/router/rules/all.mk

Code:
#rules/all.mk:131: mmc-ixp4xx.mk
#rules/all.mk:214: strace.mk
#rules/all.mk:226: oled.mk
#rules/all.mk:237: ath9k-rules/ath9k.mk
#rules/all.mk:241: atm.mk


Comment out Ralink drivers not used by our router, lines 287-296 in the Kconfig file
Code:
cd ~/DD-WRT/src/linux/universal/linux-4.4/drivers/net/wireless


#if RALINK_DEVICE
#source "drivers/net/wireless/rt3352/rt2860v2_ap/Kconfig"
#source "drivers/net/wireless/rt3352/rt2860v2_sta/Kconfig"
#endif
#if SOC_MT7620_OPENWRT
#source "drivers/net/wireless/rt7620/rt2860v2_ap/Kconfig"
#source "drivers/net/wireless/rt7620/rt2860v2_sta/Kconfig"
#source "drivers/net/wireless/rt5592/Kconfig"
#source "drivers/net/wireless/rt7612/rlt_wifi/Kconfig"
#endif


Proprietary Broadcom wireless modules
Make following corrections because of non existent source file to compile wl.ko module

cd ~/DD-WRT/src/linux/universal/linux-4.4/brcm/arm/wl/config
Comment out in the file wl.mk line 448 WLFILES_SRC_HI += src/wl/clm/src/wlc_clm_data.c

cd ~/DD-WRT/src/router
Comment out lines 181 182 inside Makefile.northstar where it copies wlc_clm_data.c

cd ~/DD-WRT/src/linux/universal/linux-4.4/drivers/net/wl/
Append following to the line 95 $(SRCBASE_OFFSET)/wl/clm/src/wlc_clm_data.o in the file wl_generic.mk


Edit emf.mk Warning! make files are space/TAB sensitive. Need TAB before each install -D command.
cd ~/DD-WRT/src/router/rules

Code:
emf:
# $(MAKE) -C emf/emfconf CROSS=$(CROSS_COMPILE)
# $(MAKE) -C emf/igsconf CROSS=$(CROSS_COMPILE)
# $(MAKE) -C emf/emf
# $(MAKE) -C emf/igs

emf-clean:
# $(MAKE) -C emf/emfconf clean
# $(MAKE) -C emf/igsconf clean
# $(MAKE) -C emf/emf clean
# $(MAKE) -C emf/igs clean

emf-install:
# $(MAKE) -C emf/igsconf CROSS=$(CROSS_COMPILE) INSTALLDIR=$(INSTALLDIR)/emf install
# $(MAKE) -C emf/emfconf CROSS=$(CROSS_COMPILE) INSTALLDIR=$(INSTALLDIR)/emf install
# $(MAKE) -C emf/igs CROSS=$(CROSS_COMPILE) INSTALL_MOD_PATH=$(INSTALLDIR)/emf install
# $(MAKE) -C emf/emf CROSS=$(CROSS_COMPILE) INSTALL_MOD_PATH=$(INSTALLDIR)/emf install
   install -D emf_bin/$(ARCH)/emf.ko $(INSTALLDIR)/emf/lib/modules/$(KERNELRELEASE)/emf.ko
   install -D emf_bin/$(ARCH)/igs.ko $(INSTALLDIR)/emf/lib/modules/$(KERNELRELEASE)/igs.ko
   install -D emf_bin/$(ARCH)/emf $(INSTALLDIR)/emf/usr/sbin/emf
   install -D emf_bin/$(ARCH)/igs $(INSTALLDIR)/emf/usr/sbin/igs


I needed to use serial recovery in my builds because included wireless modules would cause my router to "kernel panic" during the boot.
Replace emf.ko and igs.ko in the directory ~/DD-WRT/src/router/emf_bin/arm with modules which come from BS or Kong's firmware.

Good modules have the following sizes:
21864 emf.ko
16900 igs.ko

WebGUI
Remove optimizefiles from kromo/dd-wrt/Makefile line 1139

Final web interface will not be fully functional. Some solutions exists: using tsynik patches or extracting javascripts from the www binary. All asp and javascripts page sources are compiled into single www binary
~/DD-WRT/tools/filelist contains the exact listing of all files(web pages and images)




CONFIGURE

These packages need to be compiled first in this order. Kernel step will generate kernel.release file which is later needed for compilation of other packages.

Code:
make -f Makefile.northstar jansson-configure
make -f Makefile.northstar jansson
make -f Makefile.northstar nvram
make -f Makefile.northstar utils
make -f Makefile.northstar install_headers 
make -f Makefile.northstar iptables
make -f Makefile.northstar libnet
make -f Makefile.northstar kernel


Fix the typo in the libubox.mk line 7. Include dir should be json-c not json

cd ~/DD-WRT/src/router/rules
UBOX_EXTRA_CFLAGS=-I$(TOP)/_staging/usr/include -I$(TOP)/_staging/usr/include/json-c $(COPTS) $(MIPS16_OPT)

In the previous revisions compiliation of the libubox package frequently failed. Run these steps too ensure it compiles fine.

Code:
make -f Makefile.northstar json-c-configure
make -f Makefile.northstar json-c
make -f Makefile.northstar libubox-configure
make -f Makefile.northstar libubox


Final configure and all
Code:
make -f Makefile.northstar configure
make -f Makefile.northstar clean all




INSTALL


Compile trx in opt/tools and copy to src/router/tools
These files are for Netgear and D-link. Asus and TP-Link need some fixes so I skipped those.

Code:
cd ~/DD-WRT/opt/tools/
gcc -o trx trx.c
gcc -o trx_n trx_n.c
cp trx ../../src/router/tools
cp trx_n ../../src/router/tools


Create missing link to the opt folder
Code:
cd ~/DD-WRT/src/router
ln -s ../../opt opt


Original mksquashfs4 gives me illegal instruction error. I used the debian version which works fine.
Replace the line 575 in the Makefile.northstar
Code:
/usr/bin/mksquashfs $(ARCH)-uclibc/target $(ARCH)-uclibc/root.fs -comp xz -nopad  -root-owned -noappend -Xbcj arm


Since trx tools were not compiled for TP-Link and Asus comment out the following lines in the Makefile.northstar
583-588 for TP-Link
655-664 for Asus
668-669 for Buffalo

Final command
make -f Makefile.northstar install

Look for the firmware inside this folder:

~/DD-WRT/src/router/arm-uclibc



Some tips


R7000 Recovery:
During the boot ctrl+ C
At the CFE>tftpd
tftp -m binary 192.168.1.1 -c put K3_R7000.chk


Example of selectively restoring source folders:

svn checkout svn://svn.dd-wrt.com/DD-WRT/src/router/httpd -r 33941


Example of compiling single kernel module

GCCARM=/home/user/toolchain-arm_cortex-a9_gcc-6.3.0_musl-1.1.16_eabi/bin
export PATH=$GCCARM:$PATH
cd ~/DD-WRT/src/linux/universal/linux-4.4
make ARCH=arm modules SUBDIRS=drivers/net/switch

Displaying variables or paths during compilation:

Insert into Makefile.northstar following line as needed.
$(info ******************)
$(info My ARCH is $(ARCH))
$(info ******************)



33491.png
 Description:
 Filesize:  39.11 KB
 Viewed:  1168 Time(s)

33491.png



_________________
Asus RT-AC66R Target:brcm47xx SubTarget:mips74k Packages:mipsel_74kc
Netgear R7000 Target:bcm53xx Packages:arm_cortex-a9
0ddentity
DD-WRT Novice


Joined: 09 Feb 2014
Posts: 5

PostPosted: Thu Dec 28, 2017 5:14    Post subject: Reply with quote
Hi, taking my first tentative steps to building here..

First question seems to be what toolchain to use. I'm running on an linksys E4200, which is a 'mipsel'. I'm currently running beta r34080 successfully
(my goal whether I reach it or not is to add in ext3 support) and from your last post I grepped toolchain in the stdlib on the running router which gave:

'toolchain-mipsel_74kc_gcc-6.3.0_musl-1.1.16'

This is not in the .xz download, and I can't find it at the 'other downloads' location either.

Does this exist under some other name perhaps?

Also if there's any other tips on 'for that class of box, use these cfg files' or anything at all.

Cheers & Happy Christmas/New Year!
hanskloss
DD-WRT User


Joined: 30 May 2010
Posts: 97

PostPosted: Fri Dec 29, 2017 21:34    Post subject: Reply with quote
0ddentity wrote:
Hi, taking my first tentative steps to building here..

First question seems to be what toolchain to use. I'm running on an linksys E4200, which is a 'mipsel'. I'm currently running beta r34080 successfully
(my goal whether I reach it or not is to add in ext3 support) and from your last post I grepped toolchain in the stdlib on the running router which gave:

'toolchain-mipsel_74kc_gcc-6.3.0_musl-1.1.16'

This is not in the .xz download, and I can't find it at the 'other downloads' location either.

Does this exist under some other name perhaps?

Also if there's any other tips on 'for that class of box, use these cfg files' or anything at all.

Cheers & Happy Christmas/New Year!


Brainslayer updates toolchains couple times a year. If the latest one is not available try the previous one.
Grep on the E4200 firmware from December 2016 gives

toolchain-mipsel_74kc+dsp2_gcc-6.1.0_musl-1.1.15

_________________
Asus RT-AC66R Target:brcm47xx SubTarget:mips74k Packages:mipsel_74kc
Netgear R7000 Target:bcm53xx Packages:arm_cortex-a9
broly
DD-WRT Novice


Joined: 03 Apr 2010
Posts: 27
Location: edmonton

PostPosted: Fri Feb 02, 2018 2:31    Post subject: Reply with quote
hanskloss wrote:
0ddentity wrote:
Hi, taking my first tentative steps to building here..

First question seems to be what toolchain to use. I'm running on an linksys E4200, which is a 'mipsel'. I'm currently running beta r34080 successfully
(my goal whether I reach it or not is to add in ext3 support) and from your last post I grepped toolchain in the stdlib on the running router which gave:

'toolchain-mipsel_74kc_gcc-6.3.0_musl-1.1.16'

This is not in the .xz download, and I can't find it at the 'other downloads' location either.

Does this exist under some other name perhaps?

Also if there's any other tips on 'for that class of box, use these cfg files' or anything at all.

Cheers & Happy Christmas/New Year!


Brainslayer updates toolchains couple times a year. If the latest one is not available try the previous one.
Grep on the E4200 firmware from December 2016 gives

toolchain-mipsel_74kc+dsp2_gcc-6.1.0_musl-1.1.15


wish i would have found this thread sooner.

unfortunately for me, in my case, tsynik's changes produce a GUI that is no different visually from what was there before.

the key issue is the missing "common.js" in the main root. i just noticed this today after checking write4/etc.

i get a GUI similar to the one shared earlier in this thread, with the missing text.

uglifyJS is not the solution. the solution is to figure out why common.js is missing in the tree if you inspect the (broken) webgui.

this is why the webgui is ugly, because the functions in common.js are responsible for finishing the job. kong's build has this file outside of the js directory (i wish i had it, forget whether it is outside or inside any directory!), mine does not (and i suspect anyone building from SVN does not).

i commend you for getting a working build hans, and i am envious you have something that is currently more appealing visually than my build Wink however, the best solution is probably finding out why common.js isn't being picked up.

i suspect this (small, yet visually criticial) problem holds the key to all of our qualms.

tsynik should also be commended for posting his solutions, but i'm quite certain BS made sure that anyone working on northstar should be able to produce a working webgui (and that does seem to be the case for the most part).

for everyone else, however, that is not the case. i hope someone reading this can point us in the right direction. it's the last step for me, and arguably the most important since i'd like to try flashing brainslayer's version from mine to ensure anyone who doesn't like it, isn't stuck with my build.

_________________
"But of this I can assure you that there is not a movement of any body of Men however small whether on Horse-back or on foot, nor an operation or March of any description nor any Service in the field that is not formed upon some mathematical principle, and in the performance of which the knowledge and practical application of the mathematicks will be found not only useful but necessary. The application of the Mathematicks to Gunnery, Fortification, Tactics, the survey and knowledge of formal Castrenantion etc. cannot be acquired without study"
broly
DD-WRT Novice


Joined: 03 Apr 2010
Posts: 27
Location: edmonton

PostPosted: Fri Feb 02, 2018 2:53    Post subject: Reply with quote
one more thing, which i feel is worthy of a separate post:

please don't lie about what worked. for example, stating that using the etc/www unpacked from a fw obtanied on the dd-wrt site, is a blatant lie (save one case, where you somehow have access to brainslayer/kong's configuration files for each router [i doubt that you would]).

why is this a lie? how do i know this is a lie? because the www file bakes your configuration into the file. that is, it's a different size if you have tor compared to if you do not.

simply plugging in another httpd file from a separate build, even it is for the same platform, will not work (unless you were extremely lucky and configured your build identical to BS/KONG for the respective router [a rare, but possible scenario]) because of what i have said.

this isn't a pissing contest. this is dd-wrt. understand the fundamentals of what we're doing instead of saying "got it!" in order to reduce your perceived humiliation.

the webgui's appeal is supposed to reward your dedication to the fundamentals since you'd know the ins and outs of how it it is made (it is quite the piece of software. i must say. even in its current state, it is impressive).

just sayin'

_________________
"But of this I can assure you that there is not a movement of any body of Men however small whether on Horse-back or on foot, nor an operation or March of any description nor any Service in the field that is not formed upon some mathematical principle, and in the performance of which the knowledge and practical application of the mathematicks will be found not only useful but necessary. The application of the Mathematicks to Gunnery, Fortification, Tactics, the survey and knowledge of formal Castrenantion etc. cannot be acquired without study"
Goto page Previous  1, 2, 3 ... 12, 13, 14 Display posts from previous:    Page 14 of 14
Post new topic   Reply to topic    DD-WRT Forum Forum Index -> Broadcom SoC based Hardware All times are GMT

Navigation

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You can attach files in this forum
You can download files in this forum