Embedded Linux on Xilinx ML405 card: a tutorial

Text and pictures © 2008-2024 Guillaume Dargaud
Last updated on 2023/10/26

"The only thing more dangerous than a hardware guy with a code patch is a programmer with a soldering iron."

Getting Linux on a Xilinx ML405 card

• First of all, here is the reference page for the Xilinx ML405
• My first steps were based on an older Xilinx card tutorial I used as a work base.
• Big thanks go to Stéphane Mancini and Robin Roland for showing me the ropes
• Thanks as well to the members of the uClibc, buildroot, busybox and linuxppc mailing lists.

Installation of the Xilinx software

• install ISE and EDK ( ./setup ) to /softs/xilinx/ise/ and /softs/xilinx/edk/. No need to be root except to create and chown /softs and to install the cable drivers.
• make sure ~/.qt/* belong to the user
• add the following lines to your ~/.bashrc:

  source /softs/xilinx/edk/settings.sh
  source /softs/xilinx/ise/settings.sh
  

• get the latest service packs for ISE, EDK and IPs
• in /etc/group add your username to the uucp line so you can mount the CF cards without having to sudo. Examples of fstab entries:

  /dev/sdc1               /media/ML40X            vfat    pamconsole,noatime,sync,fscontext=system_u:object_r:removable_t,exec,noauto,managed 0 0
  /dev/sdc2               /media/usbdisk          ext2    pamconsole,noatime,sync,exec,noauto,user,managed 0 0
  

Creation of a project with xps

• Here I will place the xps project, the buildroot project and the kernel source in 3 directories under ~/ML405/ , only buildroot cannot be easily moved to a new position (it embeds full paths into some of its executable).
• Base system builder wizard, save to ~/ML405/xps
• New project, new design, Xilinx ML405 board, no stepping.
• PowerPC processor.
• Reference frequency 100Hz, Processor frequency 300Hz, Bus frequency 100Hz. No debug. Use the cache.
• With interrupts, configure the following (most are optional):
  • RSA232 UARTLITE or UART16550 (must match CONFIG_SERIAL_UARTLITE or CONFIG_SERIAL_8250 in the kernel .config),
  • LEDs and Push buttons GPIO (kernel CONFIG_XILINX_GPIO)
  • DDR SDRAM.
  • IIC Eprom (kernel CONFIG_XILINX_IIC)
  • SysACE Compact Flash (kernel CONFIG_XILINX_SYSACE).
  • No MGT wrapper.
  • Ethernet MAC: Trimode_Mac_GMII with DMA and interrupt (kernel CONFIG_NETDEV_1000, CONFIG_XILINX_LLTEMAC and CONFIG_XILINX_LLTEMAC_MARVELL_88E1111_GMII). In XPS, make sure the IP is configured as such: V4 hard, Include IO and BUFG, Physical interface type GMII, raio 1, Phy address 1, Fifos 4096, checksum offloads NO, Temac1 NO. Note 1: Some people have had problems with version 1.00b of the driver, it can be reverted in the system.mhs file. Note 2: the EmacLite driver fails above a certain size of packets.
  • No SRAM nor FLASH: attempts to use an SRAM will fail completely on boot.
  • XPS BRAM 64Kb. Cache set on everything.
  • Important notes: the boot program itself must be on the BRAM, it will fail on the DRAM. Programs can be anywhere (if they fit).
  • Choose the sample applications: put the memory test program in BRAM and the other in BRAM.
• [Hardware][Generate Netlist]. Go have a coffee.
• [Hardware][Generate Bitstream]. Go have another coffee.
• [Software][Build all user applications] to produce the executable elf files.
• Now [Device configuration][Update bitstream] 'should' merge the elf files and the bitstream into a download.bit file, but it doesn't seem to work
• The script ml40x_bit2ace can take an already bit/elf merged file and convert it into an ACE file, but since the previous step failed, another way to do this must be used.
• Impact can merge bit and elf file(s) but it also need additional bmm information and there are 3 such files. And impact crashes when it does it if the hardware definition is not perfect.
• The best way to do this is with genace . Here's the command line:

  xmd -tcl genace.tcl -jprog -board ml405 -target ppc_hw -hw implementation/system.bit -elf TestApp_Memory/TestApp_Memory.elf -ace MemTest.ace 
  

• You can then put it in the FAT partition of the memory card:

  mount /media/ML40X/
  rm /media/ML40X/*
  cp MemTest.ace /media/ML40X/
  umount /media/ML40X/
  

Standalone boot and execution

• Connect the card with a serial null-modem cable
• The first time, launch sudo minicom -s and configure stty0 (/dev/ttyS0) in 8N1 mode (8 bits, no parity 1 stop bit) and local echo, save the config. Afterwards just run minicom as a normal user and set the line wrap mode (Ctrl-A Z W).
• If the boot works properly the LEDs IN, DONE, STAT and PWR should all lit up green and you should receive some messages on the serial port. Try with the original CF card given by Xilinx it this step fails (follow the indications here to recreate a card). Warning, the card must be at most 512Mb.

Linux on the PowerPC

• [Software][Software platform settings]: choose one of the Linux configs (here the 2.6 version) and then set the [OS and libraries] options:
• Set MEM_SIZE to the size of the DDR: here 0x8000000 for 128Mb
• Set PLB_CLOCK_FREQ_HZ to the bus speed: 100000000
• Do not set a TARGET_DIR for the BSP, it will default to ppc405_0/libsrc/linux_2_6_v1_00_b/linux
• Set connected_periphs to the devices you want Linux to know about: RS232, ethernet mac, SysAce, interrupt controller... Basically you just need to click on the option and do OK (the interface sucks).
• Other options such as Root fs type don't matter, we'll set them ourselves in the kernel default command line.
• Set the kernel CONFIG_BLK_DEV_RAM_SIZE to be the same as the ram disk size (8192 or 16384).
• [Software][Generate libraries and BSPs]. You should now have some kernel patch source in ppc405_0/libsrc/linux_2_6_v1_00_b/linux , only one of which concerns us.
• Now let's configure and compile BuildRoot and then the kernel.

Buildroot: toolchain, uclibc and busybox

• Use a recent host system with a recent gcc and toolchain (this caused me lots of grief at first).
• Get the Buildroot source with

  cd ~/ML405
  svn co svn://uclibc.org/trunk/buildroot
  cd buildroot
  

• And then update it periodically as well with svn up from that directory.
• make menuconfig change some of the options: powerpc, 405, etc... Exclude the kernel from the build. Make sure you have the correct compiler version in [Toolchain][GCC compiler version] from your gcc -v . Use soft float by default. Remove Wchar, IPv6, large file support, etc. Create an ext2 filesystem (which will be saved in binaries/genepy/rootfs.powerpc.ext2.gz) with optional ramdisk, and a tar file.
• Optional make source to download all the sources in advance (possibly fix some package version numbers if not found)
• make when the make stops to ask you extra options, interrupt it.
• make saveconfig
• Customize uClibc: make uclibc-menuconfig where you enable the MMU, disable the FPU, disable large file support, enable RPC, disable WCHAR and locale.
• Customize busybox: make busybox-menuconfig where you can remove some of the executables that require floats: awk, dc, etc...
• make saveconfig; make
• If you changed a small configuration or source file somewhere and buildroot won't rebuild from it, locate the corresponding .configured file and 'touch' it.
• Keep a copy of all your config files: cd ..; tar cvf configs.tar $(find -name .config) buildroot/local which you can later reinstate with tar xvf configs.tar . Also the skeleton dir if you make modifs in it and the device_table.txt (see below):

  tar zcvf configs.buildroot.tgz $(fn .config) buildroot/local buildroot/toolchain/uClibc/uClibc-0.9.29.config buildroot/package/busybox/busybox-1.6.0.config buildroot/target/generic/device_table.txt buildroot/target/generic/target_skeleton $(find buildroot/target/generic/target_skeleton -type f | grep -v .svn)
  

• Note that the cleaning process of buildroot is buggy. It is often better to restart from a clean svn tree than to do make clean or make distclean . Also if you change busybox options, it won't remake it unless you do make busybox-update .
• Also of note, when you have strange compile errors after changing options, rm -rf toolchain_build_powerpc will rebuild most packages cleanly.
• Leave buildroot's directory in place, since some paths are hardcoded in the executables that buildroot created. Simply changing them in the configs, cleaning and recompiling won't work.

Xilinx kernel compilation

• Install git, one way or another (try yum install git ). I had to:

  wget http://kernel.org/pub/software/scm/git/RPMS/i386/perl-Git-1.5.4.4-1.i386.rpm http://kernel.org/pub/software/scm/git/RPMS/i386/git-1.5.4.4-1.i386.rpm
  sudo yum install perl-Git-1.5.4.4-1.i386.rpm git-1.5.4.4-1.i386.rpm perl-Error-0.15-2.el5.rf.noarch.rpm
  

• Download the Xilinx patched kernel (what I use) or the SecretLab tree:

  cd ~/kernels; 
  git clone git://git.xilinx.com/linux-2.6-xlnx.git
  git clone git://git.secretlab.ca/git/linux-2.6-virtex.git
  git pull	# Use this later from the newly created dirs to keep them up to date
  

• Make a copy and add the xparameters file from the BSP generation to it:

  cp -R ~/kernels/linux-2.6-xlnx ~/ML405/
  cd ~/ML405
  cp -iv xps/ppc405_0/libsrc/linux_2_6_v1_00_b/linux/arch/ppc/platforms/4xx/xparameters/xparameters_ml40x.h linux-2.6.24-xlnx/arch/ppc/platforms/4xx/xparameters/xparameters_ml405.h
  

• Set the location of the cross compiler: export PATH=~/ML405/buildroot/build_powerpc/staging_dir/bin:$PATH
• You can use ml405_defconfig as a .config base.
• make ARCH=ppc menuconfig and set options to match the BSP, in particular [Processor][IBM 4xx][Machine type] set to ML405 so that it matches your xparameters_ml405.h file.
• Disable the module support while setting the options but make sure to re-enable it before quitting, or verify with grep "=m" $(find -name .config) .
• Link the ramdisk (only if you want to boot with the ramdisk):

  ln -s ../../../../../buildroot/binaries/genepy/rootfs.powerpc.ext2.gz linux-2.6.24-xlnx/arch/ppc/boot/images/ramdisk.image.gz
  ln -s ../../../../../buildroot/binaries/genepy/rootfs.powerpc.ext2 linux-2.6.24-xlnx/arch/ppc/boot/images/ramdisk.image
  

• Compile the kernel with its ramdisk: make ARCH=ppc CROSS_COMPILE=powerpc-linux-uclibc- zImage.initrd
• or compile the kernel without ramdisk: make ARCH=ppc CROSS_COMPILE=powerpc-linux-uclibc-

Integration

• Once you have a kernel, generate the ACE file from your xps project directory for the ramdisk version with

  xmd -tcl genace.tcl -jprog -board ml405 -target ppc_hw -hw implementation/system.bit -elf ../linux-2.6.24-xlnx/arch/ppc/boot/images/zImage.initrd.elf -ace Initrd26.ace 
  

• or this for the partition boot version:

  xmd -tcl genace.tcl -jprog -board ml405 -target ppc_hw -hw implementation/system.bit -elf ../linux-2.6.24-xlnx/arch/ppc/boot/images/zImage.elf -ace Kernel26.ace
  

• Copy it to the card:

  mount /media/ML40X/
  rm /media/ML40X/*.ace
  cp Initrd26.ace /media/ML40X/
  umount /media/ML40X/ 
  

• Boot it. You should at least see the kernel parameters and the start of the decompression, even if there's nothing else configured.
• If /dev is empty, setup the devices in buildroot/target/generic/device_table.txt , see below. You can also copy the /dev tree from any linux.
• If not using the ramdisk, copy the ext2 partition to the already partitioned flash card and adjust its size to fit the partition size (make sure the ext2 file is not loop-mounted while you do that):

  sudo dd if=binaries/genepy/rootfs.powerpc.ext2 of=/dev/sdc2
  sudo /sbin/e2fsck -f /dev/sdc2 
  sudo /sbin/resize2fs /dev/sdc2
  

Boot process and target system configuration

• Parameters to pass to the kernel upon boot (can be manual or set in the .config as CONFIG_CMDLINE):
  • console=ttyUL0,9600 if using UartLight
  • root=/dev/ram to boot from the ramdisk
  • root=/dev/xsa2 to boot without ramdisk from the 2nd partition (and not /dev/xsysace/disc0/part2, /dev/xsysace2 or /dev/xsda2 as seen in some other documentation.
  • rw read/write access to the root
  • ip=on to get eth0 address from DHCP

Some other considerations and customization of the root:

• Edit the inittab file to add ttyUL0::respawn:/sbin/getty -L ttyUL0 9600 vt100 and remove the other ttys to configure the login
• Authorize root on tty: echo ttyUL0 >>target/generic/target_skeleton/etc/securetty (or better, disable securetty from busybox config)
• There is no root password by default, but you can add one doing passwd after booting and copying the 2nd column of /etc/shadow into buildroot/target/generic/target_skeleton/etc/shadow . You can't directly copy a shadow file from just any Linux because recent versions use md5pass or others to encrypt it while busybox still use older crypt.
• Put the following in buildroot/target/generic/device_table.txt :

  #<name>		<type>	<mode>	<uid>	<gid>	<major>	<minor>	<start>	<inc>	<count>
  
  /dev		d	755	0	0	-	-	-	-	-
  /dev/pts	d	755	0	0	-	-	-	-	-
  /dev/shm	d	755	0	0	-	-	-	-	-
  /tmp		d	1777	0	0	-	-	-	-	-
  /etc		d	755	0	0	-	-	-	-	-
  /home/default	d	2755	1000	1000	-	-	-	-	-
  #<name>					<type>	<mode>	<uid>	<gid>	<major>	<minor>	<start>	<inc>	<count>
  
  /bin/busybox				f	4755	0	0	-	-	-	-	-
  /etc/shadow				f	600	0	0	-	-	-	-	-
  /etc/passwd				f	644	0	0	-	-	-	-	-
  /etc/network/if-up.d			d	755	0	0	-	-	-	-	-
  /etc/network/if-pre-up.d		d	755	0	0	-	-	-	-	-
  /etc/network/if-down.d			d	755	0	0	-	-	-	-	-
  /etc/network/if-post-down.d		d	755	0	0	-	-	-	-	-
  /usr/share/udhcpc/default.script	f	755	0	0	-	-	-	-	-
  # uncomment this to allow starting x as non-root
  #/usr/X11R6/bin/Xfbdev		f	4755	0	0	-	-	-	-	-
  # Normal system devices
  /dev/mem	c	640	0	0	1	1	0	0	-
  /dev/kmem	c	640	0	0	1	2	0	0	-
  /dev/null	c	666	0	0	1	3	0	0	-
  /dev/zero	c	666	0	0	1	5	0	0	-
  /dev/random	c	666	0	0	1	8	0	0	-
  /dev/urandom	c	666	0	0	1	9	0	0	-
  # /dev/log	s	666	0	0	-	-	-	-	-
  /dev/ram	b	640	0	0	1	1	0	0	-
  /dev/ram	b	640	0	0	1	0	0	1	4
  /dev/loop	b	640	0	0	7	0	0	1	2
  /dev/rtc	c	640	0	0	10	135	-	-	-
  /dev/console	c	666	0	0	5	1	-	-	-
  /dev/tty	c	666	0	0	5	0	-	-	-
  /dev/tty	c	666	0	0	4	0	0	1	8
  
  /dev/nfs	b	777	0	0	0	255	0	0	-
  /dev/port	c	640	0	15	1	4	-	-	-
  /dev/full	c	666	0	7	-	-	-
  /dev/fd		b	640	0	0	2	0	0	1	2
  
  # UartLite
  /dev/ttyUL 	c 	666 	0 	0 	204  	187 	0 	1 	1
  # Some of those are necessary for dropbear to work (pty by default)
  /dev/ttyp	c	666	0	0	3	0	0	1	10
  /dev/ptyp	c	666	0	0	2	0	0	1	10
  /dev/ptmx	c	666	0	0	5	2	-	-	-
  /dev/ttyP	c	666	0	0	57	0	0	1	4
  #/dev/ttyS	c	666	0	0	4	64	0	1	4
  
  #/dev/fb	c	640	0	5	29	0	0	32	4
  #/dev/ttySA	c	666	0	0	204	5	0	1	3
  #/dev/psaux	c	666	0	0	10	1	0	0	-
  #/dev/ppp	c	666	0	0	108	0	-	-	-
  
  # Input stuff
  #/dev/input		d	755	0	0	-	-	-	-	-
  #/dev/input/mice		c	640	0	0	13	63	0	0	-
  #/dev/input/mouse	c	660	0	0	13	32	0	1	4
  #/dev/input/event	c	660	0	0	13	64	0	1	4
  #/dev/input/js		c	660	0	0	13	0	0	1	4
  
  # MTD stuff
  #/dev/mtd	c	640	0	0	90	0	0	2	4
  #/dev/mtdblock	b	640	0	0	31	0	0	1	4
  
  #Tun/tap driver
  /dev/net	d	755	0	0	-	-	-	-	-
  /dev/net/tun	c	660	0	0	10	200	-	-	-
  
  # User-mode Linux stuff
  /dev/ubda	b	640	0	0	98	0	0	0	-
  /dev/ubda	b	640	0	0	98	1	1	1	15
  
  #<name>		<type>	<mode>	<uid>	<gid>	<major>	<minor>	<start>	<inc>	<count>
  
  /dev/xsysacea 	b 	640 	0 	0	125 	0 	0 	0 	-
  /dev/xsysacea1 	b 	640 	0 	0 	125 	1	0 	0 	-
  /dev/xsysacea2 	b 	640 	0 	0 	125 	2	0 	0 	-
  #/dev/xsa 	b 	640 	0 	0	125 	0 	0 	0 	-
  #/dev/xsa1 	b 	640 	0 	0 	125 	1	0 	0 	-
  #/dev/xsa2 	b 	640 	0 	0 	125 	2	0 	0 	-
  
  

• When changing the skeleton source root tree in buildroot/target/generic/target_skeleton , doing make is not enough to propagate the changes to project_build_powerpc/genepy/root/ so you may want to manually rm-rf project_build_powerpc/genepy/root; cp -Ruv target/generic/target_skeleton/* project_build_powerpc/genepy/root/ before doing make again, or if you do changes in both directories, run the safer rsync -Cavun target/generic/target_skeleton/ project_build_powerpc/genepy/root/
• Also in the skeleton, do: mkdir target/generic/target_skeleton/root/sys target/generic/target_skeleton/root/proc if you have your kernel with CONFIG_SYSFS=y and/or CONFIG_PROC_FS=y.
• If you want to increase the speed of the serial line twelvefold to 115200 bauds, you need to change all 4 of the following parameters:
  • The speed parameter in the UARTLITE definition of the project wizard. In xps, right-click on RS232_Uart, [Configure IP][UART line baud rate];
  • The default CONFIG_CMDLINE in the kernel .config;
  • The initab ttyUL0 line;
  • The minicom configuration on the host.
• A successful boot off the card:

  loaded at:     00400000 004F559C
  board data at: 004F3520 004F359C
  relocated to:  0040505C 004050D8
  zimage at:     00405E48 004F211C
  avail ram:     004F6000 08000000
  
  Linux/PPC load: console=ttyUL0,115200 rw root=/dev/xsa2 ip=on
  Uncompressing Linux...done.
  Now booting the kernel
  [    0.000000] Linux version 2.6.24-rc8-xlnx (guinevere@lpsc6185x.in2p3.fr) (gcc version 4.1.2)
  #14 Wed Apr 23 16:57:11 CEST 2008
  [    0.000000] Xilinx Generic PowerPC board support package (Xilinx ML405) (Virtex-4 FX)
  [    0.000000] Zone PFN ranges:
  [    0.000000]   DMA             0 ->    32768
  [    0.000000]   Normal      32768 ->    32768
  [    0.000000] Movable zone start PFN for each node
  [    0.000000] early_node_map[1] active PFN ranges
  [    0.000000]     0:        0 ->    32768
  [    0.000000] Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 32512
  [    0.000000] Kernel command line: console=ttyUL0,115200 rw root=/dev/xsa2 ip=on
  [    0.000000] Xilinx INTC #0 at 0x81800000 mapped to 0xFDFFF000
  [    0.000000] PID hash table entries: 512 (order: 9, 2048 bytes)
  [    0.000220] Console: colour dummy device 80x25
  [    0.001404] Dentry cache hash table entries: 16384 (order: 4, 65536 bytes)
  [    0.003169] Inode-cache hash table entries: 8192 (order: 3, 32768 bytes)
  [    0.039202] Memory: 127744k available (1512k kernel code, 512k data, 72k init, 0k highmem)
  [    0.132645] Mount-cache hash table entries: 512
  [    0.138371] net_namespace: 64 bytes
  [    0.142163] NET: Registered protocol family 16
  [    0.146049] Registering device uartlite:0
  [    0.147055] Registering device xsysace:0
  [    0.148184] Fixup MAC address for xilinx_lltemac:0
  [    0.148231] Registering device xilinx_lltemac:0
  [    0.149210] Registering device xilinx_iic:0
  [    0.191168] NET: Registered protocol family 2
  [    0.224847] IP route cache hash table entries: 1024 (order: 0, 4096 bytes)
  [    0.227808] TCP established hash table entries: 4096 (order: 3, 32768 bytes)
  [    0.228559] TCP bind hash table entries: 4096 (order: 2, 16384 bytes)
  [    0.228926] TCP: Hash tables configured (established 4096 bind 4096)
  [    0.228961] TCP reno registered
  [    0.240640] sysctl table check failed: /kernel/l2cr .1.31 Missing strategy
  [    0.240728] Call Trace:
  [    0.240749] [c7c17eb0] [c000812c] show_stack+0x50/0x184 (unreliable)
  [    0.240852] [c7c17ed0] [c002f8c0] set_fail+0x50/0x68
  [    0.240928] [c7c17ef0] [c002ff28] sysctl_check_table+0x650/0x698
  [    0.240979] [c7c17f20] [c002ff38] sysctl_check_table+0x660/0x698
  [    0.241027] [c7c17f50] [c001e0e4] register_sysctl_table+0x64/0xb4
  [    0.241098] [c7c17f70] [c01e85a0] register_ppc_htab_sysctl+0x18/0x2c
  [    0.241160] [c7c17f80] [c01e71e4] kernel_init+0xc8/0x284
  [    0.241202] [c7c17ff0] [c0004b18] kernel_thread+0x44/0x60
  [    0.248104] io scheduler noop registered
  [    0.248166] io scheduler anticipatory registered (default)
  [    0.248198] io scheduler deadline registered
  [    0.248438] io scheduler cfq registered
  [    0.328218] uartlite.0: ttyUL0 at MMIO 0x84000003 (irq = 8) is a uartlite
  [    0.328296] console [ttyUL0] enabled
  [    0.576648] xilinx_lltemac xilinx_lltemac.0: MAC address is now  0: a:35: 1: 2: 3
  [    0.583004] xilinx_lltemac xilinx_lltemac.0: XLlTemac: using DMA mode.
  [    0.589407] XLlTemac: Dma base address: phy: 0x84600100, virt: 0xc9008100
  [    0.596137] XLlTemac: buffer descriptor size: 32768 (0x8000)
  [    0.601715] XLlTemac: Allocating DMA descriptors with kmalloc<6>XLlTemac: (buffer_descriptor_
  init) phy: 0x7ca8000, virt: 0xc7ca8000, size: 0x8000
  [    0.621196] XTemac: PHY detected at address 7.
  [    0.624497] eth0: Dropping NETIF_F_SG since no checksum feature.
  [    0.634023] xilinx_lltemac xilinx_lltemac.0: eth0: Xilinx TEMAC at 0x81C00000 mapped to 0xC90
  04000, irq=2
  [    0.643610] mice: PS/2 mouse device common for all mice
  [    0.649926] xilinx_iic.0 #0 at 0x81600000 mapped to 0xC9020000, irq=4
  [    0.655316] TCP cubic registered
  [    0.658530] NET: Registered protocol family 1
  [    0.662793] NET: Registered protocol family 17
  [    0.668262] RPC: Registered udp transport module.
  [    0.671690] RPC: Registered tcp transport module.
  [    1.180962] eth0: XLlTemac: Options: 0x3fa
  [    1.183835] eth0: XLlTemac: allocating interrupt 0 for dma mode tx.
  [    1.190128] eth0: XLlTemac: allocating interrupt 1 for dma mode rx.
  [    3.208396] eth0: XLlTemac: speed set to 100Mb/s
  [    3.211798] eth0: XLlTemac: Send Threshold = 24, Receive Threshold = 4
  [    3.218267] eth0: XLlTemac: Send Wait bound = 254, Receive Wait bound = 254
  [    4.227661] Sending DHCP requests ., OK
  [    4.251661] IP-Config: Got DHCP answer from 0.0.0.0, my address is 192.168.1.200
  [    4.258351] IP-Config: Complete:
  [    4.260842]       device=eth0, addr=192.168.1.200, mask=255.255.255.0, gw=192.168.1.185,
  [    4.268761]      host=genepy, domain=, nis-domain=(none),
  [    4.274123]      bootserver=0.0.0.0, rootserver=192.168.1.185, rootpath=/media/rootfs
  [    4.283254] Looking up port of RPC 100003/2 on 192.168.1.185
  [    4.297715] Looking up port of RPC 100005/1 on 192.168.1.185
  [    4.343925] VFS: Mounted root (nfs filesystem).
  [    4.347553] Freeing unused kernel memory: 72k init
  init started: BusyBox v1.10.1 (2008-04-24 12:26:38 CEST)
  
  Welcome to the embedded guinevere platform. Build with buildroot by Dargaud
  genepy login: root
  Password:
  [root@genepy ~]# uname -a
  Linux genepy 2.6.24-rc8-xlnx #14 Wed Apr 23 16:57:11 CEST 2008 ppc unknown
  [root@genepy ~]#
  
  

Network operation of ML405 card

After the basic settings of the Xilinx ML405 card, let's get onto network operations. On this page, 192.168.1.185 is the host PC running Linux (user guinevere) and 192.168.1.200/genepy is the target ML405 Xilinx board running a smallish busybox/uClibc Linux always as root.

Network configuration

• The only network device I managed to get working on the card is the LLTEMAC, in DMA GMII mode, after lots of trial and error.
• Add this to your host /etc/hosts file (genepy is the hostname of my target):

  192.168.1.200 genepy genepy
  

• Various network tools can be compiled into buildroot
• DropBear is a lightweight ssh server. It can take a few tweaks in the build_powerpc/dropbear-0.51/options.h and build_powerpc/dropbear-0.51/config.h files: disable openpty, X forwarding... But the main thing is to define the pty in the device_table.txt file. Once you have dropbear running on the target, consider the following:
  • You need to set a root password before you can ssh into the target.
  • Do tail -f /tmp/messages & ssh localhost from target to target to test it and create a .ssh directory
  • Take the public key (generated by OpenSSH) from your host and add it to the target-permitted list for passwordless logins:

    scp ~/.ssh/id_dsa.pub root@genepy:.ssh/authorized_keys
    

  • Now let's convert the dropbear key and add it to the authorized list on our host (NOTE: this doesn't work):

    ssh -x root@genepy "dropbearkey -y -t dss -f /etc/dropbear/dropbear_dss_host_key | tail -2 | head -1" >>~/.ssh/authorized_keys
    ssh -x root@genepy "dropbearkey -y -t rsa -f /etc/dropbear/dropbear_rsa_host_key | tail -2 | head -1" >>~/.ssh/authorized_keys
    

• Back it all up for the next time you rebuild the filesystem:

  scp -r root@genepy:/etc/dropbear project_build_powerpc/genepy/root/etc/
  scp -r root@genepy:.ssh project_build_powerpc/genepy/root/root/
  

• You can for instance use dropbear to set the date after rebooting the target system (I also compiled ntp but couldn't get it to work):

  ssh -x root@192.168.1.200 "echo $(date +"%Z") >/etc/TZ; date $(date +"%m%d%H%M%Y.%s")"
  

DHCP addressing

• The MAC address of the EmacLite defaults to 2:0:0:0:0:0 and the LLTemac defaults to 0:a:35:1:2:3. This means that if you have multiple such boards on a network, they will all have the same MAC address which is usually a problem. You can change this in arch/ppc/boot/simple/embed_config.c . Change the values on the lines:

  		bd->bi_enetaddr[0] = 0x00;
  		bd->bi_enetaddr[1] = 0x0A;
  		bd->bi_enetaddr[2] = 0x35;
  		bd->bi_enetaddr[3] = 0x01;
  		bd->bi_enetaddr[4] = 0x02;
  		bd->bi_enetaddr[5] = 0x03;
  
  

• Configure the 2nd network card of the host to work as as dhcp server: with the KDE control module, Network, eth1, set a fixed IP of 192.168.1.185 for instance.
• In /etc/sysconfig/dhcpd to bind it to the 2nd card:

  DHCPDARGS=eth1
  

• Put the following in /etc/dhcpd.conf , it covers both the IP address assignment and the network boot:

  #
  # DHCP Server Configuration file.
  #   see /usr/share/doc/dhcp*/dhcpd.conf.sample
  #
  
  allow bootp;
  allow booting;
  
  # ddns-update-style ad-hoc;
  ddns-update-style interim;
  
  subnet 192.168.1.0 netmask 255.255.255.0 {
  	default-lease-time 21600;
  	option routers 192.168.1.185;
  	option subnet-mask 255.255.255.0;
  	option ntp-servers 192.168.1.185;
  
  #	range dynamic-bootp 192.168.1.128 192.168.1.254;
  }
  
  host genepy_l {
  	# We'll probably want to use different MACs for different boards...
  	hardware ethernet 2:0:0:0:0:0;	# EMAC_LITE
  	fixed-address 192.168.1.200;
  	# server-name 192.168.1.185;
  	# Maybe this needs to be a uImage for u-boot
  	option root-path "192.168.1.185:/media/rootfs";
  }
  
  host genepy {
  	# Customize it in arch/ppc/boot/simple/embed_config.c
  	hardware ethernet 0:a:35:1:2:3;	# LLTEMAC
  	fixed-address 192.168.1.200;
  	server-name "192.168.1.185";
  	option host-name "genepy";
  	option root-path "192.168.1.185:/media/rootfs";
  }
  
  host genepy_m {
  	hardware ethernet 00:0a:35:00:22:00;	# montevista
  	fixed-address 192.168.1.200;
  	server-name "192.168.1.185";
  	option host-name "genepy_m";
  	option root-path "192.168.1.185:/media/rootfs";
  }
  

• Start the service and make it a default:

  sudo /etc/rc.d/init.d/dhcpd start
  sudo /sbin/chkconfig dhcpd on 
  

• Boot the card with the kernel option ip=on to see if you can get an IP address and:

  ping 192.168.1.185
  

NFS root mount

• Kernel compilation options: make sure to enable CONFIG_IP_PNP_BOOTP, CONFIG_NFS_FS, CONFIG_NFS_V3 and CONFIG_ROOT_NFS in addition to your network card.
• Kernel boot parameters: console=ttyUL0,9600 root=/dev/nfs rw ip=bootp
• uClibc parameters: UCLIBC_HAS_RPC, UCLIBC_HAS_FULL_RPC, UCLIBC_HAS_REENTRANT_RPC
• Buildroot parameters: CONFIG_FEATURE_HAVE_RPC, CONFIG_FEATURE_MOUNT_NFS, CONFIG_FEATURE_INETD_RPC, BR2_INET_RPC, BR2_PACKAGE_PORTMAP
• (first method) Let's mount our filesystem on the host so we can export it to the target:

  sudo mkdir /media/rootfs
  sudo mount -o loop buildroot/binaries/genepy/rootfs.powerpc.ext2 /media/rootfs
  # Remember to remount it everytime you make buildroot
  sudo mount -o remount /media/rootfs

• (first method) You can also put it in your /etc/fstab so it's always mounted and re-exported:

  /home/guinevere/ML405/buildroot/binaries/genepy/rootfs.powerpc.ext2	/media/rootfs	ext2	defaults 0 0
  

• The drawback of the method above is that if you make local modifications to your mounted filesystem from the target device, they'll be lost the next time you remake buildroot (as the exported ext2 file will be overwritten) !!! Make sure you move files around properly or work from a copy. Or better, mount a /home from another NFS source. The preferred method below will maintain changed and new files:
• (preferred method) Untar the filesystem on the exported directory, as root from the buildroot directory:

  sudo tar xvf -keep-newer-files binaries/genepy/rootfs.powerpc.tar -C /media/rootfs/
  
  

• Export the root filesystem directory via NFS in /etc/exports :

  /media/rootfs *(rw,no_root_squash,sync,insecure,fsid=0)
  

• Restart the NFS service and make sure it's always active:

  sudo /etc/rc.d/init.d/nfs restart
  sudo /sbin/chkconfig nfs on 
  

• You can monitor the DHCP and NFS exchanges with sudo tail -f /var/log/messages or at a lower level with wireshark.
• You need portmap on the target (found in the buildroot config)
• A successful boot with NFS root filesystem:

  loaded at:     00400000 004F559C
  board data at: 004F3520 004F359C
  relocated to:  0040505C 004050D8
  zimage at:     00405E48 004F211C
  avail ram:     004F6000 08000000
  
  Linux/PPC load: console=ttyUL0,115200 rw root=/dev/nfs ip=bootp
  Uncompressing Linux...done.
  Now booting the kernel
  [    0.000000] Linux version 2.6.24-rc8-xlnx (guinevere@lpsc6185x.in2p3.fr) (gcc version 4.1.2)
  #14 Wed Apr 23 16:57:11 CEST 2008
  [    0.000000] Xilinx Generic PowerPC board support package (Xilinx ML405) (Virtex-4 FX)
  [    0.000000] Zone PFN ranges:
  [    0.000000]   DMA             0 ->    32768
  [    0.000000]   Normal      32768 ->    32768
  [    0.000000] Movable zone start PFN for each node
  [    0.000000] early_node_map[1] active PFN ranges
  [    0.000000]     0:        0 ->    32768
  [    0.000000] Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 32512
  [    0.000000] Kernel command line: console=ttyUL0,115200 rw root=/dev/nfs ip=bootp
  [    0.000000] Xilinx INTC #0 at 0x81800000 mapped to 0xFDFFF000
  [    0.000000] PID hash table entries: 512 (order: 9, 2048 bytes)
  [    0.000220] Console: colour dummy device 80x25
  [    0.001404] Dentry cache hash table entries: 16384 (order: 4, 65536 bytes)
  [    0.003169] Inode-cache hash table entries: 8192 (order: 3, 32768 bytes)
  [    0.039202] Memory: 127744k available (1512k kernel code, 512k data, 72k init, 0k highmem)
  [    0.132645] Mount-cache hash table entries: 512
  [    0.138371] net_namespace: 64 bytes
  [    0.142164] NET: Registered protocol family 16
  [    0.146049] Registering device uartlite:0
  [    0.147055] Registering device xsysace:0
  [    0.148184] Fixup MAC address for xilinx_lltemac:0
  [    0.148231] Registering device xilinx_lltemac:0
  [    0.149210] Registering device xilinx_iic:0
  [    0.191168] NET: Registered protocol family 2
  [    0.224847] IP route cache hash table entries: 1024 (order: 0, 4096 bytes)
  [    0.227808] TCP established hash table entries: 4096 (order: 3, 32768 bytes)
  [    0.228559] TCP bind hash table entries: 4096 (order: 2, 16384 bytes)
  [    0.228926] TCP: Hash tables configured (established 4096 bind 4096)
  [    0.228961] TCP reno registered
  [    0.240640] sysctl table check failed: /kernel/l2cr .1.31 Missing strategy
  [    0.240728] Call Trace:
  [    0.240749] [c7c17eb0] [c000812c] show_stack+0x50/0x184 (unreliable)
  [    0.240852] [c7c17ed0] [c002f8c0] set_fail+0x50/0x68
  [    0.240928] [c7c17ef0] [c002ff28] sysctl_check_table+0x650/0x698
  [    0.240979] [c7c17f20] [c002ff38] sysctl_check_table+0x660/0x698
  [    0.241027] [c7c17f50] [c001e0e4] register_sysctl_table+0x64/0xb4
  [    0.241098] [c7c17f70] [c01e85a0] register_ppc_htab_sysctl+0x18/0x2c
  [    0.241160] [c7c17f80] [c01e71e4] kernel_init+0xc8/0x284
  [    0.241203] [c7c17ff0] [c0004b18] kernel_thread+0x44/0x60
  [    0.248107] io scheduler noop registered
  [    0.248168] io scheduler anticipatory registered (default)
  [    0.248200] io scheduler deadline registered
  [    0.248440] io scheduler cfq registered
  [    0.328251] uartlite.0: ttyUL0 at MMIO 0x84000003 (irq = 8) is a uartlite
  [    0.328330] console [ttyUL0] enabled
  [    0.576854] xilinx_lltemac xilinx_lltemac.0: MAC address is now  0: a:35: 1: 2: 3
  [    0.583216] xilinx_lltemac xilinx_lltemac.0: XLlTemac: using DMA mode.
  [    0.589617] XLlTemac: Dma base address: phy: 0x84600100, virt: 0xc9008100
  [    0.596346] XLlTemac: buffer descriptor size: 32768 (0x8000)
  [    0.601925] XLlTemac: Allocating DMA descriptors with kmalloc<6>XLlTemac: (buffer_descriptor_
  init) phy: 0x7ca8000, virt: 0xc7ca8000, size: 0x8000
  [    0.621406] XTemac: PHY detected at address 7.
  [    0.624709] eth0: Dropping NETIF_F_SG since no checksum feature.
  [    0.634232] xilinx_lltemac xilinx_lltemac.0: eth0: Xilinx TEMAC at 0x81C00000 mapped to 0xC90
  04000, irq=2
  [    0.643823] mice: PS/2 mouse device common for all mice
  [    0.650124] xilinx_iic.0 #0 at 0x81600000 mapped to 0xC9020000, irq=4
  [    0.655517] TCP cubic registered
  [    0.658730] NET: Registered protocol family 1
  [    0.662993] NET: Registered protocol family 17
  [    0.668432] RPC: Registered udp transport module.
  [    0.671856] RPC: Registered tcp transport module.
  [    1.180961] eth0: XLlTemac: Options: 0x3fa
  [    1.183829] eth0: XLlTemac: allocating interrupt 0 for dma mode tx.
  [    1.190121] eth0: XLlTemac: allocating interrupt 1 for dma mode rx.
  [    3.208390] eth0: XLlTemac: speed set to 100Mb/s
  [    3.211791] eth0: XLlTemac: Send Threshold = 24, Receive Threshold = 4
  [    3.218260] eth0: XLlTemac: Send Wait bound = 254, Receive Wait bound = 254
  [    4.227661] Sending BOOTP requests . OK
  [    4.247666] IP-Config: Got BOOTP answer from 192.168.1.185, my address is 192.168.1.200
  [    4.254912] IP-Config: Complete:
  [    4.257449]       device=eth0, addr=192.168.1.200, mask=255.255.255.0, gw=192.168.1.185,
  [    4.265333]      host=genepy, domain=, nis-domain=(none),
  [    4.270733]      bootserver=192.168.1.185, rootserver=192.168.1.185, rootpath=/media/rootfs
  [    4.280259] Looking up port of RPC 100003/2 on 192.168.1.185
  [    4.293726] Looking up port of RPC 100005/1 on 192.168.1.185
  [    4.366790] VFS: Mounted root (nfs filesystem).
  [    4.370528] Freeing unused kernel memory: 72k init
  init started: BusyBox v1.10.1 (2008-04-24 12:26:38 CEST)
  [    5.576541]   Signal: b
  [    5.576551]   Code:   30001
  [    5.576562]   Addr:   7fa65003
  
  Welcome to the embedded guinevere platform. Build with buildroot by Dargaud
  genepy login: root
  Password:
  [root@genepy ~]# uname -a
  Linux genepy 2.6.24-rc8-xlnx #14 Wed Apr 23 16:57:11 CEST 2008 ppc unknown
  

Preparing for Network kernel boot

• yum install tftp-server dhcp
• tftpd configuration on host, edit file /etc/xinetd.d/tftp and restart the service sudo /etc/rc.d/init.d/xinetd restart :

  service tftp
  {
  	socket_type		= dgram
  	protocol		= udp
  	wait			= yes
  	user			= root
  	server			= /usr/sbin/in.tftpd
  	server_args		= -s /tftpboot
  	disable			= no
  	per_source		= 11
  	cps			= 100 2
  	flags			= IPv4
  }
  

• Create an appropriate boot image (from the kernel directory):

  make ARCH=ppc CROSS_COMPILE=powerpc-linux-uclibc- uImage
  cp arch/ppc/boot/images/uImage /tftpboot/
  

TODO: Creating a U-Boot bootloader

• Das U-Boot is part of buildroot, but since it doesn't contain ML405 specific code, we need to compile a custom version from scratch. So let's start with:

  git clone git://www.denx.de/git/u-boot.git
  cd u-boot
  

• Add the following to the Makefile:

  ml405_config:	unconfig
  	@mkdir -p $(obj)include
  	@echo "#define CONFIG_ML401 1" > $(obj)include/config.h
  	@$(MKCONFIG) -a $(@:_config=) ppc ppc4xx ml405 xilinx
  

• Duplicate and update the ml401.h files:

  cp include/configs/ml401.h include/configs/ml405.h 
  cp -R board/xilinx/ml401 board/xilinx/ml405
  mv board/xilinx/ml405/ml401.c board/xilinx/ml405/ml405.c
  cp ../xps_UartLite_lltemacDma_Ace_Gpio/ppc405_0/libsrc/linux_2_6_v1_00_b/linux/arch/ppc/platforms/4xx/xparameters/xparameters_ml40x.h board/xilinx/ml405/xparameters.h
  

• Make appropriate changes to the files (that's the tough part !)
• Configure and compile U-Boot:

  export PATH=/home/guinevere/ML405/buildroot/build_powerpc/staging_dir/bin:$PATH
  make CROSS_COMPILE=powerpc-linux-uclibc-  ml405_config
  make CROSS_COMPILE=powerpc-linux-uclibc-
  

Cross-compiling applications

After the basic settings of the Xilinx ML405 card, and its network operations, let's now see how to compile and install user code on it.

Simple compilation

• Most traditional toolchain tools are not installed on the target board (though they can be), so everything needs to be compiled on the host computer and then moved onto the target. A simple hello world can be compiled thusly:

  $ export PATH=/home/guinevere/ML405/buildroot/build_powerpc/staging_dir/bin:$PATH
  $ powerpc-linux-gcc -o hello hello.c
  $ cp hello ~/ML405/buildroot/project_build_powerpc/genepy/root/bin
  

• Rebuild the filesystem (or sudo cp hello /media/rootfs/bin/

./configure method

• For traditional applications based on the ./configure; make; make install method, proceed as this example:

  $ svn co svn://svn.savannah.gnu.org/nano/trunk/nano; cd nano
  $ export PATH=/home/guinevere/ML405/buildroot/build_powerpc/staging_dir/bin:$PATH
  $ ./configure --host=powerpc-linux --prefix=/home/guinevere/ML405/buildroot/target/generic/target_skeleton
  $ make
  $ make install
  $ ll /home/guinevere/ML405/buildroot/target/generic/target_skeleton/bin
  

• Rebuild your root tree afterwards, or do the same thing again with --prefix=/home/guinevere/ML405/buildroot/project_build_powerpc/genepy/root
• Don't forget to make buildroot again if you need the ext2

DIM server

• For exchanging variable values from the target to the host, all you need is a DIM server running on the target and a DIM client as well as DIM dns running on the host.
• Compilation for the host:

  wget http://dim.web.cern.ch/dim/dim_v17r3.zip
  unzip -a dim_v17r3.zip
  cd dim_v16r13
   
  export OS=Linux
  . ./setup.sh
  gmake --warn-undefined-variables OS=Linux ODIR=linux DIMDIR=$(pwd) 
  

• now let's test it locally:

  export DIM_DNS_NODE=$(hostname)
  cd linux
  ./dns &
  ./did &
  
  # run this
  ./testServer &
  
  ./testClient &
  
  # or this
  ./test_server &
  ./test_client &
  

• Cross compilation for the target board (back to the dim_v16r13 directory):

  mkdir genepy
  gmake --warn-undefined-variables OS=Linux ODIR=linux DIMDIR=$(pwd) realclean
  gmake --warn-undefined-variables OS=Linux GUI=no ODIR=genepy ARCH=ppc DIMDIR=$(pwd) CPP=no CC=powerpc-linux-gcc CXX=powerpc-linux-gcc 
  rsync -Cavu genepy/ ~/ML405/buildroot/project_build_powerpc/genepy/root/dim/
  cp genepy/libdim.* ~/ML405/buildroot/project_build_powerpc/genepy/root/lib/