John's research

Calling this research is probably a bit wild. Anybody with better writing or organizational skills, feel free to tweak. If there are stuff missing, either add or tell me.

Here I'll try to capture the work done with our FreeBSD based wireless routers.

How the config files on our distro fit together

The idea was to try and leverage as much of FreeBSD's startup scripts and procedures as possible. This makes for less work and more testing, at least for the common parts. The standard FreeBSD startup scripts already cater for a diskless case, where /etc and /var are ramdisks and the rest can be read-only. That fits well with our case where we want the Compact Flash (CF) mostly read-only for reliability, but still needs things in /etc and /var to change during and after bootup. So we force this by creating the file /etc/diskless.

So maybe a description of the boot process is in order. I'll skip some detail that I think is not important to understand this. The kernel will at some stage get bored configuring RAM and probe for devices and look for something else to do. It will then load and run /sbin/init. Init will do some things and then run the /etc/rc script. And this is where the interesting stuff begins.

rc will try to determine if it is the diskless case. If so it will run /etc/rc.initdiskless, which will create a ramdisk over /etc and populate it from /conf/base/etc and then from /conf/default/etc. The idea being that you can have a standard /etc tree in /conf/base/ and your local changes in /conf/default/. So all the files that we change will be in /conf/default/. After that, rc.initdiskless will also create and populate a /var ramdisk. When finished, it will return to rc.

FreeBSD startup scripts reside in /etc/rc.d/ and /usr/local/etc/rc.d/ for packages. Now rc will use rcorder on those two directories to determine the dependencies and order they must be run in. /etc/rc.d/early.sh is one of the first to run. It will run /etc/rc.early if it exists. It does not exist by default.

Most startup scripts in rc.d/ can be enabled/disabled in /etc/rc.conf. Maybe that is a bit too simple. They will use a function called load_rc_config to load /etc/defaults/rc.conf, /etc/rc.conf and /etc/rc.conf.local, in that order. These files just contain shell variables for example:

hostname="arm-tst.mesh"
mesh_dns_enable="YES"

So for our distro we install /etc/rc.early where the biggest chunk of our configuration will happen. A lot of it is just to put the right stuff in /etc/rc.conf and other app specific config files. Then the rest of the normal FreeBSD startup procedure, to run the startup scripts in rc.d/ will take care of the rest.

So the most important config files are:

• rc.conf - edited by auto script if auto_update_enable and _url set
• rc.conf.mesh - included early by rc.conf with defaults for the mesh
• rc.conf.local - overrides with partly manual conf

The config directories:

• /etc - ramdisk with end results, everything is lost on a reboot
• /conf/base/etc - original FreeBSD stuff
• /conf/default/etc - changed files, our and packages
• /usr/local/etc - a symlink to /etc/usr.local.etc

The basic idea is that config overrides that are done manually, have to be done in /conf/default/etc/rc.conf.local because it is read last and so overrides previous settings.

The startup scripts will create the ramdisk /etc, then recursively copy /conf/base/etc into it and then /conf/default/etc. The real /etc needs diskless, rc, fstab and rc.initdiskless in it for this to work.

Files that will need to be twiddled by scripts on startup:

• /etc/rc.conf - interface conf
• /usr/local/etc/olsrd.conf - interface, dns
• /usr/local/etc/dhcpd.conf - dhcpd
• /etc/resolv.conf - ?
• /usr/local/etc/dnsmasq.conf - dhcp + dns ?

There are probably more.

What is there to tweak

In theory we want to end up tweaking / configuring nothing, but we are not there yet. :-)

These configs go in /conf/default/etc/rc.conf.local unless otherwise specified. You can look in rc.conf.mesh to see what the defaults are, but do not change it there. That will break upgrades.

A network needs one box that is the dns server. On that box you will need to set

mesh_dns_enable="YES"

One thing that is often needed is to swap channels on the BackBone links. You can change it with lines like this:

mesh_bbchana="140"
mesh_bbchanb="136"

If you want to give the box a name:

hostname="ajay-rtr.cids.org.za"

If you need to override everything for an interface:

ifconfig_ath0="mode 11a mediaopt adhoc channel 165 ssid ptabb"

A specific prefix for an interface:

ipv6_prefix_ath0="fd9c:6829:597c:10"

If you need to override the ipv4 address:

ipv4_addrs_npe0="10.3.2.1/24"

Common files that also might need to be tweaked are:

/conf/default/usr.local.etc/olsrd.conf
/conf/default/usr.local.etc/dnsmasq.conf

Although, if you need to, tell me so that we can see if it can be handled automagically by a script.

Installing FreeBSD on a Avila/Pronghorn Metro ARM board

Gateworks Avila boards

Note! FreeBSD detects the mini-pci cards different from the labeled order on the PC board. It detects it in this order: J3, J1, J4, J2 So the card in J3 will be ath0...

ADI Pronghorn Metro boards

Note! FreeBSD detects the mini-pci cards in reverse order from the labeling on the PC board. So it will detect from PCI3, PCI2, PCI1 to PCI0. So the card in slot PCI3 will be ath0.

• Redboot use 115200 baud, so use
• > cu -l /dev/your-serial-dev -s 115200

• Break RedBoot with ^C where it says you can. :-)
• > ip -l 10.0.0.234/24 -h 10.0.0.1 # LAB C155
• > ip -l 146.64.5.234/24 -h 146.64.5.1 # JHAY-LAB
• > load boot2

• On the ADI Pronghorn boards you might get an write error. They seem to lock the flash. Just unlock it. Look for the address in the error message and do something like this and do the fis create command again:
  • > fis free
  • > fis unlock -f 0x50060000 -l 0x20000

• > fis create boot2

• > fconfig
• Change the "run script at boot" to true
• Change the startup script to:
  • fis load boot2
  • go

• Change the timeout to 3.
• Change BOOTP enabled to false.

• Enter a . on the next option to end fconfig. Choose y to save it.

• > load -b 0x200000 kernel-avila
• > go

• Type <enter> at the shell prompt.
• > ./writedisk disk-arm-7-20070315.tgz # Normally use the latest
• > reboot

• Reboot to make sure everything is working.

Todo

Ideas that might get implemented.

• Handle extra hna{4|6} entries from rc.conf