HPN design discussions page

FreeBSD vs Linux

Licencing

Extract form a FreeBSD vs Linux page regarding licensing: http://www.lemis.com/bsdpaper.html

How does the BSD license differ from the GNU Public license?

• Linux is available under the GNU General Public License (GPL), which is designed to eliminate closed source software. In particular, any derivative work of a product released under the GPL must also be supplied with source code if requested. By contrast, the BSD license is less restrictive: binary-only distributions are allowed. This is particularly attractive for embedded applications.

Atheros based wifi cards

We are currently using Atheros based wifi cards in our HPN's. The reason for this is that almost all the mini-PCI wifi cards on the market are Atheros based and they are by far the best supported in the open source community. Atheros worked with the open source community (Sam Leffler) and they provided a HAL (hardware abstraction layer) that interfaces between the Atheros wifi card and between the open source OS. At some stage someone release an reverse engineered HAL (called openHAL) based on a very old version from Atheros. By the end of 2008 Atheros decided to open up the source code of their latest HAL for the open source community.

FreeBSD

Pros

• All field deployments we have so far are on systems running FreeBSD; so we have great knowledge on its performance.
• We have far more than enough experience; hence turn-around for changes/additions are way too short, compared to Linux.
• Sam Leffler that wrote most of the Atheros HAL source code is also one of the big FreeBSD developers.
• All the Atheros driver code (from HAL) is now fully integrated into FreeBSD and is part of the OS.
• Works great for wireless modes, like ad-hoc, compared to Linux.
• All tests done on arm/avila boards involved only FreeBSD and none on Linux.

Cons

• With regard to the HPN, none.
• Under-appreciated

Linux

Pros

• Runs on more (+cheaper) hardware platforms.

Cons

• There is three different trees for the Atheros diver.

1) AR5K - based on a very old, the reversed engineered HAL
2) OpenWRT version - I think it is based on the Madwifi driver, but not sure
3) Madwifi - This one also have different trees. One base on the old closed HAL and one based on the new open source HAL that Atheros released. I'm not sure it they are also busy incorporating it into the OS.

• I've heard people complaining about the stability of Atheros adhoc mode under Linux.

OLSR vs B.A.T.M.A.N

This discussion is around what mesh routing protocol to use for our High Performance Nodes. Before we look into the protocols we need to understand the environment where the HPNs are going to be used in, who's going to install and maintain the network and what other system requirements will have an influence on the protocol selection.

From the project contract it is clear that the target is not a pilot/research network, but a real commercial network that must provide connectivity to several schools and government departments, as well as for the community. The installation and maintenance will be the responsibility of external (not Meraka/CSIR) partners. This means that eventually it will not be a research network and whatever protocol we select needs to be very stable and well tested. Another requirement is that the network should be so easy do install (auto configure) and maintain and that it can be done by someone in the community. (It must not need an Engineer to design/configure/tweak the network before it is usable).

The biggest requirement of any network is that it should be stable. After that you get all the other features like speed, ease of use, supported features etc. In a development network one can tolerate some degree of instability while working on new functionalities, but on a network that will be used by other parties should be rock solid and stable. The biggest factor about introducing any new protocols in a network, is that one needs a certain amount of time (years) to test, fix bugs and make sure that everything is stable and usable.

OLSR

Pros

• Proven and stable, lots of big networks using it
• Scales well
• Support both IPv4 and IPv6
• Support for Linux, FreeBSD, MacOS
• Active development

Conns

• Routing information packets can become big
• Lots of network overheads to food routing information
• Can cause high CPU loads on small CPU's in low cost HW platforms
• Has difficulty routing in the presence of multiple gateways on the mesh

B.A.T.M.A.N

B.A.T.M.A.N is a fairly new protocol with many different versions, flavors (L2, L3) and derivatives (R.O.B.I.N, Mikrotik MME). https://www.open-mesh.net/wiki/BranchesExplained

B.A.T.M.A.N Layer2

Pros

• Works on L2 and is protocol independent (support both IPv4 and IPv6)
• Can use DHCP for automatic IPv4 addressing
• Small and effective routing table

Conns

• No FreeBSD port
• Not a mature protocol, still in developing state
• Seems like B.A.T.M.A.N development is more active with the Layer3 version than the Layer2 one
• Problems with multi-radio nodes: https://lists.open-mesh.net/pipermail/b.a.t.m.a.n/2009-February/002405.html

B.A.T.M.A.N Layer3

Pros

• Small and effective routing table
• Smaller, less frequent routing information packets flooding the network
• Cannot have routing loops (according to the design, but some people seem to differ)

http://www.mail-archive.com/babel-users@lists.alioth.debian.org/msg00148.html

• Active development
• Intelligent management of multiple gateways with varying bandwidth characteristics

Conns

• No IPv6 support
• No FreeBSD port
  • In progress: http://www.mail-archive.com/b.a.t.m.a.n@open-mesh.net/msg00560.html
• Not a mature protocol, still in developing state

Random thoughts.

• Our current Meraka team only have experience with OLSR, but not with BATMAN (Except for David, but he's overseas). Maybe we need to setup a test network to get a feeling of the amount of tweaking that is needed to get it to run and how much is done automatically. In the end we want a mesh node that is totally auto configured, including all IPv4 and IPv6 addresses.
  • I (Antoine) am in the process of helping George set up a small B.A.T.M.A.N. mesh for the dashboard development work.
• A google search for a up-to-date comparison between OLSR and BATMAN did not deliver much. From a quick scan in a couple articles and comments it seems like OLSR is very stable and performs well in mesh networks, but BATMAN seems to be very promising and may be the preferred mesh protocol in future.
• With another google search I could find a couple of big operational networks based on OLSR, but I could not find any big operational BATMAN networks. If there are any big operational BATMAN networks out there then I would really like to get more information and comments on it.
  • http://wiki.opennet-initiative.de/index.php - mixed OLSR & B.A.T.M.A.N.
  • http://wiki.leipzig.freifunk.net/BATMAN - mixed OLSR & B.A.T.M.A.N.
  • http://www.open-mesh.net/wiki/BerlinExperience
  • http://www.open-mesh.net/wiki/Experience
• Meraka has an operational and stable HPN node that is based on FreeBSD. The current HPN mesh protocol is OLSR and the mesh supports both IPv4 and IPv6. The biggest drawback at the moment is that it does not have an user interface and some vi tweaking is needed.
• The exhaustion of the remaining pool of unallocated IPv4 addresses is approaching within the next few years. IPv4 will not expire or stop to work, but there will be an migration phase where both IPv4 and IPv6 will be used. Some new services and web sites will only be reachable via IPv6. The default protocol on most operating systems are IPv6 and there is a big drive from Microsoft to push IPv6. We cannot just stick our heads in the ground and ignore this fact like an Ostridge. Our current HPN have full IPv6 support and it will not make a lot of sense move backwards to something that does not support IPv6. If we do feel like BATMAN is a better choice then it will make sense to add IPv6 support to it as well.
• Is there enough development time and money on the project to add a lot of new and promising features to the HPN, or should we rather concentrate on the parts that are missing in order to get a stable network out there.