WirelessAfrica - User contributions [en]

High Performance Node - Setup

2008-08-01T13:56:31Z

Tom:

This document is specifically for the Bokkie routers at Ndlovu, but the instructions could be adapted for other sites. It's meant for someone with networking experience but almost no Unix or Linux-specific knowledge.

<pre>
This is how you would configure a new node to be added to the mesh.
Note, however, that many of the values here would have to change, and
they can't just be arbitrary! It depends on what other nodes this one is
going to point to, so that channels and IP addresses don't interfere.
We'll make an assumption about these values and then go from there; it's
a good idea to consult with Johann or at least take a hard look at the
physical layout of the nodes to make sure that channels aren't
interfering and you're using a proper IP range. I'm attaching the
version of the layout I have, but as you know it's out of date.

Lets assume this is node number 12. So all it's IP4 addresses are going
to end with 12.

Config instructions:

Plug into the Ethernet port of the node with your laptop and get a DHCP
lease.

Look at your default gateway after you get a lease; this is the IP
address of the node.

SSH to default gateway.

You can either use Putty on Windows or the following command on *nix
[note: the "$" indicates that you can run this as a normal user, it
represents a command prompt; you actually type the part coming
afterwards. This is a standard convention.]

$ ssh coin@10.36.145.1

where 10.36.145.1 is the IP address of the router; it will be different
on each one. Enter the password when prompted.

Once you're logged into the node you should get a shell. Make yourself
the root user with the command:

$ su

It shouldn't ask you for a password. Now you're root, and you can
execute commands that start with a "#" (again, don't actually type the
"#").

First, you need to mount the entire filesystem read/write (it's
read-only by default).

# mount -u -w /

Next, edit that crucial file:

# vi /conf/default/etc/rc.conf.local

This will bring up VI, in command mode, editing this file (which may not
exist yet). Press "i" to go into insert mode, then add:

hostname="Ndlovu"
olsrd4_enable="YES"
ifconfig_ath0="10.0.10.12/24 mode 11a channel 136 mediaopt adhoc ssid
ptabb bssid 22:b7:1a:b8:9c:7f"
ifconfig_ath1="10.0.50.12/24 mode 11a channel 128 mediaopt adhoc ssid
ptabb"

Note that these values vary per-node! So change the hostname to
something reasonable, but make sure that the IP addresses and channels
are correct for the place you're putting the node. Note that each of
thoese something="something else" lines is one line, but it in some
views it can wrap.

Save and quit VI by going into command mode (hit ESC while in insert
mode), then typing ":wq" and pressing return. Once out of VI, reboot:

# shutdown -r now

You don't need to do anything in olsrd4.conf, because
the /conf/default/rc.early script on the Ndlovu routers will generate
the olsrd4.conf file, with all it's HNA entries, if you enable
olsrd4 in rc.conf.local. After the reboot you can do:

$ ps -ax | grep olsrd

And look for lines that list "olsrd4" to ensure that olsrd4 is running
(i.e. OLSR is working for IPv4).

Note that some of the non-deployed nodes may not have the right rc.early
script. If that's the the case, you can copy the /conf/default/rc.early
script from anyone the other routers, or the one from the tar file
attached to this email. To do that, you want to either use an SFTP
client on Windows to copy the rc.early file to /conf/default/rc.early
(after mounting the filesystem read-write as above!), or from a *nix
machine you can run:

$ scp /path/on/local/computer/to/rc.early coin@10.36.145.1:

Where 10.36.145.1 is again the IP of the node, your default gateway.
Then log into the box, run:

$ ls

To make sure that the file is there (in your current directory, which
should be /root). Then you can do this (note again that anything with a
"#" in front of it you need to first be root to do):

# cp rc.early /conf/default/etc/rc.early

Reboot, as above, and you should be good. Check again for olsrd4.

Debugging:

Login to router using SSH as above.

Do a "list sta" to see if you can see the other nodes. Should be
something like this:

# ifconfig ath0 list sta

ADDR AID CHAN RATE RSSI IDLE TXSEQ RXSEQ CAPS FLAG
00:80:48:50:9a:0b 0 136 18M 38 0 29660 61984 I A
00:80:48:50:9e:c1 0 136 48M 43 0 26851 7568 I A

OR, if you want to fine tune the alignment, this is a continuous
printout of the above:

# rssi -i ath0

Do a ping Ndlovu and other sites to test connectivity.

$ ping 10.0.10.4

Some general hints:

To inspect a node's routing table in FreeBSD:

# netstat -r

To make it more readable, print only IPv4 addresses:

# netstat -r -f inet

To change the default gateway of a node to 1.2.3.4:

Temporarily:

# route add default 1.2.3.4

Permanently:

Edit /conf/default/etc/rc.conf as root, and at the end of the file add
the line:

defaultrouter="1.2.3.4"

To get a list of network interfaces and their IP addresses, etc:

$ ifconfig

On your routers, ones that start npe are Ethernet, ones that start ath
are wireless.

To look at *all* traffic going through a particular interface:

# tcpdump -i npe0

To make this more readable, search for only certain lines. To filter any
command's output to only lines that include a specific text, add

| grep "search"

to the end of it. For example:

# tcpdump -i npe0 | grep "ICMP"

dumps all traffic on the npe0 interface, but only shows lines that
include "ICMP" (eg Ping traffic). Note that the search is
case-sensitive! To make it not so, use `grep -i "search"` instead.

To simply view a file:

$ less /path/to/file.txt

To navigate use arrow keys, to exit this press "q".

To quit any general command, particularly continuous ones like ping or
tcpdump, press Control+C.

How to use that beast of a program, VI:

http://www.cs.colostate.edu/helpdocs/vi.html
</pre>

[[Category: High Performance Node]]

Category:High Performance Node - Setup

2008-08-01T13:53:34Z

Tom: Removing all content from page

Title:High Performance Node - Setup

2008-08-01T13:51:29Z

Tom: Removing all content from page

High Performance Node - Setup

2008-08-01T13:50:57Z

Tom: New page: This document is specifically for the Bokkie routers at Ndlovu, but the instructions could be adapted for other sites. It's meant for someone with networking experience but almost no Unix ...

Title:High Performance Node - Setup

2008-08-01T13:50:28Z

Tom's research

2008-08-01T13:49:25Z

Tom:

This is a brief document outlining current thinking on the WISP-in-a-box project that I (Tom) have been working on. It is meant to eventually be merged into the rest of the WISPiab articles, but for the time being it's here so new thinking can be easily seen and commented upon.

==Use Cases==

Our basic mesh setup consists of mesh nodes that provide a wireless backhaul to a central server and gateway and end-user access in the form of ethernet connections to the mesh node. We aren't explicitly concerned with running access points; if that functionality is desired then separate devices will need to be attached to the mesh nodes to act as wireless bridges. Each mesh node will have its own captive portal, supporting multiple clients, and communicate with a central AAA server.

==Design Goals==

The mantra here is reusing existing solutions on low-cost hardware. We don't want to reinvent the wheel, and we need to do everything as cheaply as possible. The cost decisions mainly influence [hardware choice]. The goal of software reuse means we want to do as little development in-house as possible, instead relying on synthesizing a single product from various existing components. At the end of the day we may need to do some of our own development, but the decisions I'm suggesting in this document are made with the intent of minimizing that kind of work.

==Hardware Choice==

The platform for the mesh nodes will be the Linksys WRT. On the basis of cost and availability, there's really no better platform. These needs trump the technical ones that the Broadcom chipsets fail to meet. There are some consequences of this choice, however, in terms of what software can be run on the mesh nodes. More on this in later section on the mesh software.

==Network Component Layout==

See [[WISP-in-a-box Way Forward]]

==Other thoughts & issues: Past Concerns, Future Considerations==

====Positioning of the billing system====

There is some question as to where the captive portal should sit. We've thus far assumed it would reside on the nodes themselves, in the form of CoovaAP. CoovaAP is no good for this purpose, however, because in order for it to run a captive portal it automatically takes over the wireless interface to run as an access point (so it can't be used for the mesh backhaul). Just using the CoovaChilli component, however, and swapping around it's normal outgoing / incoming interfaces, might work. The idea would be to run BATMAN on the wireless interface, tell CoovaChilli that the wireless interface is its outgoing interface, and then somehow bridge the LAN ports together and run a DHCP server only on them, not on the wireless. This seems easy enough, but as far as I can tell has never really been done, so this is a good point for experimentation.

An alternate strategy would be to have the captive portal reside on the server/gateway. This wouldn't allow for control of traffic inside the mesh, but that's probably all right (local traffic isn't what's going to be billed). This has the distinct advantage of only having *one* instance in the entire network, so configuration changes made to it don't need to be pushed out to all mesh nodes. The problem with this method is that it needs to be implemented at Layer 3, because clients won't be on the same Ethernet segment as the captive portal (they'll be multiple wireless hops away). CoovaChilli is a Layer 2 captive portal solution, and furthermore it *needs* to be the DHCP server for its clients (there may be ways to hack it to not require this, but at least in its stock form). Layer 3 captive portals exist, and this may be worth looking into as well. If we can find one that works with RADIUS, we can just plug it in to our existing FreeRADIUS / phpMyPrepaid billing architecture.

'''Update''': I've gotten CoovaChilli running on a WRT running Kamikaze, configured such that the wireless is used as the default gateway (right now its in client mode tied to another AP, but could be used in ad-hoc mode for a mesh too) and the LAN hands out DHCP leases. Radius configuration is a bit tedious, so it's not quite functioning fully yet, but it seems to work no problem. This is a good thing, because it means we can run just CoovaChilli on all the mesh nodes without it affecting the wireless interface, and alongside any other software that runs on Kamikaze. There is one open problem here, however: CoovaChilli itself doesn't come with a web interface. It's interface is built in to the rest of the interface for the CoovaAP firmware. There are a few possibilities to handle this (write our own interface, rip the existing one out of the CoovaAP firmware) but none are totally desirable.

'''Update 2''': We will write a CoovaChilli configuration page as a component of our dashboard. It's configuration will be pushed out to all nodes just like the standard network configuration. One remaining open question: the access controller (i.e. CoovaChilli) may need to monitor traffic going inside the mesh. As long as the destination is not behind the same CoovaChilli instance as the original host then this is no problem. But if they are both behind the same CoovaChilli instance (i.e. connected to the same mesh node), it may be more difficult. Doable, but it may take some configuration, I'm just not sure. This also brings up the question of how two end hosts on the network can communicate if they're both behind separate NAT's (in the form of CoovaChilli DHCP servers). Probably some kind of coordination via the server will be necessary to rendezvous (this is how Skype works), but it may limit the services we can offer.

====Single-configuration of Mesh Nodes====

One of the major goals of this project is to have a single interface from which we can make changes to all of the mesh nodes. There are several approaches to this that I can see:

* Orangemesh provides this functionality through ROBIN, but for the reasons mentioned above it doesn't really suit our purposes. We could try to modify it or otherwise use their update system. See below for a discussion of how

* Alternatively, I have a nascent idea that rather than updating the mesh nodes by simply modifying their configuration files, we could design a small utility that would run on the mesh node connected to the server. This mesh node would have everything we needed on it (e.g. OpenWRT, BATMAN, possibly CoovaChilli, etc.). After it was satisfactorily configured, using existing interfaces hopefully, this utility would essentially make an image of the entire node, and then upload this image to every other node in turn using private-key based SSH file transfers. Each node, upon receiving such an image, would write it to disk and reboot. This is certainly a little slower than directly modifying configuration files, but I think it has the possibility to be a simpler, more robust, and much more modular solution (one could even add whole packages or so to the master node and then have those changes propagate throughout the network). The feasibility of this approach isn't certain, but I think its worth further investigation.

'''Update:''' The need to support configuration changes across multiple devices (e.g. Ubiquity Nanostations and Mesh Potatoes) makes the flashing system untenable. Rather, sending out update messages that are received by a client running on the mesh node seems to be a better option. This is basically how ROBIN works. See below.

=====How a flash-based configuration solution might work=====

A discussion with David Johnson made it seem like this is possible. Some specifics: The master node (node connected to server) gets configured. Once it looks good, the flash distribution utility is run. It commits all changes and writes an image file based on the master node's disk. The master node then goes through the list of every node it knows about (i.e. the rest of the mesh) and using a private SSH key initiates a script on each of those nodes. That script goes and fetches the image from the master node. When it gets it, it does some basic sanity checks (checksum, etc.) on the image, then sends a message to the master node saying it's ready. When the master node has gotten a message from every node in its list saying that that node is ready it sends out another command, telling each node to execute the change. If it doesn't get a message from every node, it informs the user which ones are missing, indicating some human intervention is needed at those nodes. Every node, upon receiving the execute command, waits twice the maximum transmission time or so, then writes the change to disc and reboots. Furthermore (and this is borrowed from the FreeBSD guys) it keeps the old image around (it should have enough RAM to do so) and upon reboot has a watchdog timer running. If it can't find the mesh after a few minutes, it will revert to the old image and reboot again. This is probably the trickiest part, but the entire system seems fairly simple. It'll take a little while for the entire process to go through, but many configuration changes can be lumped into one update, so for normal operation this should be fine. A further feature that was discussed is the possibility of all nodes, at some set time (say midnight each night), switching to a certain "update" SSID and channel and checking the master node for a more recent firmware version than what it currently has. Most nodes will be running the current firmware and immediately switch back to their normal SSID and channel, resulting in little downtime. And "lost" nodes, however, will download the correct firmware, thereby "rejoining the herd" so to speak.

====How ROBIN updates work, what we can reuse====

ROBIN nodes periodically (via cron) run a script that contacts the dashboard (in ROBIN parlance, "update server") and sends it all of the information the dashboard requires (in the form of a URI parameter sent to a PHP script), and in return receives an update file (that same PHP script returns the update file, so for the client all it needs to do is call `wget dashboard.com?information-the-dashboard-wants` and it receives an update file). This update file is comprised of sections, each corresponding to a configuration file. The /sbin/update script sends and receives this information, writes a cleaned version of it to file, and then calls /etc/init.d/settings, which expects the file to be there already, parses it by splitting it up into its various sections, and calls a different update script for each section (e.g. update-batman for /etc/config/batman, update-mesh for /etc/config/mesh, etc). These scripts are in /usr/sbin/.

This aspect of ROBIN we might be able to make use of, if we wanted to take these scripts and adapt them slightly. The system is somewhat modular in that only config sections that get sent out in updates and written to config files, which is good for us. The format of the configuration updates is extremely simple, which is good as well. ROBIN seems to have a lot of other, yet-unexplored (by me) aspects, however. I don't yet know how crucial they are for the system to function, but compared to a basic OpenWRT Kamikaze install there are quite a few files specific to ROBIN spread out over the system. This is very bad for portability; we want to make it easy to implement clients for use on various hardware platforms (even initially we need to support 3: Linksys WRT, Ubiquity NanoStation and the yet-to-be-designed Mesh Potato). This would suggest we should have a simple, single program or script that handles all interaction with the dashboard and configuration.

Many other pieces of ROBIN are other software components (e.g. BATMAN, CoovaChilli) that we would want to install separately and have run independently anyway. We still want to be able to configure them, which ROBIN does through the above-mentioned system of update-* scripts. We aren't particularly interested in them themselves, however, because we assume that they will be installed already on a system that we're writing the client for.

== Site Visits ==

=== Peebles Valley Visit ===

Thursday July 24th - Friday July 25th a large delegation from Meraka went to Peebles Valley to visit the mesh there. We toured the site, David showed everyone the basic setup and Harry, the director of the clinic there, gave us a quite interesting tour of the medical facilities as well. On Friday we spent some time doing a little network debugging, replacing one of the routers on a house there and plugging back in & resetting a router at the school. We also spoke with some local guys who are maybe interested in turning the wireless mesh into some kind of business, which is clearly crucial for the network to continue functioning; as it is, we can come out and fix things as often as we like, they'll always just fall apart again. I pitched the WISP-in-a-box system to one of the people who's interested in making the mesh a business, he seemed interested but I think without an actual product to demo it's tough to really understand whether it would be useful. This kind of entrepreneur is precisely our target audience, however, so this might be an excellent first test site.

=== Ndlovu Clinic Visit ===

Monday July 28th - Friday Aug 1st I was able to go to the Ndlovu Medical Clinic in Elansdoorn. There's a fairly sophisticated IT setup there, essentially a large Windows Active Directory setup with about 100 client PCs. Additionally, there's interest in using a wireless mesh to run telemedicine between the main clinic and some of the outlying Nutritional Unit sites. The telemedicine itself is currently tied up in political discussions, but the network is largely in place. It uses the Bokkie routers developed at Meraka, running both IPv4 and IPv6 on the mesh. I helped Henry, Ndlovu's main IT guy, to add another node to the mesh connecting the main clinic building, where most of the computer systems reside and the Internet uplink comes in, to their computer lab, a couple hundred meters away. Most of the nodes are currently off because they can't be used for telemedicine and if they're on they risk getting fried by lightning. But with just those two nodes everything worked largely as expected.

We configured the main clinic's node to act as a default gateway for the network, hooking in to the main clinic network. All of the computers in the lab are now set up to use the node there as their DHCP server. Traffic can be routed out to the Internet, but there's one remaining issue: the mesh itself has no DNS support, and so the computer lab nodes need to have DNS servers manually configured. Johann has indicated that this would be fairly easy to fix, however, simply by configuring the clinic node to be the mesh's DNS server then pointing it to the DNS server of the main clinic network. Johann and Henry should be in contact to resolve this last issue (pun not intended ;)

The main challenges in getting things set up at Ndlovu was not related to the mesh so much as the rest of the clinic's setup. As mentioned, it's entirely a Microsoft domain, and there was a Microsoft ISA 2004 firewall set up at the perimeter of the network. We added another one in between the mesh and the rest of the network, to limit access and also bandwidth-limit the computer lab so it doesn't take out the entire Internet uplink. This caused all manner of chaos on the internal network; at one point the entire Active Domain stopped working. We traced the problem to a "known issue" mentioned in an MSDN article; the solution was manually changing some registry key. This was a full day and a half's worth of headache, however. After that it was relatively smooth sailing, the only other issue was configuring the newly added firewall to route to the mesh network properly. This required some fiddling with routing tables on the Windows command line, no mean feat but doable. After that we were in good shape, save the DNS problem mentioned above.

The node itself on the computer lab is mounted on the computer lab by means of a nice mast that the Ndlovu workmen welded us. It should offer no problems, but the Ethernet cable still needs to be routed inside the building and protected from the elements.

As soon as they get permission to start doing telemedicine or have other needs for the mesh it should be quite easy to just turn on the other Bokkie routers and expand the mesh rapidly. Before this is done, however, the DNS issue should be resolved.

For the sake of completeness, I'll also link here to the instructions I wrote for Henry, Ndlovu's IT guy, on how to work with the Bokkie routers. This is based on the instructions Johann gave me. [[High Performance Node - Setup]]

[[Category: WISP in a box]]

Category:High Performance Node - Setup

2008-08-01T13:47:55Z

Tom:

Category:High Performance Node - Setup

2008-08-01T13:46:51Z

Tom:

This document is specifically for the Bokkie routers at Ndlovu, but the instructions could be adapted for other sites. It's meant for someone with networking experience but almost no Unix or Linux-specific knowledge.

This is how you would configure a new node to be added to the mesh.
Note, however, that many of the values here would have to change, and
they can't just be arbitrary! It depends on what other nodes this one is
going to point to, so that channels and IP addresses don't interfere.
We'll make an assumption about these values and then go from there; it's
a good idea to consult with Johann or at least take a hard look at the
physical layout of the nodes to make sure that channels aren't
interfering and you're using a proper IP range. I'm attaching the
version of the layout I have, but as you know it's out of date.

Lets assume this is node number 12. So all it's IP4 addresses are going
to end with 12.

Config instructions:

Plug into the Ethernet port of the node with your laptop and get a DHCP
lease.

Look at your default gateway after you get a lease; this is the IP
address of the node.

SSH to default gateway.

You can either use Putty on Windows or the following command on *nix
[note: the "$" indicates that you can run this as a normal user, it
represents a command prompt; you actually type the part coming
afterwards. This is a standard convention.]

$ ssh coin@10.36.145.1

where 10.36.145.1 is the IP address of the router; it will be different
on each one. Enter the password when prompted.

Once you're logged into the node you should get a shell. Make yourself
the root user with the command:

$ su

It shouldn't ask you for a password. Now you're root, and you can
execute commands that start with a "#" (again, don't actually type the
"#").

First, you need to mount the entire filesystem read/write (it's
read-only by default).

# mount -u -w /

Next, edit that crucial file:

# vi /conf/default/etc/rc.conf.local

This will bring up VI, in command mode, editing this file (which may not
exist yet). Press "i" to go into insert mode, then add:

hostname="Ndlovu"
olsrd4_enable="YES"
ifconfig_ath0="10.0.10.12/24 mode 11a channel 136 mediaopt adhoc ssid
ptabb bssid 22:b7:1a:b8:9c:7f"
ifconfig_ath1="10.0.50.12/24 mode 11a channel 128 mediaopt adhoc ssid
ptabb"

Note that these values vary per-node! So change the hostname to
something reasonable, but make sure that the IP addresses and channels
are correct for the place you're putting the node. Note that each of
thoese something="something else" lines is one line, but it in some
views it can wrap.

Save and quit VI by going into command mode (hit ESC while in insert
mode), then typing ":wq" and pressing return. Once out of VI, reboot:

# shutdown -r now

You don't need to do anything in olsrd4.conf, because
the /conf/default/rc.early script on the Ndlovu routers will generate
the olsrd4.conf file, with all it's HNA entries, if you enable
olsrd4 in rc.conf.local. After the reboot you can do:

$ ps -ax | grep olsrd

And look for lines that list "olsrd4" to ensure that olsrd4 is running
(i.e. OLSR is working for IPv4).

Note that some of the non-deployed nodes may not have the right rc.early
script. If that's the the case, you can copy the /conf/default/rc.early
script from anyone the other routers, or the one from the tar file
attached to this email. To do that, you want to either use an SFTP
client on Windows to copy the rc.early file to /conf/default/rc.early
(after mounting the filesystem read-write as above!), or from a *nix
machine you can run:

$ scp /path/on/local/computer/to/rc.early coin@10.36.145.1:

Where 10.36.145.1 is again the IP of the node, your default gateway.
Then log into the box, run:

$ ls

To make sure that the file is there (in your current directory, which
should be /root). Then you can do this (note again that anything with a
"#" in front of it you need to first be root to do):

# cp rc.early /conf/default/etc/rc.early

Reboot, as above, and you should be good. Check again for olsrd4.

Debugging:

Login to router using SSH as above.

Do a "list sta" to see if you can see the other nodes. Should be
something like this:

# ifconfig ath0 list sta

ADDR AID CHAN RATE RSSI IDLE TXSEQ RXSEQ CAPS FLAG
00:80:48:50:9a:0b 0 136 18M 38 0 29660 61984 I A
00:80:48:50:9e:c1 0 136 48M 43 0 26851 7568 I A

OR, if you want to fine tune the alignment, this is a continuous
printout of the above:

# rssi -i ath0

Do a ping Ndlovu and other sites to test connectivity.

$ ping 10.0.10.4

Some general hints:

To inspect a node's routing table in FreeBSD:

# netstat -r

To make it more readable, print only IPv4 addresses:

# netstat -r -f inet

To change the default gateway of a node to 1.2.3.4:

Temporarily:

# route add default 1.2.3.4

Permanently:

Edit /conf/default/etc/rc.conf as root, and at the end of the file add
the line:

defaultrouter="1.2.3.4"

To get a list of network interfaces and their IP addresses, etc:

$ ifconfig

On your routers, ones that start npe are Ethernet, ones that start ath
are wireless.

To look at *all* traffic going through a particular interface:

# tcpdump -i npe0

To make this more readable, search for only certain lines. To filter any
command's output to only lines that include a specific text, add

| grep "search"

to the end of it. For example:

# tcpdump -i npe0 | grep "ICMP"

dumps all traffic on the npe0 interface, but only shows lines that
include "ICMP" (eg Ping traffic). Note that the search is
case-sensitive! To make it not so, use `grep -i "search"` instead.

To simply view a file:

$ less /path/to/file.txt

To navigate use arrow keys, to exit this press "q".

To quit any general command, particularly continuous ones like ping or
tcpdump, press Control+C.

How to use that beast of a program, VI:

http://www.cs.colostate.edu/helpdocs/vi.html

Category:High Performance Node - Setup

2008-08-01T13:46:33Z

This document is specifically for the Bokkie routers at Ndlovu, but the instructions could be adapted for other sites. It's meant for someone with networking experience but almost no Unix or Linux-specific knowledge.

<nowiki>
This is how you would configure a new node to be added to the mesh.
Note, however, that many of the values here would have to change, and
they can't just be arbitrary! It depends on what other nodes this one is
going to point to, so that channels and IP addresses don't interfere.
We'll make an assumption about these values and then go from there; it's
a good idea to consult with Johann or at least take a hard look at the
physical layout of the nodes to make sure that channels aren't
interfering and you're using a proper IP range. I'm attaching the
version of the layout I have, but as you know it's out of date.

Lets assume this is node number 12. So all it's IP4 addresses are going
to end with 12.

Config instructions:

Plug into the Ethernet port of the node with your laptop and get a DHCP
lease.

Look at your default gateway after you get a lease; this is the IP
address of the node.

SSH to default gateway. Use this username and password:

username = coin
password = coin#123

You can either use Putty on Windows or the following command on *nix
[note: the "$" indicates that you can run this as a normal user, it
represents a command prompt; you actually type the part coming
afterwards. This is a standard convention.]

$ ssh coin@10.36.145.1

where 10.36.145.1 is the IP address of the router; it will be different
on each one. Enter the password when prompted.

Once you're logged into the node you should get a shell. Make yourself
the root user with the command:

$ su

It shouldn't ask you for a password. Now you're root, and you can
execute commands that start with a "#" (again, don't actually type the
"#").

First, you need to mount the entire filesystem read/write (it's
read-only by default).

# mount -u -w /

Next, edit that crucial file:

# vi /conf/default/etc/rc.conf.local

This will bring up VI, in command mode, editing this file (which may not
exist yet). Press "i" to go into insert mode, then add:

hostname="Ndlovu"
olsrd4_enable="YES"
ifconfig_ath0="10.0.10.12/24 mode 11a channel 136 mediaopt adhoc ssid
ptabb bssid 22:b7:1a:b8:9c:7f"
ifconfig_ath1="10.0.50.12/24 mode 11a channel 128 mediaopt adhoc ssid
ptabb"

Note that these values vary per-node! So change the hostname to
something reasonable, but make sure that the IP addresses and channels
are correct for the place you're putting the node. Note that each of
thoese something="something else" lines is one line, but it in some
views it can wrap.

Save and quit VI by going into command mode (hit ESC while in insert
mode), then typing ":wq" and pressing return. Once out of VI, reboot:

# shutdown -r now

You don't need to do anything in olsrd4.conf, because
the /conf/default/rc.early script on the Ndlovu routers will generate
the olsrd4.conf file, with all it's HNA entries, if you enable
olsrd4 in rc.conf.local. After the reboot you can do:

$ ps -ax | grep olsrd

And look for lines that list "olsrd4" to ensure that olsrd4 is running
(i.e. OLSR is working for IPv4).

Note that some of the non-deployed nodes may not have the right rc.early
script. If that's the the case, you can copy the /conf/default/rc.early
script from anyone the other routers, or the one from the tar file
attached to this email. To do that, you want to either use an SFTP
client on Windows to copy the rc.early file to /conf/default/rc.early
(after mounting the filesystem read-write as above!), or from a *nix
machine you can run:

$ scp /path/on/local/computer/to/rc.early coin@10.36.145.1:

Where 10.36.145.1 is again the IP of the node, your default gateway.
Then log into the box, run:

$ ls

To make sure that the file is there (in your current directory, which
should be /root). Then you can do this (note again that anything with a
"#" in front of it you need to first be root to do):

# cp rc.early /conf/default/etc/rc.early

Reboot, as above, and you should be good. Check again for olsrd4.

Debugging:

Login to router using SSH as above.

Do a "list sta" to see if you can see the other nodes. Should be
something like this:

# ifconfig ath0 list sta

ADDR AID CHAN RATE RSSI IDLE TXSEQ RXSEQ CAPS FLAG
00:80:48:50:9a:0b 0 136 18M 38 0 29660 61984 I A
00:80:48:50:9e:c1 0 136 48M 43 0 26851 7568 I A

OR, if you want to fine tune the alignment, this is a continuous
printout of the above:

# rssi -i ath0

Do a ping Ndlovu and other sites to test connectivity.

$ ping 10.0.10.4

Some general hints:

To inspect a node's routing table in FreeBSD:

# netstat -r

To make it more readable, print only IPv4 addresses:

# netstat -r -f inet

To change the default gateway of a node to 1.2.3.4:

Temporarily:

# route add default 1.2.3.4

Permanently:

Edit /conf/default/etc/rc.conf as root, and at the end of the file add
the line:

defaultrouter="1.2.3.4"

To get a list of network interfaces and their IP addresses, etc:

$ ifconfig

On your routers, ones that start npe are Ethernet, ones that start ath
are wireless.

To look at *all* traffic going through a particular interface:

# tcpdump -i npe0

To make this more readable, search for only certain lines. To filter any
command's output to only lines that include a specific text, add

| grep "search"

to the end of it. For example:

# tcpdump -i npe0 | grep "ICMP"

dumps all traffic on the npe0 interface, but only shows lines that
include "ICMP" (eg Ping traffic). Note that the search is
case-sensitive! To make it not so, use `grep -i "search"` instead.

To simply view a file:

$ less /path/to/file.txt

To navigate use arrow keys, to exit this press "q".

To quit any general command, particularly continuous ones like ping or
tcpdump, press Control+C.

How to use that beast of a program, VI:

http://www.cs.colostate.edu/helpdocs/vi.html

</nowiki>

Tom's research

2008-08-01T13:42:18Z

Tom:

2008-07-23T08:00:40Z

Tom:

This document remains very much a draft, but is the evolving specification for the WISP-in-a-box project

=== Overview ===

[[Image:Grafic_(Modified).png|thumb|400px]]

[[Image:Grafic-telco_(Modified).png|thumb|400px]]

The entire WISP-in-a-box system consists of:

* A central dashboard, running on the gateway server, from which:
** the network can be monitored and all node can be configured
** pre-paid vouches can be administered
** software running on the server (web & mail services, upstream connection QoS software, etc) can be administered
** all functionality will work without upstream Internet access
* Mesh nodes which:
** receive configuration updates from the central dashboard
** run a captive portal that interacts with clients and authenticates against the gateway server
** run intra-mesh QoS software
** route themselves intelligently using BATMAN

=== Component Diagram ===

The basic setup looks like the digram to the right. For the Village Telco project, many components are the same. There is some additional functionality that's desired, however, that can reside on top of the base WISP-in-a-box system.

This document is going to mainly focus on specifying the needs of the mesh & networking side of things, with an eye towards exactly how voice-related services will be integrated (but not an elaborate discussion of the technologies themselves).

=== Dashboard & Server ===
==== Existing Dashboard Projects ====
The OrangeMesh and Open-Mesh dashboards provide almost exactly the functionality we require. However, they have a number of problems:

* Both dashboards require the nodes to be running ROBIN firmware to receive configuration updates. ROBIN, as it is now, only runs on Atheros-based devices, not Broadcom devices (like the Linksys WRT). Some semi-serious hacking will be required to add support for Broadcom devices into ROBIN, since right now it assumes the use of the Atheros-only MadWifi drivers.
* With an eye towards localization & modularity, the OrangeMesh sources aren't quite what we'd like them to be.

Because of these limitations, we may not want to use the OrangeMesh/Open-Mesh dashboard. We still want to imitate the same basic style of system, however, while adding some functionality to meet the additional needs of our dashboard (e.g. ability to administer vouchers and server services, things that the OrangeMesh/Open-Mesh dashboards don't cover).

==== Our Approach ====
We will start essentially from scratch on the dashboard, with the OrangeMesh dashboard as an example to work off of and borrow heavily from.

The dashboard we will design will be highly modular, with a high-level menu in which the user can select a module to work with. A module might be an existing software component or a new one we've written. In the case of Network Status and Network Configuration, we can use the Orangemesh-Openmesh/ROBIN setup as a reference, and where appropriate can adapt their code to suit our needs. Other tools to borrow from include Zeroshell for network QoS configuration. Essential modules include:

* Server Administration -- brings up Webmin
** This will not only allow us to configure the server, but also execute arbitrary commands on all the mesh nodes using Webmin's cluster functionality. We will strip down a version of Webmin to run on the client nodes and use Webmin's cluster modules on the server to communicate with them.
* Vouchers -- brings up phpMyPrepaid, our data billing control system
** phpMyPrepaid additionally controls the CoovaChilli instances running on all routers, including the ability to restrict or permit free access to the rest of the mesh network
** We need to also design in some way of dumping the user usage data that FreeRADIUS stores so that Michael Jenkins can do usage studies on it. The local user needs to be able to easily dump this data and send it back to the CSIR for study.
* Network Status -- new custom dashboard component with following features:

* A semi-detailed list of all the nodes in the mesh, including:
** the node's IP and MAC addresses
** the node's current status (simple up or down)
** when they were last heard from (when an update from that node was last received)
** how many hops and/or what route the node currently has to the gateway
* Basic network usage statistics, data taken from RADIUS:
** Total bandwidth going out of the gateway in the last X amount of time
** Heaviest users of the network to monitor abuse (further information on individual users is available already in phpMyPrepaid)
* Ability to test network throughput, either through:
** Simple interface to run iperf between any two nodes, or:
** Visual display of metrics & signal strengths of every link, data reported from clients' BATMAN daemons and network tools

* Backbone Network Configuration -- new custom dashboard component with following features (note this is designed to be minimal for ease of use):
** Change SSID of mesh backhaul
** Change channel of mesh backhaul
** Other features? Open to discussion.
* QoS Network Configuration – new custom dashboard component for adjusting QoS settings both:
** On the server, for outgoing traffic going to the upstream connection
** On the various nodes in the mesh, for intra-mesh network bandwidth management.
* AP-only node configuration – new custom dashboard component that configures node that are only AP's, not part of the mesh. Configures:
** SSID of AP
** Channel of AP
** Any kind of WPA authentication a user might want to run on the AP

Possible other voice-related modules include Asterisk and A2billing pages.

The dashboard will send out all configuration updates using a standardized API; any mesh node in the network will have to run a client implementing this API that receives configuration messages from the dashboard and makes the necessary changes to files on its system. The API will be just as modular as the dashboard, so if a user wants to only implement certain features (say, add billing to an existing network) she would only need to implement a client for the platform her nodes are running (or, ideally, there will already be a client for that platform). Through the dashboard the user could then disable the unwanted modules. The dashboard will only send out updates for the components that are enabled, and the client will only modify configuration files for the components it receives.

All of the components of the larger dashboard will be skinned in a unified CSS style, with the potential for branding built in.

=== Mesh Nodes ===
The mesh nodes themselves will run OpenWRT. Node software includes:

* Batman for routing
* CoovaChilli as a local captive portal
* Intra-mesh QoS software, configurable from the dashboard. Uncertain as to what tools exactly to use here, however borrowing from Zeroshell it may be possible to simply use netfilter/iptables & l7-filter. Using an existing tool here would be very nice, as this is a complex task.
* A client daemon modeled on the ROBIN scripts that receives updates from the dashboard and sends any necessary information back to the dashboard. The client daemon will have to be co-developed with the dashboard.
* Stripped-down Webmin server component so we can run remote Webmin commands on the node through the server's Webmin installation.
* In addition to configuration through the dashboard, we will include a minimal web interface for configuring each individual node. This interface will run on the nodes, and can be a stripped-down version of X-WRT's webif^2 or FFLuCI. It would be useful if the interface could be designed such that it would run on any device regardless of that device's platform. This may be easy to do with existing abstractions in OpenWRT, but an alternative method would be for this interface to change the configuration of the local node via the update API, which is guaranteed to be fully abstracted, and let the update client for that device take care of making the actual changes.

=== AP-only Nodes ===
The mesh nodes will use their radios exclusively for the mesh backhaul. Clients can be attached directly to the Ethernet ports of these nodes, however it may be useful to have wireless client access as well. In this case, a second device can be attached to the mesh node to act as an access point. These devices can also be Linksys WRTs or similar. They will be bridged to the DHCP server of the captive portal running on the mesh node, and so need no special functionality.

* If central configuration isn't desirable then any device that can operate as a pure AP without a DHCP server will do (the Linksys stock firmware, for instance).
* If configuration via the dashboard is desired, then the nodes will run OpenWRT and a barebones dashboard client that allows for configuration of the AP's SSID, etc.

=== Task Breakdown ===
Once the specification is agreed upon work can begin. At that time further task breakdowns may be useful, but at a high level the expectation currently is as follows (refer to above for details on what individual software piece entail):

# Dashboard & first client development: Meraka
## Further examination of ROBIN to see which parts of it can be modified for our purposes
## Adaptation of Webmin's cluster tools for use on the mesh
## Update API design & specification
## Dashboard framework design and implementation
## Mesh update client framework design and implementation
## Dashboard module implementation
### Network Monitoring
### Network Configuration – basics (Channel, SSID, etc)
### Network Configuration – QoS settings
### Network Configuration – AP-only settings
### Adapt phpMyPrepaid's CoovaChilli configuration to send out updates via standardized API rather than only changing local configuration file.
## Co-development with dashboard modules: mesh client update modules for each dashboard component, designed initially for the WRT but as portable as possible (for most modules portability should be trivial, as the software on the node the update client is configuring will be the same).
## CSS skinning for all dashboard components (including pre-existing ones such as Webmin and phpMyPrepaid).

Notes: These need to be done relatively sequentially, however once the dashboard and mesh client frameworks are in place different modules can be developed in parallel.

# Minimal interface development for starter devices (at least the WRT, Mesh Potato, and Ubiquity nodes): Inveneo
# Porting of mesh client to Ubiquity and Mesh Potato (most modules should be identical, but basic network configuration settings may be device-specific): Inveneo

[[Category:WISP in a box]]

WISP-in-a-box Way Forward

2008-07-23T08:00:10Z

Tom:

WISP-in-a-box Way Forward

2008-07-23T07:59:43Z

Tom:

2008-07-23T07:55:41Z

Tom:

File:Grafic (Modified).png

2008-07-23T07:55:17Z

Tom:

2008-07-17T12:14:53Z

Tom:

File:Wisp-network-diagram.png

2008-07-17T12:14:03Z

Tom:

WISP-in-a-box Way Forward

2008-07-17T12:11:44Z

Tom:

This is a draft of the spec for the WISP-in-a-box development. It is fairly up-to-date but may not be the latest version.

=== Overview ===
The entire WISP-in-a-box system consists of:

* A central dashboard, running on the gateway server, from which:
** the network can be monitored and all node can be configured
** pre-paid vouches can be administered
** software running on the server (web & mail services, etc) can be administered
** all functionality will work without upstream Internet access
* Mesh nodes which:
** receive configuration updates from the central dashboard
** run a captive portal that interacts with clients and authenticates against the gateway server
** route themselves intelligently using BATMAN

=== Component Diagram ===
[[Image:|thumb|''Illustration 1: WISP-in-a-box basic components'']]The basic setup looks like this:

For the Village Telco project, many components are the same. There is some additional functionality that's desired, however, so the steup might look more like:

[[Image:|thumb|''Illustration 2: Village Telco network, built on top of WISP-in-a-box mesh'']]

This document is going to mainly focus on specifying the needs of the mesh & networking side of things, with an eye towards exactly how voice-related services will be integrated (but not an elaborate discussion of the technologies themselves).

=== Dashboard & Server ===
==== Existing Projects ====
The OrangeMesh and Open-Mesh dashboards provide almost exactly the functionality we require. However, they have a number of problems:

* Both dashboards require the nodes to be running ROBIN firmware to receive configuration updates. ROBIN, as it is now, only runs on Atheros-based devices, not Broadcom devices (like the Linksys WRT). Some semi-serious hacking will be required to add support for Broadcom devices into ROBIN, since right now it assumes the use of the Atheros-only MadWifi drivers.
* With an eye towards localization & modularity, the OrangeMesh sources aren't quite what we'd like them to be.

Because of these limitations, we may not want to use the OrangeMesh/Open-Mesh dashboard. We still want to imitate the same basic style of system, however, while adding some functionality to meet the additional needs of our dashboard (e.g. ability to administer vouchers and server services, things that the OrangeMesh/Open-Mesh dashboards don't cover).

==== Our Approach ====
We will start essentially from scratch on the dashboard, with the OrangeMesh dashboard as an example to work off of and borrow heavily from.

The dashboard we will design will be highly modular, with a high-level menu in which the user can select a module to work with. A module might be an existing software component or a new one we've written. In the case of Network Status and Network Configuration, we can use the Orangemesh/Openmesh/ROBIN setup as a reference, and where appropriate can adapt their code to suit our needs. Other tools to borrow from include Zeroshell for network QoS configuration. Essential modules include:

* Server Administration -- brings up Webmin
* Vouchers -- brings up phpMyPrepaid, our data billing control system
** phpMyPrepaid additionally controls the CoovaChilli instances running on all routers, including the ability to restrict or permit free access to the rest of the mesh network
* Network Status -- new custom dashboard component with following features:

* A semi-detailed list of all the nodes in the mesh, including:
** the node's IP and MAC addresses
** the node's current status (simple up or down)
** when they were last heard from (when an update from that node was last received)
* Basic network usage statistics, data taken from RADIUS:
** Total bandwidth going out of the gateway in the last X amount of time
** Heaviest users of the network to monitor abuse (further information on individual users is available already in phpMyPrepaid)
* Ability to test network throughput, either through:
** Simple interface to run iperf between any two nodes, or:
** Visual display of metrics & signal strengths of every link, data reported from clients' BATMAN daemons and network tools

* Network Configuration -- new custom dashboard component with following features (note this is designed to be minimal for ease of use):
** Change SSID of mesh backhaul
** Change channel of mesh backhaul
** “Advanced” features:
*** push command to all nodes via SSH
*** Change QoS settings both:
**** On the server, for outgoing traffic going to the upstream connection
**** On the various nodes in the mesh, for intra-mesh network bandwidth management.
** Other features? Open to discussion. Potentially:
*** Change basic firewall features of each node (particularly, can clients access each others' computers when connected to the same CoovaChilli instance)

Possible other modules include Asterisk and A2billing pages.

The dashboard will send out all configuration updates using a standardized API; any mesh node in the network will have to run a client implementing this API that receives configuration messages from the dashboard and makes the necessary changes to files on its system. The API will be just as modular as the dashboard, so if a user wants to only implement certain features (say, add billing to an existing network) she would only need to implement a client for the platform her nodes are running (or, ideally, there will already be a client for that platform). Through the dashboard the user could then disable the unwanted modules. The dashboard will only send out updates for the components that are enabled, and the client will only modify configuration files for the components it receives.

All of the components of the larger dashboard will be skinned in a unified CSS style, with the potential for branding built in.

=== Mesh Nodes ===
The mesh nodes themselves will run OpenWRT. Node software includes:

* Batman for routing
* CoovaChilli as a local captive portal
* Intra-mesh QoS software, configurable from the dashboard. Uncertain as to what tools exactly to use here.
* A client daemon modeled on the ROBIN scripts that receives updates from the dashboard and sends any necessary information back to the dashboard. The client daemon will have to be co-developed with the dashboard.
* In addition to configuration through the dashboard, we will include a minimal web interface for configuring each individual node. This interface will run on the nodes, and can be a stripped-down version of X-WRT's webif^2 or FFLuCI.

=== Task Breakdown ===
Once the specification is agreed upon work can begin. At that time further task breakdowns may be useful, but at a high level the expectation currently is:

# Dashboard & first client development: Meraka
# Minimal interface development: Inveneo
# Potential further client development for other systems (such as those used in Village Telco): Inveneo

Tom's research

2008-07-17T11:48:02Z

Tom: /* Single-configuration of Mesh Nodes */

Tom's research

2008-07-17T11:29:36Z