Tom's research - Revision history

Tom at 13:49, 1 August 2008

2008-08-01T13:49:25Z

← Older revision		Revision as of 15:49, 1 August 2008
Line 70:		Line 70:
	As soon as they get permission to start doing telemedicine or have other needs for the mesh it should be quite easy to just turn on the other Bokkie routers and expand the mesh rapidly. Before this is done, however, the DNS issue should be resolved.		As soon as they get permission to start doing telemedicine or have other needs for the mesh it should be quite easy to just turn on the other Bokkie routers and expand the mesh rapidly. Before this is done, however, the DNS issue should be resolved.

	For the sake of completeness		For the sake of completeness, I'll also link here to the instructions I wrote for Henry, Ndlovu's IT guy, on how to work with the Bokkie routers. This is based on the instructions Johann gave me. [[High Performance Node - Setup]]

	[[Category: WISP in a box]]		[[Category: WISP in a box]]

Tom at 13:42, 1 August 2008

2008-08-01T13:42:18Z

← Older revision		Revision as of 15:42, 1 August 2008
Line 69:		Line 69:

	As soon as they get permission to start doing telemedicine or have other needs for the mesh it should be quite easy to just turn on the other Bokkie routers and expand the mesh rapidly. Before this is done, however, the DNS issue should be resolved.		As soon as they get permission to start doing telemedicine or have other needs for the mesh it should be quite easy to just turn on the other Bokkie routers and expand the mesh rapidly. Before this is done, however, the DNS issue should be resolved.

			For the sake of completeness

	[[Category: WISP in a box]]		[[Category: WISP in a box]]

Tom at 13:41, 1 August 2008

2008-08-01T13:41:19Z

← Older revision		Revision as of 15:41, 1 August 2008
Line 52:		Line 52:


	==== Site Visits ====		== Site Visits ==

	=== Peebles Valley Visit ===		=== Peebles Valley Visit ===
Line 67:		Line 67:

	The node itself on the computer lab is mounted on the computer lab by means of a nice mast that the Ndlovu workmen welded us. It should offer no problems, but the Ethernet cable still needs to be routed inside the building and protected from the elements.		The node itself on the computer lab is mounted on the computer lab by means of a nice mast that the Ndlovu workmen welded us. It should offer no problems, but the Ethernet cable still needs to be routed inside the building and protected from the elements.

			As soon as they get permission to start doing telemedicine or have other needs for the mesh it should be quite easy to just turn on the other Bokkie routers and expand the mesh rapidly. Before this is done, however, the DNS issue should be resolved.

	[[Category: WISP in a box]]		[[Category: WISP in a box]]

Tom at 13:39, 1 August 2008

2008-08-01T13:39:42Z

← Older revision		Revision as of 15:39, 1 August 2008
Line 60:		Line 60:
	=== Ndlovu Clinic Visit ===		=== Ndlovu Clinic Visit ===

	Monday July 28th - Friday Aug 1st I was able to go to the Ndlovu Medical Clinic in Elansdoorn. There's a fairly sophisticated IT setup there, essentially a large Windows Active Directory setup with about 100 client PCs. Additionally, there's interest in using a wireless mesh to run telemedicine between the main clinic and some of the outlying Nutritional Unit sites. The telemedicine itself is currently tied up in political discussions, but		Monday July 28th - Friday Aug 1st I was able to go to the Ndlovu Medical Clinic in Elansdoorn. There's a fairly sophisticated IT setup there, essentially a large Windows Active Directory setup with about 100 client PCs. Additionally, there's interest in using a wireless mesh to run telemedicine between the main clinic and some of the outlying Nutritional Unit sites. The telemedicine itself is currently tied up in political discussions, but the network is largely in place. It uses the Bokkie routers developed at Meraka, running both IPv4 and IPv6 on the mesh. I helped Henry, Ndlovu's main IT guy, to add another node to the mesh connecting the main clinic building, where most of the computer systems reside and the Internet uplink comes in, to their computer lab, a couple hundred meters away. Most of the nodes are currently off because they can't be used for telemedicine and if they're on they risk getting fried by lightning. But with just those two nodes everything worked largely as expected.

			We configured the main clinic's node to act as a default gateway for the network, hooking in to the main clinic network. All of the computers in the lab are now set up to use the node there as their DHCP server. Traffic can be routed out to the Internet, but there's one remaining issue: the mesh itself has no DNS support, and so the computer lab nodes need to have DNS servers manually configured. Johann has indicated that this would be fairly easy to fix, however, simply by configuring the clinic node to be the mesh's DNS server then pointing it to the DNS server of the main clinic network. Johann and Henry should be in contact to resolve this last issue (pun not intended ;)

			The main challenges in getting things set up at Ndlovu was not related to the mesh so much as the rest of the clinic's setup. As mentioned, it's entirely a Microsoft domain, and there was a Microsoft ISA 2004 firewall set up at the perimeter of the network. We added another one in between the mesh and the rest of the network, to limit access and also bandwidth-limit the computer lab so it doesn't take out the entire Internet uplink. This caused all manner of chaos on the internal network; at one point the entire Active Domain stopped working. We traced the problem to a "known issue" mentioned in an MSDN article; the solution was manually changing some registry key. This was a full day and a half's worth of headache, however. After that it was relatively smooth sailing, the only other issue was configuring the newly added firewall to route to the mesh network properly. This required some fiddling with routing tables on the Windows command line, no mean feat but doable. After that we were in good shape, save the DNS problem mentioned above.

			The node itself on the computer lab is mounted on the computer lab by means of a nice mast that the Ndlovu workmen welded us. It should offer no problems, but the Ethernet cable still needs to be routed inside the building and protected from the elements.

	[[Category: WISP in a box]]		[[Category: WISP in a box]]

Tom at 13:19, 1 August 2008

2008-08-01T13:19:36Z

← Older revision		Revision as of 15:19, 1 August 2008
Line 52:		Line 52:


			==== Site Visits ====

	=== Peebles Valley Visit ===		=== Peebles Valley Visit ===

	Thursday ~~June~~ 24th - Friday ~~June~~ 25th a large delegation from Meraka went to Peebles Valley to visit the mesh there. We toured the site, David showed everyone the basic setup and Harry, the director of the clinic there, gave us a quite interesting tour of the medical facilities as well. On Friday we spent some time doing a little network debugging, replacing one of the routers on a house there and plugging back in & resetting a router at the school. We also spoke with some local guys who are maybe interested in turning the wireless mesh into some kind of business, which is clearly crucial for the network to continue functioning; as it is, we can come out and fix things as often as we like, they'll always just fall apart again. I pitched the WISP-in-a-box system to one of the people who's interested in making the mesh a business, he seemed interested but I think without an actual product to demo it's tough to really understand whether it would be useful. This kind of entrepreneur is precisely our target audience, however, so this might be an excellent first test site.		Thursday July 24th - Friday July 25th a large delegation from Meraka went to Peebles Valley to visit the mesh there. We toured the site, David showed everyone the basic setup and Harry, the director of the clinic there, gave us a quite interesting tour of the medical facilities as well. On Friday we spent some time doing a little network debugging, replacing one of the routers on a house there and plugging back in & resetting a router at the school. We also spoke with some local guys who are maybe interested in turning the wireless mesh into some kind of business, which is clearly crucial for the network to continue functioning; as it is, we can come out and fix things as often as we like, they'll always just fall apart again. I pitched the WISP-in-a-box system to one of the people who's interested in making the mesh a business, he seemed interested but I think without an actual product to demo it's tough to really understand whether it would be useful. This kind of entrepreneur is precisely our target audience, however, so this might be an excellent first test site.

			=== Ndlovu Clinic Visit ===

			Monday July 28th - Friday Aug 1st I was able to go to the Ndlovu Medical Clinic in Elansdoorn. There's a fairly sophisticated IT setup there, essentially a large Windows Active Directory setup with about 100 client PCs. Additionally, there's interest in using a wireless mesh to run telemedicine between the main clinic and some of the outlying Nutritional Unit sites. The telemedicine itself is currently tied up in political discussions, but

	[[Category: WISP in a box]]		[[Category: WISP in a box]]

Tom at 12:12, 27 July 2008

2008-07-27T12:12:18Z

← Older revision		Revision as of 14:12, 27 July 2008
Line 52:		Line 52:



			=== Peebles Valley Visit ===

			Thursday June 24th - Friday June 25th a large delegation from Meraka went to Peebles Valley to visit the mesh there. We toured the site, David showed everyone the basic setup and Harry, the director of the clinic there, gave us a quite interesting tour of the medical facilities as well. On Friday we spent some time doing a little network debugging, replacing one of the routers on a house there and plugging back in & resetting a router at the school. We also spoke with some local guys who are maybe interested in turning the wireless mesh into some kind of business, which is clearly crucial for the network to continue functioning; as it is, we can come out and fix things as often as we like, they'll always just fall apart again. I pitched the WISP-in-a-box system to one of the people who's interested in making the mesh a business, he seemed interested but I think without an actual product to demo it's tough to really understand whether it would be useful. This kind of entrepreneur is precisely our target audience, however, so this might be an excellent first test site.

	[[Category: WISP in a box]]		[[Category: WISP in a box]]

Tom: /* Single-configuration of Mesh Nodes */

2008-07-17T11:48:02Z

Single-configuration of Mesh Nodes

← Older revision		Revision as of 13:48, 17 July 2008
Line 33:		Line 33:
	One of the major goals of this project is to have a single interface from which we can make changes to all of the mesh nodes. There are several approaches to this that I can see:		One of the major goals of this project is to have a single interface from which we can make changes to all of the mesh nodes. There are several approaches to this that I can see:

	* ~~There has been the suggestion of simply designing our own web interface that directly modifies the config files of every node. I am a little concerned about how robust and modular~~ this ~~solution would be. If we want to change a component~~, ~~we need to redesign~~ the ~~web interface, and if a component~~'~~s config file changes (say, between versions, and we want~~ to ~~upgrade to get some new feature~~ or ~~patch a security hole) then the web interface could potentially break~~. ~~Furthermore, the task of this is~~ a ~~bit tedious, although certainly not impossible. In sticking with the design goal~~ of reusing existing tools (not reinventing the wheel), I think we may be better off with a solution that doesn't require us to develop as heavily for a specific task like this unless we absolutely need to.		* Orangemesh provides this functionality through ROBIN, but for the reasons mentioned above it doesn't really suit our purposes. We could try to modify it or otherwise use their update system. See below for a discussion of how

	* Alternatively, ~~Orangemesh provides this functionality through ROBIN~~, ~~but for~~ the ~~reasons mentioned above~~ it ~~doesn't really suit our purposes~~. ~~We could try~~ to ~~modify~~ it or ~~otherwise use their update system (which they must~~ have, but I~~'m unfamiliar with)~~.		* Alternatively, I have a nascent idea that rather than updating the mesh nodes by simply modifying their configuration files, we could design a small utility that would run on the mesh node connected to the server. This mesh node would have everything we needed on it (e.g. OpenWRT, BATMAN, possibly CoovaChilli, etc.). After it was satisfactorily configured, using existing interfaces hopefully, this utility would essentially make an image of the entire node, and then upload this image to every other node in turn using private-key based SSH file transfers. Each node, upon receiving such an image, would write it to disk and reboot. This is certainly a little slower than directly modifying configuration files, but I think it has the possibility to be a simpler, more robust, and much more modular solution (one could even add whole packages or so to the master node and then have those changes propagate throughout the network). The feasibility of this approach isn't certain, but I think its worth further investigation.

	* Lastly, I have a nascent idea that rather than updating the mesh nodes by simply modifying their configuation files, we could design a small utility that would run on the mesh node connected to ~~the server. This mesh node would have everything we needed on it~~ (e.g. ~~OpenWRT, BATMAN, possibly CoovaChilli, etc.~~). ~~After it was satisfactorily configured~~, ~~using existing interfaces hopefully, this utility would essentially make an image of~~ the ~~entire~~ node~~, and then upload this image~~ to ~~every other node in turn using private-key based SSH file transfers. Each node, upon receiving such an image, would write it to disk and reboot~~. This is certainly a little slower than directly modifying configuration files, but I think it has the possibility to be a simpler, more robust, and much more modular solution (one could even add whole packages or so to the master node and then have those changes propagate throughout the network). ~~The feasibility of this approach isn't certain, but I think its worth further investigation~~.		'''Update:''' The need to support configuration changes across multiple devices (e.g. Ubiquity Nanostations and Mesh Potatoes) makes the flashing system untenable. Rather, sending out update messages that are received by a client running on the mesh node seems to be a better option. This is basically how ROBIN works. See below.

	=====How a flash-based configuration solution might work=====		=====How a flash-based configuration solution might work=====

	A discussion with David Johnson made it seem like this is possible. Some specifics: The master node (node connected to server) gets configured. Once it looks good, the flash distribution utility is run. It commits all changes and writes an image file based on the master node's disk. The master node then goes through the list of every node it knows about (i.e. the rest of the mesh) and using a private SSH key initiates a script on each of those nodes. That script goes and fetches the image from the master node. When it gets it, it does some basic sanity checks (checksum, etc.) on the image, then sends a message to the master node saying it's ready. When the master node has gotten a message from every node in its list saying that that node is ready it sends out another command, telling each node to execute the change. If it doesn't get a message from every node, it informs the user which ones are missing, indicating some human intervention is needed at those nodes. Every node, upon receiving the execute command, waits twice the maximum transmission time or so, then writes the change to disc and reboots. Furthermore (and this is borrowed from the FreeBSD guys) it keeps the old image around (it should have enough RAM to do so) and upon reboot has a watchdog timer running. If it can't find the mesh after a few minutes, it will revert to the old image and reboot again. This is probably the trickiest part, but the entire system seems fairly simple. It'll take a little while for the entire process to go through, but many configuration changes can be lumped into one update, so for normal operation this should be fine. A further feature that was discussed is the possibility of all nodes, at some set time (say midnight each night), switching to a certain "update" SSID and channel and checking the master node for a more recent firmware version than what it currently has. Most nodes will be running the current firmware and immediately switch back to their normal SSID and channel, resulting in little downtime. And "lost" nodes, however, will download the correct firmware, thereby "rejoining the herd" so to speak.		A discussion with David Johnson made it seem like this is possible. Some specifics: The master node (node connected to server) gets configured. Once it looks good, the flash distribution utility is run. It commits all changes and writes an image file based on the master node's disk. The master node then goes through the list of every node it knows about (i.e. the rest of the mesh) and using a private SSH key initiates a script on each of those nodes. That script goes and fetches the image from the master node. When it gets it, it does some basic sanity checks (checksum, etc.) on the image, then sends a message to the master node saying it's ready. When the master node has gotten a message from every node in its list saying that that node is ready it sends out another command, telling each node to execute the change. If it doesn't get a message from every node, it informs the user which ones are missing, indicating some human intervention is needed at those nodes. Every node, upon receiving the execute command, waits twice the maximum transmission time or so, then writes the change to disc and reboots. Furthermore (and this is borrowed from the FreeBSD guys) it keeps the old image around (it should have enough RAM to do so) and upon reboot has a watchdog timer running. If it can't find the mesh after a few minutes, it will revert to the old image and reboot again. This is probably the trickiest part, but the entire system seems fairly simple. It'll take a little while for the entire process to go through, but many configuration changes can be lumped into one update, so for normal operation this should be fine. A further feature that was discussed is the possibility of all nodes, at some set time (say midnight each night), switching to a certain "update" SSID and channel and checking the master node for a more recent firmware version than what it currently has. Most nodes will be running the current firmware and immediately switch back to their normal SSID and channel, resulting in little downtime. And "lost" nodes, however, will download the correct firmware, thereby "rejoining the herd" so to speak.



	====How ROBIN updates work, what we can reuse====		====How ROBIN updates work, what we can reuse====

Tom: /* Positioning of the billing system */

2008-07-17T11:29:36Z

Positioning of the billing system

← Older revision		Revision as of 13:29, 17 July 2008
Line 26:		Line 26:

	'''Update''': I've gotten CoovaChilli running on a WRT running Kamikaze, configured such that the wireless is used as the default gateway (right now its in client mode tied to another AP, but could be used in ad-hoc mode for a mesh too) and the LAN hands out DHCP leases. Radius configuration is a bit tedious, so it's not quite functioning fully yet, but it seems to work no problem. This is a good thing, because it means we can run just CoovaChilli on all the mesh nodes without it affecting the wireless interface, and alongside any other software that runs on Kamikaze. There is one open problem here, however: CoovaChilli itself doesn't come with a web interface. It's interface is built in to the rest of the interface for the CoovaAP firmware. There are a few possibilities to handle this (write our own interface, rip the existing one out of the CoovaAP firmware) but none are totally desirable.		'''Update''': I've gotten CoovaChilli running on a WRT running Kamikaze, configured such that the wireless is used as the default gateway (right now its in client mode tied to another AP, but could be used in ad-hoc mode for a mesh too) and the LAN hands out DHCP leases. Radius configuration is a bit tedious, so it's not quite functioning fully yet, but it seems to work no problem. This is a good thing, because it means we can run just CoovaChilli on all the mesh nodes without it affecting the wireless interface, and alongside any other software that runs on Kamikaze. There is one open problem here, however: CoovaChilli itself doesn't come with a web interface. It's interface is built in to the rest of the interface for the CoovaAP firmware. There are a few possibilities to handle this (write our own interface, rip the existing one out of the CoovaAP firmware) but none are totally desirable.

			'''Update 2''': We will write a CoovaChilli configuration page as a component of our dashboard. It's configuration will be pushed out to all nodes just like the standard network configuration. One remaining open question: the access controller (i.e. CoovaChilli) may need to monitor traffic going inside the mesh. As long as the destination is not behind the same CoovaChilli instance as the original host then this is no problem. But if they are both behind the same CoovaChilli instance (i.e. connected to the same mesh node), it may be more difficult. Doable, but it may take some configuration, I'm just not sure. This also brings up the question of how two end hosts on the network can communicate if they're both behind separate NAT's (in the form of CoovaChilli DHCP servers). Probably some kind of coordination via the server will be necessary to rendezvous (this is how Skype works), but it may limit the services we can offer.

	====Single-configuration of Mesh Nodes====		====Single-configuration of Mesh Nodes====

Tom: /* How ROBIN updates work, what we can reuse */

2008-07-17T07:51:19Z

How ROBIN updates work, what we can reuse

← Older revision		Revision as of 09:51, 17 July 2008
Line 45:		Line 45:
	====How ROBIN updates work, what we can reuse====		====How ROBIN updates work, what we can reuse====

	ROBIN nodes periodically (via cron) run a script that contacts the dashboard (in ROBIN parlance, "update server") and sends it all of the information the dashboard requires, and in return receives an update file. This update file is comprised of sections, each corresponding to a configuration file. The /sbin/update script sends and receives this information, writes a cleaned version of it to file, and then calls /etc/init.d/settings, which expects the file to be there already, parses it by splitting it up into its various sections, and calls a different update script for each section (e.g. update-batman, update-mesh, etc). These scripts are in /usr/sbin/.		ROBIN nodes periodically (via cron) run a script that contacts the dashboard (in ROBIN parlance, "update server") and sends it all of the information the dashboard requires (in the form of a URI parameter sent to a PHP script), and in return receives an update file (that same PHP script returns the update file, so for the client all it needs to do is call `wget dashboard.com?information-the-dashboard-wants` and it receives an update file). This update file is comprised of sections, each corresponding to a configuration file. The /sbin/update script sends and receives this information, writes a cleaned version of it to file, and then calls /etc/init.d/settings, which expects the file to be there already, parses it by splitting it up into its various sections, and calls a different update script for each section (e.g. update-batman for /etc/config/batman, update-mesh for /etc/config/mesh, etc). These scripts are in /usr/sbin/.

	This aspect of ROBIN we might be able to make use of, if we wanted to take these scripts and adapt them slightly. The system is somewhat modular in that only config sections that get sent out in updates and written to config files, which is good for us. The format of the configuration updates is extremely simple, which is good as well. ROBIN seems to have a lot of other, yet-unexplored (by me) aspects, however. I don't yet know how crucial they are for the system to function, but compared to a basic OpenWRT Kamikaze install there are quite a few files specific to ROBIN spread out over the system. This is very bad for portability; we want to make it easy to implement clients for use on various hardware platforms (even initially we need to support 3: Linksys WRT, Ubiquity NanoStation and the yet-to-be-designed Mesh Potato). This would suggest we should have a simple, single program or script that handles all interaction with the dashboard and configuration.		This aspect of ROBIN we might be able to make use of, if we wanted to take these scripts and adapt them slightly. The system is somewhat modular in that only config sections that get sent out in updates and written to config files, which is good for us. The format of the configuration updates is extremely simple, which is good as well. ROBIN seems to have a lot of other, yet-unexplored (by me) aspects, however. I don't yet know how crucial they are for the system to function, but compared to a basic OpenWRT Kamikaze install there are quite a few files specific to ROBIN spread out over the system. This is very bad for portability; we want to make it easy to implement clients for use on various hardware platforms (even initially we need to support 3: Linksys WRT, Ubiquity NanoStation and the yet-to-be-designed Mesh Potato). This would suggest we should have a simple, single program or script that handles all interaction with the dashboard and configuration.

	Many other pieces of ROBIN are other software components (e.g. BATMAN, CoovaChilli) that we would want to install separately and have run independently anyway. We still want to be able to configure them, which ROBIN does through the above-mentioned system of update-* scripts. We aren't particularly interested in them, however, because we assume that they will be installed already on a system that we're writing the client for.		Many other pieces of ROBIN are other software components (e.g. BATMAN, CoovaChilli) that we would want to install separately and have run independently anyway. We still want to be able to configure them, which ROBIN does through the above-mentioned system of update-* scripts. We aren't particularly interested in them themselves, however, because we assume that they will be installed already on a system that we're writing the client for.



	[[Category: WISP in a box]]		[[Category: WISP in a box]]

Tom: /* Other thoughts & issues */

2008-07-15T12:26:11Z

Other thoughts & issues

← Older revision		Revision as of 14:26, 15 July 2008
Line 17:		Line 17:
	See [[WISP-in-a-box Way Forward]]		See [[WISP-in-a-box Way Forward]]

	==Other thoughts & issues==		==Other thoughts & issues: Past Concerns, Future Considerations==

	====Positioning of the billing system====		====Positioning of the billing system====