Johann's Research: Difference between revisions

From WirelessAfrica
Jump to navigation Jump to search
No edit summary
 
(34 intermediate revisions by 2 users not shown)
Line 1: Line 1:
[[Category: Development Team Research Pages]]
[[Category: Development Team Research Pages]][[Category: Johann's Research Pages]]


I'm not sure if my stuff is really research, but this is some of the stuff that I'm busy with:
migrated to wireless dev
 
== High Performance Node - WEP/WPA ==
One of the requirements of the High Performance Node is to have WEP or WPA enabled between the wireless links of the mesh nodes. This is one of the functionalities that we would like to sort out before we install any of the nodes in the field. It's very difficult to change the encryption mode after an installation because the mesh nodes will lose connectivity if one node is upgraded to use WEP, while another is still using older software without WEP.
 
WEP and WPA are both methods to enable the encryption of data that it is send over the air. WEP (Wired equivalence privacy) is an older standard and it makes use of a WEP encryption engine while the newer WPA standard added the use of an AES encryption engine. WEP encryption has some security flaws embedded into the protocol and there are several tools available on the Internet that can crack WEP keys. These tools are more effective with the cracking of 64bit WEP keys than with 128bit keys and they require a large amount of captured data and processing power to crack a key. Note that not all WEP keys can be cracked, but only weak keys and that flaw was addressed and fixed by WPA.
 
Wireless adapters on Unix Systems can operate in three different modes: Client, Hostap (Access Point) and Adhoc mode. Most of the implementations of wireless (802.11) networks are based on a model where there is one Access Point with several wireless clients attached to to it.
 
[[Image:AP-client.png|center|One Access point with many nodes]]
Wireless Mesh networks make use of the less tested Adhoc mode of 802.11.
 
[[Image:Adhoc.png|center|Many nodes using Adhoc mode (many point to point links)]]
 
WEP and WPA requires a couple of kld's to be loaded before they can be configured. The following kld's should be added to loader.conf
wlan_acl_load="YES"
wlan_amrr_load="YES"
wlan_ccmp_load="YES"
wlan_tkip_load="YES"
wlan_wep_load="YES"
wlan_xauth_load="YES"
 
==== WEP ====
Enable and test WEP in Adhoc mode on the HPN.
WEP makes use of a single PSK that needs to be configured on all the wireless nodes. Any node or wireless device that is configured with this PSK will have the capability to crypt and decrypt these wireless packets. There are two methods to configure WEP in FreeBSD. You can use either  use ifconfig directly or you can make use of the WPA supplicant utility.
 
ifconfig e.g.
mesh-9e69:~ # ifconfig ath0 10.10.1.2/24 wep deftxkey 1 wepkey 128bitwepison
mesh-9e69:~ # ifconfig ath0
ath0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        ether 00:80:48:50:9e:69
        inet6 fe80::280:48ff:fe50:9e69%ath0 prefixlen 64 scopeid 0x1
        inet6 fd9c:6829:597c:20:280:48ff:fe50:9e69 prefixlen 64
        inet6 fd9c:6829:597c:20:: prefixlen 64 anycast
        inet 10.10.1.2 netmask 0xffffff00 broadcast 10.10.1.255
        media: IEEE 802.11 Wireless Ethernet autoselect mode 11g <adhoc>
        status: associated
        ssid ptamesh channel 13 (2472 Mhz 11g) bssid 56:e5:be:30:14:5a
        authmode OPEN privacy ON deftxkey 1 wepkey 1:104-bit txpower 31.5
        scanvalid 60 bgscan bgscanintvl 300 bgscanidle 250 roam:rssi11g 7
        roam:rate11g 5 protmode CTS burst
 
Test configuration:
 
    Host A ----- wep ------- Host B ----- wep Host C
 
Enable WEP on three mesh nodes and test connectivity.
mesh-9e69:~ # ping6 ff02::1%ath0
16 bytes from fe80::280:48ff:fe50:9e69%ath0, icmp_seq=87 hlim=64 time=2.122 ms
16 bytes from fe80::280:48ff:fe50:9ddd%ath0, icmp_seq=87 hlim=64 time=5.625 ms(DUP!)
16 bytes from fe80::280:48ff:fe50:9a44%ath0, icmp_seq=87 hlim=64 time=32.358 ms(DUP!)
 
Everything seems to work fine (as expeted) with WEP enabled in the mesh.
 
'''WPA'''
 
WPA was designed to work in an environment where you have one AP and several clients and not for Adhoc (mesh) networks. On the AP you have the Authenticator software and on the client you have the supplicant software. This means that if one would like to use WPA to it's full then every mesh node needs to be an Authenticator for all the other nodes it can see, as well as a supplicant for every node it can see. According to the WPA design document http://wirelessafrica.meraka.org.za/wiki/images/3/39/Wpa_supplicant-devel-04.pdf one can use WPA in Adhoc mode only in a static way with PSK's:
IEEE 802.11 operation mode (Infrastucture/IBSS).
0 = infrastructure (Managed) mode, i.e., associate with an AP.
1 = IBSS (ad-hoc, peer-to-peer)
Note: IBSS can only be used with key_mgmt NONE (plaintext and static WEP) and key_mgmt=WPA-
NONE (fixed group key TKIP/CCMP). In addition, ap_scan has to be set to 2 for IBSS. WPA-None requires
following network block options: proto=WPA, key_mgmt=WPA-NONE, pairwise=NONE, group=TKIP
(or CCMP, but not both), and psk must also be set (either directly or using ASCII passphrase).
 
This is methods is very similar to the way that WEP is being used. This means the the only advantage of using WPA over WEP is that one can make use of the AES encryption engine that comes with WPA. Please note that with this mode of WPA and with WEP that anyone can decrypt the data being send over the air if they get hold of the mesh-wide PSK that is configured on every node in the network. If security is of importance them end users should consider the use of point-to-point security mechanisms like VPN's.
 
The configuration of WPA is not supported in command line mode, but only with the wpa_supplicant software. This is an example of a wpa_supplicant.conf file to enable WPA in Adhoc mode:
 
ap_scan=2
#
network={
  ssid="ptamesh"
# Channel  13 : 2472  Mhz 11g
  frequency=2472
  mode=1
  proto=WPA
  key_mgmt=WPA-NONE
  pairwise=NONE
  group=TKIP
  psk="mesh-ipv6"
}
 
To start the WPA software during boot-up one needs to add WPA to the ifconfig line in rc.conf. E.g.
 
ifconfig_ath0="WPA 10.10.1.1/24 mode 11g mediaopt adhoc channel 13 ssid ptamesh"
 
Test configuration:
 
    Host A ----- wep ------- Host B ----- wep Host C
 
Unfortunately WPA did not work as easy as WEP, so it's off to debugging.
 
Starting wpa_supplicant in the foreground with debugging enabled. E.g
 
wpa_supplicant -d -i ath0 -c /etc/wpa_supplicant.conf
 
The debugging output shows that something happens that initiates a wlan scan operation, while the card is in Adhoc mode. This should not happen and wpa_supplicant software never completes the configuration due to this error.
 
mesh-9ddd:~ # wpa_supplicant -d -i ath0 -c /etc/wpa_supplicant.conf
Initializing interface 'ath0' conf '/etc/wpa_supplicant.conf' driver 'default'
ctrl_interface 'N/A' bridge 'N/A'
Configuration file '/etc/wpa_supplicant.conf' -> '/etc/wpa_supplicant.conf'
Reading configuration file '/etc/wpa_supplicant.conf'
ap_scan=2
Priority group 0
  id=0 ssid='ptamesh'
Initializing interface (2) 'ath0'
EAPOL: SUPP_PAE entering state DISCONNECTED
EAPOL: KEY_RX entering state NO_KEY_RECEIVE
EAPOL: SUPP_BE entering state INITIALIZE
EAP: EAP entering state DISABLED
EAPOL: External notification - portEnabled=0
EAPOL: External notification - portValid=0
Own MAC address: 00:80:48:50:9d:dd
wpa_driver_bsd_set_wpa: enabled=1
wpa_driver_bsd_set_wpa_internal: wpa=3 privacy=1
wpa_driver_bsd_del_key: keyidx=0
wpa_driver_bsd_del_key: keyidx=1
wpa_driver_bsd_del_key: keyidx=2
wpa_driver_bsd_del_key: keyidx=3
wpa_driver_bsd_set_countermeasures: enabled=0
wpa_driver_bsd_set_drop_unencrypted: enabled=1
Setting scan request: 0 sec 100000 usec
Added interface ath0
State: DISCONNECTED -> SCANNING
Trying to associate with SSID 'ptamesh'
Cancelling scan request
WPA: clearing own WPA/RSN IE
Automatic auth_alg selection: 0x1
wpa_driver_bsd_set_auth_alg alg 0x1 authmode 1
WPA: No WPA/RSN IE available from association info
WPA: Set cipher suites based on configuration
WPA: Selected cipher suites: group 8 pairwise 1 key_mgmt 16 proto 1
WPA: clearing AP WPA IE
WPA: clearing AP RSN IE
WPA: using GTK TKIP
WPA: using PTK NONE
WPA: using KEY_MGMT WPA-NONE
WPA: Set own WPA IE default - hexdump(len=24): dd 16 00 50 f2 01 01 00 00 50
f2 02 01 00 00 50 f2 00 01 00 00 50 f2 00
No keys have been configured - skip key clearing
wpa_driver_bsd_set_key: alg=TKIP addr=ff:ff:ff:ff:ff:ff key_idx=0 set_tx=1
seq_len=6 key_len=32
wpa_driver_bsd_set_drop_unencrypted: enabled=1
State: SCANNING -> ASSOCIATING
wpa_driver_bsd_associate: ssid 'ptamesh' wpa ie len 24 pairwise 0 group 2 key
mgmt 4
ioctl[SIOCS80211, op 22, len 24]: Invalid argument
Association request to the driver failed
wpa_driver_bsd_set_key: alg=TKIP addr=ff:ff:ff:ff:ff:ff key_idx=0 set_tx=1
seq_len=6 key_len=32
Cancelling authentication timeout
State: ASSOCIATING -> COMPLETED
CTRL-EVENT-CONNECTED - Connection to 00:00:00:00:00:00 completed (auth) [id=-1
id_str=]
EAPOL: External notification - portControl=ForceAuthorized
 
Next was to install fresh copy of the FreeBSD src code onto my PC. I found the wpa_supplicant source code under /usr/src/usr.sbin/wpa/wpa_supplicant/driver_freebsd.c Build a new driver from source code and run it on a router. After that I've added in a lot of printf statements to trace how the driver works and to locate the code that puts the wifi card in scanning mode. In the end I found out that wpa_supplicant driver did not put the card in scanning mode, but it somehow caused the 802.11 stack to start the scanning. Unfortunately I don't know enough about the 802.11 code in FreeBSD to do any mode debugging. I've send a mail about the problem to one of the FreeBSD mailing lists and the originator of most of the 802.11 code in FreeBSD (Sam Leffler) said I must log a problem report. So I logged a problem report with FreeBSD and now we have to wait and see if someone to take responsibility to fix this problem. More information on the problem report can be found at: http://www.freebsd.org/cgi/query-pr.cgi?pr=126822
 
For now I recommend that we make use of WEP instead of WPA in our implementation as it is working and there isn't such a big advantage in using WPA over WEP in this mode except for the encryption algorithm that changes from WEP to AES.
 
== Voip - IP04 ==
 
 
== High performance node - Antenna calculations ==
 
Every HP-node has two 5.15-5.6 GHz antennas. The one as a build in patch antenna and the other is an 5.1-5.8 GHz omni-directional antenna. The idea is that the patch antenna of a node will either be aligned to the omni or the patch antenna of another node. The following calculations is to get an feeling of what distances should be possible between these antennas.
 
South African Wifi channels:
Channel  1 : 2412  Mhz 11g          Channel  48 : 5240  Mhz 11a
Channel  2 : 2417  Mhz 11g          Channel  52 : 5260* Mhz 11a
Channel  3 : 2422  Mhz 11g          Channel  56 : 5280* Mhz 11a
Channel  4 : 2427  Mhz 11g          Channel  60 : 5300* Mhz 11a
Channel  5 : 2432  Mhz 11g          Channel  64 : 5320* Mhz 11a
Channel  6 : 2437  Mhz 11g          Channel 100 : 5500* Mhz 11a
Channel  7 : 2442  Mhz 11g          Channel 104 : 5520* Mhz 11a
Channel  8 : 2447  Mhz 11g          Channel 108 : 5540* Mhz 11a
Channel  9 : 2452  Mhz 11g          Channel 112 : 5560* Mhz 11a
Channel  10 : 2457  Mhz 11g          Channel 116 : 5580* Mhz 11a
Channel  11 : 2462  Mhz 11g          Channel 120 : 5600* Mhz 11a
Channel  12 : 2467  Mhz 11g          Channel 124 : 5620* Mhz 11a
Channel  13 : 2472  Mhz 11g          Channel 128 : 5640* Mhz 11a
Channel  36 : 5180  Mhz 11a          Channel 132 : 5660* Mhz 11a
Channel  40 : 5200  Mhz 11a          Channel 136 : 5680* Mhz 11a
Channel  44 : 5220  Mhz 11a          Channel 140 : 5700* Mhz 11a
 
HP-Node detailed specs: http://wirelessafrica.meraka.org.za/wiki/images/0/07/WLAN-A0033.pdf
 
Pigtail detail specs: http://wirelessafrica.meraka.org.za/wiki/images/0/0a/Ca178_cable_assemblies_datasheet.pdf
 
Wireless adapter detail specs: http://wirelessafrica.meraka.org.za/wiki/images/2/21/Wlm54sag23.pdf
 
OR the Mikrotik version of the same adapter called a R52H (also manufactured by Compex, but usually cheaper)
http://wirelessafrica.meraka.org.za/wiki/index.php/Image:R52H.pdf
 
Another popular adapter is the DCMA-82 from Wistron. They have better RX sensitivity, but lower output power and they are a bit more expensive than the Compex cards. In some of the discussions that I've seen they state that the DCMA-82 has much cleaner output power, better RX sensitivity, and a lower failure rate/quality issues than Compex's cards.
http://wirelessafrica.meraka.org.za/wiki/index.php/Image:DCMA-82.pdf
 
Node specs summary:
Patch antenna            - 21-23 dBi,  16 deg H,  11 deg V, 5.15 -5.6 GHz
5Ghz Omni antenna        - 8-11 dBi,    360 deg H, 10 deg V, 5.1 - 5.85 GHz
2.4Ghz Omni antenna      - 7.5-8.5 dBi, 360 deg H, 12 deg V, 2.4 - 2.5 GHz
30mm U.FL-SMA pigtail    - RG178 cable insertion loss, 2.4 GHZ = -1.1dB, 5.6 GHz = -2dB
Compex WLM54AG23 adapter - IEEE 802.11a/b/g (2.4/5GHz)
                          - AR5413/5414(AR5006X/XS) Atheros Chipset
                          - RECEIVER SENSITIVITY 802.11a -
                            -90 dBm @ 6Mbps
                            -70 dBm @ 54Mbps
                          - OUTPUT POWER 802.11a
                            23dBm @ 6-24Mbps
                            22dBm @ 36Mbps
                            19dBm @ 48Mbps
                            17dBm @ 54Mbps
Mikrotik (Compex) R52H  - IEEE 802.11a/b/g (2.4/5GHz)
                          - AR5413/5414(AR5006X/XS) Atheros Chipset
                          - RECEIVER SENSITIVITY 802.11a -
                            -90 dBm @ 6Mbps
                            -70 dBm @ 54Mbps
                          - OUTPUT POWER 802.11a
                            24dBm @ 6-24Mbps
                            22dBm @ 36Mbps
                            19dBm @ 48Mbps
Wistron DCMA-82          - IEEE 802.11a/b/g (2.4/5GHz)
                          - AR5413/5414(AR5006X/XS) Atheros Chipset
                          - RECEIVER SENSITIVITY 802.11a -
                            -90 dBm @ 6Mbps
                            -82 dBm @ 36Mbps
                            -72 dBm @ 54Mbps
                          - OUTPUT POWER 802.11a
                            22.5dBm @ 6-24Mbps
                            21.5dBm @ 36Mbps
                            18dBm @ 48Mbps
 
I could not find any receive sensitivity info for ranges between 6Mbps and 54Mbps for the WLM54AG23. The closest I could find was the specs of another manufacturer card, using the same Atheros chipset as the WLM54AG23. Here is the receive sensitivity info for a WMIA-166AG Dual-Band miniPCI module (802.11a/g)
Nominal Temp Range:
  6Mbps 10-5 BER @ -90 dBm, typical
  9Mbps 10-5 BER @ -89 dBm, typical
12Mbps 10-5 BER @ -88 dBm, typical
18Mbps 10-5 BER @ -86 dBm, typical
24Mbps 10-5 BER @ -82 dBm, typical
36Mbps 10-5 BER @ -78 dBm, typical
48Mbps 10-5 BER @ -72 dBm, typical
54Mbps 10-5 BER @ -68 dBm, typical
 
The WLM54AG23 is not the best wifi adapter in the market, but it gives a good compromise between price vs performance. If one need to push the limits of a specific link then I would probable select the SR5 from Ubiquity. They have excellent TX power and RX sensitivity.
There are lots of wifi/antenna/distance calculators available on the WEB. Most of these calculators cannot calculate the max distance for a link, you have to specify the distance and it will calculate you dBm margin. Links to a couple of Wifi/antenna/distance calculators:
 
http://www.radiolabs.com/stations/wifi_calc.html
http://www.zytrax.com/tech/wireless/calc.htm
http://www.widgetbox.com/widget/wifi-link-calculator
http://www.wifiextreme.com.au/index.php?main_page=page_5
http://www.rflinx.com/help/calculations
http://www.olotwireless.net/castella/radio.htm
 
Here are some formulas use to calculate a link budget:
  Free space loss = 36.56 + 20Log10(Frequency) + 20Log10(Dist in miles)
  mW to dBm = 10Log10(milliWatts) + 30
  dBm to mW = 10(dBm/10)
  RX Power = Margin - RX sensitivity
  Theoretical margin = TX power budget + RX power budget - free space loss
  SAD factor = Theoretical margin/TX power budget * 100 and shows the percentage of spare power on transmission.
 
(Free Space Loss) = 92.5 + (20 Log fequency) + (20 log km distance)
 
Free space loss @ 5400GHz:
1km = -107.07dB      11km = -128.028dB
2km = -113.09dB      12km = -128.784dB
3km = -116.61dB      13km = -129.479dB
4km = -119.11dB      14km = -130.123dB
5km = -121.05dB      15km = -130.722dB
6km = -122.63dB      16km = -131.282dB
7km = -123.97dB      17km = -131.809dB
8km = -125.13dB      18km = -132.305dB
9km = -126.15dB      19km = -132.775dB
10km = -127.07dB    20km = -133.221dB
 
The following are the minimum and maximum link calculations for our HPNode with a 5Ghz patch to omni link under ideal conditions with a decent fresnel zone. Just note that one very seldom get ideal conditions in real life installations. These calculations are done to get a feeling for what to expect from our equipment. Link distance calculations was done with these calculator at http://www.swisswireless.org/wlan_calc_en.html. I've first played around with the Free-space-loss values in the Link Budget tool until I would get a 6dB margin. They recommend to take a sufficient security margin (5-6 dB or more on large distances). Then I've used the Free space loss tool to convert my Free-space-loss value to a km distance. Lets do some calculations to see what we get.
 
Patch - Omni:
Min link distance @ 54Mbps 0.9km:              Max link distance @ 54Mbps 1.55km:
patch min        = 21dBi                      patch max      = 23dBm
omni min        = 8dBi                        omni max        = 11dBm
pigtail          = -2dBm                      pigtail        = -2dBm
TX power        = 17dBm                      TX power        = 17dBm
RX sensitivity  = -70dBm                      RX sensitivity  = -70dBm
Free space loss  = -106dB                      Free space loss = -111dB
link margin      = 6dB                        link margin    = 6dB
 
Min link distance @ 24Mbps 6.9km:              Max link distance @ 24Mbps 12.3km:
patch min        = 21dBi                      patch max      = 23dBm
omni min        = 8dBi                        omni max        = 11dBm
pigtail          = -2dBm                      pigtail        = -2dBm
TX power        = 23dBm                      TX power        = 23dBm
RX sensitivity  = -82dBm                      RX sensitivity  = -82dBm
34km free space  = -124dB                      Free space loss = -129dB
link margin      = 6dB                        link margin    = 6dB
 
Min link distance @ 6Mbps 17km:                Max link distance @ 6Mbps 30.9km:
patch min        = 21dBi                      patch max      = 23dBm
omni min        = 8dBi                        omni max        = 11dBm
pigtail          = -2dBm                      pigtail        = -2dBm
TX power        = 23dBm                      TX power        = 23dBm
RX sensitivity  = -90dBm                      RX sensitivity  = -90dBm
34km free space  = -132dB                      Free space loss = -137dB
link margin      = 6dB                        link margin    = 6dB
 
The following are the minimum and maximum link calculations for our HPnodes with a 5Ghz patch to patch link under ideal conditions with a decent fresnel zone. Just note that one very seldom get ideal conditions in real life installations. These calculations are done to get a feeling for what to expect from our equipment. Link distance calculations was done with these calculator at http://www.swisswireless.org/wlan_calc_en.html. I've first played around with the Free-space-loss values in the Link Budget tool until I would get a 6dB margin. They recommend to take a sufficient security margin (5-6 dB or more on large distances). Then I've used the Free space loss tool to convert my Free-space-loss value to a km distance. Lets do some calculations to see what we get.
 
Patch - Patch:
Min link distance @ 54Mbps 3.9km:              Max link distance @ 54Mbps 6km:
patch min        = 21dBi                      patch max      = 23dBm
pigtail          = -2dBm                      pigtail        = -2dBm
TX power        = 17dBm                      TX power        = 17dBm
RX sensitivity  = -70dBm                      RX sensitivity  = -70dBm
Free space loss  = -119dB                      Free space loss = -123dB
link margin      = 6dB                        link margin    = 6dB
 
Min link distance @ 24Mbps 30.9km:            Max link distance @ 24Mbps 38.9km:
patch min        = 21dBi                      patch max      = 23dBm
pigtail          = -2dBm                      pigtail        = -2dBm
TX power        = 23dBm                      TX power        = 23dBm
RX sensitivity  = -82dBm                      RX sensitivity  = -82dBm
Free space loss  = -137dB                      Free space loss = -139dB
link margin      = 6dB                        link margin    = 6dB
 
Min link distance @ 6Mbps 77.6km:              Max link distance @ 6Mbps 123km:
patch min        = 21dBi                      patch max      = 23dBm
pigtail          = -2dBm                      pigtail        = -2dBm
TX power        = 23dBm                      TX power        = 23dBm
RX sensitivity  = -90dBm                      RX sensitivity  = -90dBm
Free space loss  = -145dB                      Free space loss = -149dB
link margin      = 6dB                        link margin    = 6dB
 
== Installing cacti on FreeBSD ==
 
Recipe to install FreeBSD, add ports mysql, php5 and cacti:
http://wirelessafrica.meraka.org.za/wiki/index.php/Image:Installing_Cacti_on_FreeBSD.zip
 
If you intall it for the first time,
you may have to follow this steps to make it work correctly:
 
1. Create the MySQL database:
# mysqladmin --user=root create cacti
2. Create a mysql user/password for cacti:
    (change user and/or password if requered)
# echo "GRANT ALL ON cacti.* TO cactiuser@localhost IDENTIFIED BY 'cactiuser'; F          LUSH PRIVILEGES;" | mysql
3. Import the default cacti database:
# mysql cacti < /usr/local/share/cacti/cacti.sql
4. Edit /usr/local/share/cacti/include/config.php.
5. Add a line to your /etc/crontab file similar to:
*/5 * * * * cacti /usr/local/bin/php /usr/local/share/cacti/poller.php > /dev/nu          ll 2>&1
6. Add alias in apache config for the cacti dir:
Alias /cacti "/usr/local/share/cacti/"
7. Be sure apache gives an access to the directory ('Allow from' keywords).
8. Open a cacti login page in your web browser and login with admin/admin.
 
Add the following line in the end of httpd.conf
vi /usr/local/etc/apache22/httpd.conf
Include etc/apache/Includes/*.conf
 
vi /usr/local/etc/cacti.conf
Alias /cacti/ "/usr/local/share/cacti/"
<Location /cacti>
AddHandler php-script php
DirectoryIndex index.php
AllowOverride None
Order allow,deny
Allow from all
</Location>
 
== Community wireless networks ==
 
For the last year or so, I've been using some of my spare time to help with the creation of wireless community network in our suburb. This exercise helps me to get firsthand experience all the issues associated with community wireless networks and it lets me get a better understanding of normal network users and their needs.
 
The network started with a bunch of friends that all know each other, but when the word got out and a lot of other people wanted to join the network as well. Our network has now grown to almost 20 Nodes, running on Linksys WRT54GL routers with Freifunk firmware. We have a community backup server on the network where people can make an off-site backup of data. Internet connectivity is available via a 4Mbps ADSL link at one of the users homes. The guy with the ADSL link is also an active gamer and his only concern was that the community network will not affect his gaming.
 
Up to now it was unmonitored and free (sponsored by the guy with the ADSL link) to all the users. At this stage we have reached a point where we need to find a way for users to contribute to the costs of the Internet connection and to make sure that the rest of the network does not influence his gaming.
 
Live drawing of the nodes in our network:
[http://stats.meraka.csir.co.za/mesh/groenkloof-mesh.html]
 
=== We've decided on the following: ===
 
Log and count all Internet traffic to get some Internet usage statistics
Get a list of email addresses and telephone numbers for all the users
Create a club and let all the users join it
Open a bank account for the club (used for user deposits and Internet expenses)
All contributions/payments are voluntary
We will have two types of users: free-users and paying-users
Paying users will have full Internet access at full speed
Free users will not be capped, but their bandwidth to the Internet will be reduced to xKbps
Free users will not be restricted inside the community network
Build our own Internet Gateway that can split traffic between SA traffic and International traffic
Open two ADSL accounts: One SA only and one full Internet account
Find a way to provide some quality of service for the gaming ports
 
For now we will implement these measures and then see how it turns out. My responsibility is to work on our new Internet gateway and the following discuss some of the functions of our new gateway.
 
=== Splitting SA traffic and International traffic ===
Why do we want to split the traffic, well it makes good sense if you look at the difference in pricing between a local SA only ADSL account and a normal (SA+International) ADSL account. One can get a 1 GByte local SA only account for R10-R20 while normal 1 GByte account can be anything from R70-R120 depending if it's shaped or unshaped.
 
The first thing one need to do is to find a list of all the IP networks that are being used in South Africa. Then you need to buy two ADSL accounts, one SA-only account and one International account. Most of the ADSL routers don't have the ability to handle two different PPP accounts at the same time. This means that the ADSL router should be configured to bridge all PPP traffic, while our new router needs to handle two PPPoE accounts. Then you need to route the traffic according to it's destination. Here is a couple of methods to split the traffic.
 
On Windows there is a program called Route Sentry. This program will split and route the local and
International traffic for you via two different PPPoE accounts. This method limits you to a specific PC only.
 
Make use of IPcop in Linux. Haven't looked at this option because I'm using FreeBSD.
 
Get hold of a list of SA only IP subnets and then set-up static routes according to list of SA subnets.
 
Beg some ISP to give you a BGP feed. Ok, we don't have an ASN number, but one can make use of a
reserved-for-private-use ASN number. Unfortunately ISP's don't like to give private people BGP feeds.
 
I've decided to implement static routes according to a list of SA only subnets. SA subnets can be retrieved from public SA route servers. A Route Server serves as BGP information exchange to other BGP routers. Unlike normal BGP router, Route Server must have several routing tables for managing different routing policies for each BGP speaker. Some of these Route Servers allows public access via telnet. E.g.
 
Public routes servers in SA.
routeserver.ginx.org.za
tpr-route-server.saix.net
route-server.is.co.za
 
telnet route-server.is.co.za
terminal length 0
show ip bgp
exit
 
Here is a perl script that I wrote to get all the SA routes from route-server.is.co.za. Then it adds a mask to routes with default mask and it filters out routes with smaller masks if one with a bigger mask already exist.
 
Here is the new SA routes file that I've downloaded.
 
Here is a nice link that explains how to do it on Linux
[http://www.tlug.org.za/wiki/index.php/Local/International_Router]
 
=== Building our GroenWifi router (v1) ===
The first step was to find a suitable hardware platform. I've looked at various options and in the end I decided to use a Soekris 4801. The 4801 is a embedded i386 compatible device with three Ethernet ports + I can run FreeBSD on it. See [http://www.soekris.com/net4801.htm] for a full description of the hardware platform
 
[[Image:Net4801.jpg]]
 
==== Initial network setup ====
My initial idea was to make use two ADSL routers, one with a SA-only ADSL account and the other with a normal SA+International ADSL account. The soekris box will then slit the traffic and forward it to the correct ADSL router. This is one of the big reasons I've selected a hardware platform with three Ethernet interfaces (Mesh, SA-only and International)
 
picture ?
 
===== Installing FreeBSD on the 4801 =====
Quick job, just took our standard mesh-router distribution and installed it on the 4801 (complements to John)
 
===== Configuring to box to split the traffic =====
Not so difficult. I've added all the SA local subnets into Quagga (routing daemon) and directed it to my SA local interface, while pointing my default gateway to the other interface (International). I did a quick test with two NAT boxes and it worked very well. With the aid of traceroute I could see that International and SA traffic would take different routes to the internet.
 
picture ?
 
===== Add Accounting for Local and International traffic =====
Now it becomes more interesting. The easiest method I know to get some stats is to use the counters of some IPFW rules. Then one can do a "ipfw show" to read the counters. The fist step is to split the rules based on interface name and the direction of the traffic. e.g.
 
00200 skipto 1000 ip from any to any in recv sis0
00300 skipto 2000 ip from any to any out xmit sis0
00400 skipto 3000 ip from any to any out xmit tun1
00500 skipto 4000 ip from any to any in recv tun1
00600 skipto 5000 ip from any to any out xmit tun0
00700 skipto 6000 ip from any to any in recv tun0
 
That way one can count the traffic going to, and coming from our International interface, as well as the SA interface,  each subnet that we use in the mesh. This way I get four counters/household on the mesh (Internat-in, Internat-out, SA-in, SA-out). E.g. counters for SA-in and SA-out
 
03001    51      6069 allow ip from 10.4.3.0/24 to any
03002  3549    174403 allow ip from 10.4.4.0/24 to any
03003    0        0 allow ip from 10.4.5.0/24 to any
03004  347    41661 allow ip from 10.4.6.0/24 to any
03005  169      7680 allow ip from 10.4.8.0/24 to any
03006    0        0 allow ip from 10.4.9.0/24 to any
03007    0        0 allow ip from 10.4.10.0/24 to any
04001    0        0 allow ip from any to 10.4.3.0/24
04002  5181  4248108 allow ip from any to 10.4.4.0/24
04003    0        0 allow ip from any to 10.4.5.0/24
04004  471    504541 allow ip from any to 10.4.6.0/24
04005  178    53005 allow ip from any to 10.4.8.0/24
04006    0        0 allow ip from any to 10.4.9.0/24
04007    0        0 allow ip from any to 10.4.10.0/24
 
Now one just need a script to add the in and out traffic per subnet/household
 
===== Add Bandwidth management =====
Another great feature of FreeBSD is it's build in support for Dummynet. Dummynet is a great traffic shaper, bandwidth manager. We decided that we have basically three types of users in the network.
 
Free user - They get free access to the network, but their bandwidth is limited.
Paying user - They pay a monthly fee and they share the available bandwidth (using fair queuing).
Gamers - A paying user, but with some reserved bandwidth for gaming
 
Dummynet configuration is integrated into ipfw. You first have to define a pipe. Then you configure that parameters for the pipe e.g. 50kbit/s etc. Then you attach a pipe to a subnet. Remember, you should have one pipe for outgoing and another for incoming traffic. E.g. ipfw rules
 
#Bandwidth allocated for Wireless Mesh
pipe 1 config bw 448Kbit/s
pipe 2 config bw 3584Kbit/s
pipe 3 config bw 25Kbit/s
pipe 4 config bw 25Kbit/s
 
#paying people get a queue weigth of 100
queue 1 config pipe 1 weight 100
queue 2 config pipe 2 weight 100
 
#free people get a queue weight of 20
queue 3 config pipe 1 weight 20
queue 4 config pipe 2 weight 20
 
#adding queues to subnets
01001 queue 1 ip from 10.4.3.0/24 to any
01002 queue 1 ip from 10.4.4.0/24 to any
01003 queue 1 ip from 10.4.5.0/24 to any
01004 queue 1 ip from 10.4.6.0/24 to any
01005 queue 1 ip from 10.4.8.0/24 to any
01006 queue 1 ip from 10.4.9.0/24 to any
01007 queue 1 ip from 10.4.10.0/24 to any
02001 queue 2 ip from any to 10.4.3.0/24
02002 queue 2 ip from any to 10.4.4.0/24
02003 queue 2 ip from any to 10.4.5.0/24
02004 queue 2 ip from any to 10.4.6.0/24
02005 queue 2 ip from any to 10.4.8.0/24
02006 queue 2 ip from any to 10.4.9.0/24
02007 queue 2 ip from any to 10.4.10.0/24
 
For more info on Dummynet follow this link [http://info.iet.unipi.it/~luigi/ip_dummynet/] and this one for FreeBSD firewalls [http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/firewalls.html].
 
===== Testing everything =====
I've used the same setup as the one that I use to test the splitting of network traffic. See picture. With the ipfw show command I could see how the counters would increase when traffic was send over the different networks. To test the bandwidth shaping I've did a file transfer from different subnets behind the router to a ftp server on our network. Works great, OK, after fixing some bugs :-)
 
picture ?
 
Next step was to test it with the actual ADSL routers that we planned to use in the network. Got host with one of the routers and started to play with it at home. I don't have an ADSL line at home, but I can test the Ethernet side. Now for the fist hick-up. The router had limited features and it did not support routing or static routes. This meant that the ADSL box will not know how to send traffic back to the mesh. OK, back to the drawing board to look at all the options (Another ADSL router, double NAT, pppoe on FreeBSD etc).
 
=== Building our GroenWifi router (v2) ===
 
== Putting FreeBSD onto my EeePC ==
OK, finally get around to it. First read John's info on the wiki:
http://wiki.meraka.csir.co.za/wiki/index.php/Category:ASUS_eeePC
 
Then copy John's FreeBSD EeePC distribution (img.tgz) to my PC.
Follow Werner's instructions to get the distro on USB memory stick
 
fdisk -I da0
fdisk -B da0
bsdlabel -w da0s1 auto
bsdlabel -B da0s1
newfs /dev/da0s1a
mount /dev/da0s1a /mnt
tar -xvf img.tgz -C /mnt/
echo /dev/da0s1a / ufs rw 1 1 > /mnt/etc/fstab
echo ifconfig_DEFAULT=DHCP > /mnt/etc/rc.conf
echo hostname=demo >> /mnt/etc/rc.conf
umount /mnt
 
Plug into USB port on EeePC.
Select boot form USB (Press ESC at bootup to get menu)
Boot FreeBSD
Dump WinXP image on a USB disk
dd if=/dev/ad2 of=/dev/da2 bs=4k
 
Follow instructions above to get FreeBSD on ad2
 
fdisk -I ad2
fdisk -B ad2
bsdlabel -w ad2s1 auto
bsdlabel -B ad2s1
newfs -U /dev/ad2s1a
mount /dev/ad2s1a /mnt
tar -cf - --exclude /mnt / | tar -xvpf - -C /mnt
echo /dev/ad2s1a / ufs rw 1 1 > /mnt/etc/fstab
 
Enable ipv6, ssh and moused in /etc/rc.conf
ipv6_enable="YES"
sshd_enable="YES"
moused_enable="YES"
 
Shutdown and reboot.
 
Install Xorg and KDE
 
setenv PACKAGEROOT ftp://ftp2.za.freebsd.org
pkg_add -r xorg
pkg_add -r kde-lite
 
Enable and test sound. Add the following line in /boot/loader.conf
snd_hda_load="YES"
Search for app that can show the battery status in X. Install and test desktopbsd-tools. Battery status works. Test wifi install gui. Not impressed, it only supports WEP
 
pkg_add -r desktopbsd-tools
 
Battery status works. Test wifi install gui. Not impressed, it only supports WEP. Delete package and search for other battery apps. See that KDE has a package that can show battery status. Install klaptopdaemon.
 
pkg_add -r kdeutils-klaptopdaemon
 
Battle to get it to work. Eventually find out that you have to load AMP (advanced power management). Edit /etc/rc.conf and add the following lines.
amp_enable="YES"
ampd_enable="YES"
 
Shutdown and reboot
 
Login and start KDE. Enable battery monitor under kde "Start" "Settings" "Power control" "Laptop Battery". Yes, I get a new battery icon. Remove power and test it. Battery shows power, but reporting of "time left" is broken.
 
Enable power manager to preserve power while running on battery. Add following to rc.conf
 
powerd_enable="YES"
 
and lower CPU speed. Add the following to /etc/sysctl.conf
 
dev.cpu.0.cx_lowest=C3
 
Try to get dstumbler to work so that we can use the EeePC as a wireless measurement PC. Install BSD-airtools (dstumbler part of it), but it looks like the package is broken. Search for other packages that we can use, but I can't find anything. Start looking at what command line stats available.
 
Try to enable virtual screen resolution in Xorg. Add the following in the Screen section of xorg.conf
 
SubSection "Display"
                Viewport  0 0
                Depth    24
                Virtual 1200 720
EndSubSection
 
Restart X, but it doesn't work. Do a lot of reading and playing, but I cannot get it to work.
 
== Meraka wireless Network ==
We are making use of Linksys WRT54 wifi routers, re-flashed with dd-wrt software. We've also enabled WPA corporate mode, using a Radius server that verify users against the CSIR LDAP server.
 
====Upgrading the dd-wrt firmware on a linksys.====
Firmware upgrades are usually done via the WEB gui. Here is the instructions to do it on the command line. It's important to know how to do it, so that we can script it in future. Fist enable ssh via the WEB gui. You can find it under Administrator -> Services -> Secure Shell. Select SSHd en then add your ssh public key under the  Authorized Keys section. Save settings. Now you can login without a password as well run remotes commands via ssh.
 
scp ftp/linksys/dd-wrt/dd-wrt.v24_std_generic.bin root@192.168.1.1:/tmp
ssh root@192.168.1.1 "write /tmp/dd-wrt.v24_std_generic.bin linux"
ssh root@192.168.1.1 reboot
 
Load WEB gui and verify if upgrade worked. Yes - all new.

Latest revision as of 17:37, 17 June 2009


migrated to wireless dev