Revision as of 15:43, 6 August 2008

The current information here builds on content that has been transferred from the more technically-oriented [WISP Coova phpmyprepaid http://wirelessafrica.meraka.org.za/wiki/index.php/WISP_Coova_phpMyPrepaid] link to this page.

Overview on ISP requirements/specifications

This section attempts to formulate a basic set of specifications for an ISP (billing / services / etc), and assumptions taken regarding these specs during technical implementation described in the next section. Some of the information here is also based on that gathered by Henk Kotze.

High level definition of use cases

From ISP manager's point of view:

Web based user interface, that provides seamless access to all required tools and efficient enough to require very little use of commands/text-based configuration.
ISP manager gets all info and set up radius, etc....
Linksys should be preconfigured by ISP manager

From Client's point of view:

Contact ISP (physically go to person / telephone / ?)
Buy equipment (wireless/wired NIC) and prepaid time from ISP manager?
Assume that client is in range of hotspot (for now)
Client installs equipment (probably part of package from ISP)
switch on
connect PC / laptop to network
opens browser with own PC (assume client has PC...)
Enter username , password (web portal?)
One should be able to go to internet
When limit is reached, cut off

Billing Model

Prepaid
- Credit Card-related payment (which also includes paypal.com,etc.) is not suitable. With the prepaid billing system, users can pay directly and get a ticket with login details. Implementation at the moment is based on this model.
Contracts / Subscriptions
- ?

Usage plan
- Time-based usage
  - Users get disconnected after a set period of time for which they have paid.
  - Different plans available (e.g, 10 min / 15 min/ 30 min / 1 hr / 2+ hrs / etc)
- Bandwidth-based usage
  - (total of upload and download)
  - Different plans available (e.g, 5MB / 10 MB / 20MB / 50MB / 100MB / 1GB/ 2GB /etc)

User interface / Captive Portal

When a user tries to access an external website (a site that requires paid access), he/she is re-directed to login page where the user should enter details provided on the ticket. The user should also be able to check how much bandwidth / time remaining at any time before/after/while surfing. It would be nice for the user to be able access this information through the captive portal.
Additional thoughts: In the case of ISPs located within a community network, it would be nice for the captive portal to somehow bond with the local community web portal (Perhaps important updates/announcements from community members/leaders on the login portal, etc ?)
More thoughts on this: For user to have a object in the browser that will show his/her usage details, possibilities while browsing:
- Frames: E.g, if the user is not logged on and types URL.com in his browser, once he/she is re-directed to the login portal, logs in and is authenticated, the user is then re-directed to a page containing a small upper frame that shows the users usage/bill details, and the main frame of the page will show URL.com . Pros: fairly easy to implement. Cons: If the user closes this window or types another url in the browser address input, then this ¨closes¨ the frames page.
- Browser extension/ActiveX/object: E.g, if the user uses the firefox browser for access, a firefox extension can be designed to show usage. Pros: available any time to the user when browsing in firefox. Cons: Can be complex/take more time to implement; More importantly, a firefox extension will only work on firefox, hence if another browser is used instead (e.g. Internet Explorer/Safari/Opera), this becomes pointless.

ISP interface

ISP should easily be able to manage users (add/edit/reset password/ban/etc)
ISP should also be able to monitor usage of users (individual / general usage trend/tracking of all users)
ISP should also be able manage / diagnose network problems (push configs to nodes, track problematic nodes, etc)
- Some information available on Tom's research page regarding this.

Technical information on work done

This section contains documentation on work done regarding testing of an implementation that would make it as easy as possible for a would-be entrepreneur in a wireless mesh network to deploy an ISP service. The entrepreneur should be able to manage, monitor and charge users with ease.

List of tasks involved

The breakdown below is more short-term oriented, to make it easier to follow.

Testing of coova, phpmyprepaid, webmin.
These tools were used to setup a ISP testbed. (see below)
Ironing out phpMyPrepaid bugs/issues to make it suitable for functioning in such a setup.
TODO: Improve ISP client side functionality (client logon service, Browser extension?, etc.)
Skinning of phpMyPrepaid to make it user-friendly and easy-on-the-eyes.
TODO: Tweak dashboard (mock-up designed by Tom) in terms of looks and functionality.
TODO: Test coova-chilli FreeBSD port for functionality on the "Bokkie" Router
TODO: Identify and "extract" aspects of Zeroshell that are useful for WISP-in-a-box (QoS, etc).

I have added a more detailed task breakdown for the long-term, based on that from Tom´s page.

Detailed, long-term Task Breakdown

Dashboard & first client development: Meraka
1. Further examination of ROBIN to see which parts of it can be modified for our purposes
2. Adaptation of Webmin's cluster tools for use on the mesh
3. Determine appropriate tools for QoS [starting point: Zeroshell]
4. Update API design & specification
5. Dashboard framework design and implementation
6. Mesh update client framework design and implementation
7. Dashboard module implementation
  1. Network Monitoring
  2. Network Configuration – basics (Channel, SSID, etc)
  3. Network Configuration – QoS settings
  4. Network Configuration – AP-only settings
  5. Adapt phpMyPrepaid's CoovaChilli configuration to send out updates via standardized API rather than only changing local configuration file.
8. Co-development with dashboard modules: mesh client update modules for each dashboard component, designed initially for the WRT but as portable as possible (for most modules portability should be trivial, as the software on the node the update client is configuring will be the same).
9. CSS skinning for all dashboard components (including pre-existing ones such as Webmin and phpMyPrepaid).

Setup Ingredients

The tools used to setup this testbed:

Coova WISP-in-a-box Wireless Africa wiki, URL
phpMyPrepaid Wireless Africa wiki, URL
FreeRADIUS
MySQL
Ubuntu Hardy Server - This is the distro that was used for the gateway server. In future, it is intended to have this setup (when finalized) integrated into Inveneo Hub Server Linux.

Setup Recipe

Diagram of initial WISPiab-billing setup using Coova, myPhpPrepaid.

At the moment the intention of this setup is to mainly test billing support, and is more of a wifi-hotspot setup. It will later on be modified to adapt to a wireless mesh network environment. The gateway server has an external network interface (e.g, eth0) that is connected to the Internet, and an internal network interface (e.g, eth3), connected to the Coova Linksys Router (internal network). In this case the external network interface gets its IP address automatically (DHCP). The internal network interface is assigned a static ip address (e.g, 192.168.5.1), therefore the Coova WAN interface is also set to static (e.g, 192.168.5.4). All instructions below are provided with the assumption that this is a fresh install of Ubuntu. Performing these steps on an existing installation does not guarantee proper functionality.

Though I (George) am the main contributor to this section at the moment, this is still work in progress, and is new work for me, so all information stated here may be not be of the highest accuracy. However, I will try my best to keep the information here as accurate as possible.

Coova

Setup coova on Linksys router - Howto guide on Coova website
Configure Coova WAN settings. (the following points are with assumption that the gateway server settings described here are being used)
- On Coova administration web portal : Network->WAN.
- Under WAN Configuration, set Connection Type to Static IP.
- Under IP Settings, set IP address, Netmask and Default Gateway settings (e.g, 192.168.5.4, 255.255.255.0, 192.168.5.1 in this example).
- Under DNS Servers, add Gateway server internal network interface IP address (e.g 192.168.5.1).
- Leave the Dyanmic DNS Settings as is. Save changes.
Configure Coova Wireless Network. Click on Network->Wireless.
- Under Wireless Configuration, ensure Wireless Network is set to Enabled and Mode is set to Access Point. Configure Wireless Network to however it suits you. Save changes.
- For the purposes of testing in this case, Wireless Network is not encrypted, with minimal access configuration (ESSID - wifibox, ESSID Broadcast - Show, Channel - Auto, Encryption Type - Disabled).
Setup coova to act as a Chillispot-type hotspot, with auto-configuration disabled. (Setup may be tested later on with WifiDog instead of Chillispot).
- Click on the Hotspot tab. Under HotSpot Configurations
  - Set Hotspot type to ChilliSpot UAM
  - Set Hotspot Mode to Wireless Only
  - Choose if you prefer to Deny or Allow LAN access through the hotspot.
- Under ChilliSpot Configurations
  - Set Auto Configuration to Disabled
  - Fill in UAM Hostname, UAM Secret, NAS Identifer information. Save changes. In this case:

UAM Hostname   : 192.168.5.1
UAM Secret     : yoursecret
NAS Identifier : wifibox

- Click on Access Lists under the HotSpot tab.
  - Walled-Garden Hosts...
  - Walled-Garden Domains... Save Changes.
- Leave Hotspot->DHCP settings as is.
- Click on RADIUS under the HotSpot tab. Fill in the details under RADIUS Configurations. In this case:

Primary RADIUS Server   :  192.168.5.1
Secondary RADIUS Server :  192.168.5.1
RADIUS Auth Port        :  1812
RADIUS Acct Port        :  1813
Shared secret           :  yoursecret

- - For testing in this case, we leave MAC Address Authentication as Disabled.
- Under the Optional RADIUS Configurations :
  - Set RADIUS Admin Username to phpmyprepaid.
  - Set RADIUS Admin Password to a password of your choice.
  - Leave the rest of the settings in this section as is. Save Changes.
  - Note that the RADIUS Admin Username and Password and Shared secret settings are to be inserted into FreeRADIUS configuration on Gateway server.
- Click on Advanced under the HotSpot tab. Fill in the details under Advanced ChilliSpot Configurations. In this scenario:

Internal UAM Port             : 3990
HotSpot Services Provider     : yourhotspot
HotSpot Services Provider URL : http://192.168.5.1/cgi-bin/hotspotlogin.cgi
UAM URL Format                : http://192.168.5.1/cgi-bin/hotspotlogin.cgi (I'm not too sure but this value should not really matter if the full URL has been filled in for the hotspot provider url)
UAM HomePage (splash page)    : http://10.1.0.1:3990/www/coova.html
Local Content Directory       : /etc/chilli/www

Gateway server

Install Ubuntu Server (Hardy), with minimum MySQL, FreeRADIUS, Apache webserver. DO NOT Install the DNS Server. This is because I have decided to use dnsmasq instead, as I found it much easier to setup for any user who is not familiar with bind9. Bind9 is the default DNS server that is included with Ubuntu distributions.
Install Webmin. Webmin is a great tool with a web-based frontend to administer many important settings under Linux (Startup/Shutdown Scripts, DNS, DHCP, Firewall, NAT; also supports plugins for other tools). (an apparently better alternative is ispconfig, but I've not tested this tool yet.)
Setup up network configuration for both network interfaces. [Todo: Should check out the dnsmasq webmin module..]
- Configure NAT using iptables (can be done using the Webmin interface).
  - Click on Networking -> Linux Firewall on the left panel of the Webmin interface.
  - On the Linux Firewall page, at the top left, there is a list box next to the Showing IPtable: button. Click on this list box and ensure Network address translation (nat) is selected.
  - Click on Showing IPtable: The page will reload with NAT iptable configuration.
  - Under the section Packets after routing (POSTROUTING), click on the Add Rule button. The page reloads with the Add Rule page.
  - In the Chain and action details section, you may fill the Rule comment section with a description of your choice (e.g., Internet access for intranet). Choose Masquerade option in the Action to take field. Leave other fields as is.
  - In the Condition details, select outgoing interface to the external interface (eth0 for the purpose of this document.) Leave other details as is.
  - Click 'the 'Create button.
  - This page will reload the general Linux Firewall page. You should see a rule under the postrouting section to the following effect:
```
Action: Masquerade;                        Condition:If output interface is eth0
```
  - Click on Apply Configuration.
  - Next to the Activate at boot button, select the Yes option. Then click 'Activate at boot.
- Enable IP forwarding. (In this case we are using IPv4. For IPv6, replace "ipv4" in all the settings below with "ipv6".
  - Open terminal and type the following command in terminal:
```
echo 1 > /proc/sys/net/ipv4/ip_forward 
```
  - Open the file /etc/sysctl.conf with superuser privileges.
```
sudo nano /etc/sysctl.conf
```
    If nano is not installed, you can type
```
sudo apt-get install nano
```
  - Look for the the term net.ipv4.ip_forward in this file. If it is commented uncomment it. Edit this line as necessary to ensure that it looks like
```
 net.ipv4.ip_forward=1
```
- Install and setup dnsmasq and ipmasq by typing from terminal
  - ```
  sudo apt-get install dnsmasq ipmasq
```
- ```
sudo /etc/init.d/dnsmasq restart
```
  - ```
  dpkg-reconfigure ipmasq
```
  This will load a window that requests some configuration steps. Select Yes to the first screen (recompute firewall). Select OK to the second screen. Select After network services have been started on the next screen (when should ipmasq be started) and press OK. Reboot.
- Setup FreeRADIUS and MySQL. Note: I have had a lot of hassle trying to smoothly set up FreeRADIUS. A lot of issues can be expected during this part of the setup.
  - Ensure that the FreeRADIUS MySQL plugin is installed:
```
apt-get install freeradius-mysql
```

More to follow within the next few days.

Problems / Errors

Coova

Spoofed source packets

Coova (1.0-beta7): chilli started in debug mode (chilli -fd)
When client attempting to access any URL, coova debug message -

chilli.c: 2566: 0 (Debug) Client MAC=XX-XX-XX-XX-XX-XX assigned IP a.b.c.d
chilli.c: 2747: 0 (Debug) Received packet with spoofed source!
chilli.c: 2747: 0 (Debug) Received packet with spoofed source!
..

Reason: Sometimes if the chilli daemon stops, the connected client A, which up until now had the IP address a.b.c.d, may renew its IP address and receive a new address w.x.y.z from LAN DHCP service (instead of coova-chilli's DHCP service). If the chilli daemon is restarted again, it might not renew client A's IP address to a.b.c.d (or an address in the range of a.b.c.X). The above debug indicates that the chilli daemon thinks it has assigned the relevant IP address, but is confused when it receives packets with a different IP source.
Solution: Stop chilli daemon, release client's IP address, restart chilli daemon, re-connect client and it should renew its IP address correctly.

Leaky bucket

chilli.c: 2939: 0 (Debug) Successful UAM login from username=user123 IP=x.x.x.x
chilli.c: 2942: 0 (Debug) Received login from UAM

chilli.c: 110: 0 (Debug) Leaky bucket timediff: 0, bucketup: 0, bucketdown: 0, up: 656, down: 0
chilli.c: 110: 0 (Debug) Leaky bucket timediff: 0, bucketup: 656, bucketdown: 0, up: 0, down: 1500
chilli.c: 110: 0 (Debug) Leaky bucket timediff: 0, bucketup: 656, bucketdown: 1500, up: 0, down: 1500
chilli.c: 110: 0 (Debug) Leaky bucket timediff: 0, bucketup: 656, bucketdown: 3000, up: 0, down: 1171
chilli.c: 110: 0 (Debug) Leaky bucket timediff: 0, bucketup: 656, bucketdown: 4171, up: 66, down: 0
chilli.c: 110: 0 (Debug) Leaky bucket timediff: 0, bucketup: 722, bucketdown: 4171, up: 66, down: 0
chilli.c: 110: 0 (Debug) Leaky bucket timediff: 0, bucketup: 788, bucketdown: 4171, up: 66, down: 0
chilli.c: 110: 0 (Debug) Leaky bucket timediff: 0, bucketup: 854, bucketdown: 4171, up: 698, down: 0
chilli.c: 110: 0 (Debug) Leaky bucket timediff: 0, bucketup: 1552, bucketdown: 4171, up: 0, down: 52
chilli.c: 110: 0 (Debug) Leaky bucket timediff: 0, bucketup: 1552, bucketdown: 4223, up: 0, down: 1500
chilli.c: 110: 0 (Debug) Leaky bucket timediff: 0, bucketup: 1552, bucketdown: 5723, up: 0, down: 1500
chilli.c: 110: 0 (Debug) Leaky bucket timediff: 0, bucketup: 1552, bucketdown: 7223, up: 0, down: 1500
chilli.c: 110: 0 (Debug) Leaky bucket timediff: 0, bucketup: 1552, bucketdown: 8723, up: 66, down: 0
...

Errors received when coova-chilli running in debug mode
Web browsing seems fine, but can this affect binary file downloads? ..Downloaded a 5MB quicktime video, which did not seem affected..

phpMyPrepaid

Monitoring bandwidth usage

phpMyPrepaid does not seem to be accurately measuring bandwidth usage. After downloading a 5MB file, phpMyPrepaid reports usage of 0.92 Mo (phpMyPrepaid addresses in octets)...
Reason: phpMyPrepaid configuration has the RADIUS Download/Upload database values switched.
Solution: Fix this value in config.inc.php in the include directory of the phpMyPrepaid www folder.

Unknown attribute "Max-All-Session"

When logging in from client using time-based account created using phpMyPrepaid, freeradius server gives the following error and rejects this user:

...
radius_xlat:  'SELECT id, UserName, Attribute, Value, op           FROM radcheck           WHERE Username = 'odiznn'           ORDER BY id'
rlm_sql (sql): Reserving sql socket id: 0
rlm_sql_mysql: query:  SELECT id, UserName, Attribute, Value, op           FROM radcheck           WHERE Username = 'odiznn'           ORDER BY id
rlm_sql: Failed to create the pair: Unknown attribute "Max-All-Session"
rlm_sql (sql): Error getting data from database
rlm_sql (sql): SQL query error; rejecting user
...

Only seems to happen with this type of account..
Reason: Attribute Max-All-Session needs to defined an sql module in radiusd.conf (the freeradius configuration file).
Solution: See http://wiki.freeradius.org/Rlm_sqlcounter. Add the sql module as described on this page.

Installation Problems: PHP not executing

phpMyPrepaid uses shorthand PHP tags: <? instead of <?php. Shorthand open tags need to be enabled in the php.ini file (/etc/php5/apache2/php.ini on Ubuntu).

Webmin

Webmin HTML frame issues

Does not operate within HTML frames. Re-directs to its own url as soon as it detects that it is within a frame.
Solution: Comment out the following: (/* at begin and */ at end of the code below) in /usr/share/webmin/session_login.cgi and /usr/share/webmin/pam_login.cgi ).

if (window != window.top) {
	window.top.location = window.location;
	}
</script>

Review on tools/ingredients tested

It is already a fixed decision to use FreeRADIUS (the best known open-source RADIUS tool) for back-end AAA (authorization, authentication, accounting) purposes. Hence tools required to deal with AAA aspects, (e.g, Captive Portal, Billing System, etc.) must be capable of integration with FreeRADIUS.

Captive Portal

Coova-chilli
- Based on the now defunct Coova.
- Active development.
- Best known-tool for comprehensive RADIUS support at the moment.

Billing system / interface

daloRadius
- Very nice css-interface, and neat layout.
- Easy to setup.
- User is expected to have an understanding of basic FreeRADIUS-related attributes, in order to create user accounts, etc.
- Since the users of this are expected to be non-technical, phpMyPrepaid is considered to be a better option in this regard.

phpMyPrepaid (final choice)
- Not an aesthetic interface. Needs skinning.
- A bit more complicated to setup, compared to daloRadius.
- More inituitive to the non-technical user, compared to daloRadius.
- Though setup is complicated, this can be rectified having phpMyPrepaid pre-installed. Skinning can also be resolved.