WirelessAfrica - New pages [en]

Gnu Radio

2010-04-20T04:20:23Z

Kingdavid: /* Installing Gnuradio */

== Purpose ==

To trial a white spaces network in a rural area and learn what is possible

== Team Members ==

* David Johnson
* Veljko Pejovic
* Elizabeth Belding

==Idea description==

==Main challenges==

==Milestones==

==Primary obstacles==

==Evaluation==

==Technical notes==

===Installing Gnuradio===
* The main installation page (http://gnuradio.org/redmine/wiki/gnuradio/)
* Building from source (http://gnuradio.org/redmine/wiki/gnuradio/BuildGuide)
* Ubuntu specific installation instructions (http://gnuradio.org/redmine/wiki/gnuradio/UbuntuInstall)

My notes for customization:
./configure --prefix=/home/djohnson/gnuradio
make
make check
make install
This will install all the libraries and programs in /home/djohnson/gnuradio

Add gnuradio python libraries to PYTHONPATH ... add to your ~/.bashrc or /etc/bash.basrc file
export PYTHONPATH = /home/djohnson/gnuradio/lib/python2.6/site-packages

== Useful links ==

Macha Monitoring

2010-02-13T00:31:22Z

Kingdavid: /* This to be put in a new section - academic writing guide */

== Purpose ==

Rural wireless networks in developing regions often depend on slow satellite links for their Internet access. This paper will try to quanitify the traffic patterns and usage of a slow satellite internet link in a rural wireless network and compare this to recent studies of Internet uisage patterns in the developed world to understand key challenges. This will be used as a starting point for further work to try and optimize Internet usage over slow satellite links in developing regions.

== Team Members ==

* David Johnson
* Elizabeth Belding
* Kevin Almeroth
* Gertjan van Stam

==Idea description==

==Main challenges==

==Milestones==

==Changes made on machines in Macha to enable monitoring==

==Tools installed on my machine for analysis==

===Python tools fro analysing tcpdump files or live interface===
* pypcap
sudo apt-get install python-pypcap
dpkg --listfiles python-pypcap ... to see files that it installes
svn checkout http://dpkt.googlecode.com/svn/trunk/ dpkt-read-only
cd dpkt-read-only
sudo make install
cd /usr/share/doc/python-pypcap/examples
python test.py

* Examples using pypcap
* Pyscan logger (http://code.activestate.com/recipes/576690/)
* Packet monitoring with dpkt (http://code.activestate.com/recipes/576678/)

==Primary obstacles==

==Evaluation==

=== Metrics that will be measured ===

* Percentage of local traffic
* Caching efficiency (ratio of incoming traffic to external destination to outgoing traffic to external destination)
* DNS hit rate
* DNS delays (can be quite significant) , % of DNS hit on local DNS
* Up/Down Traffic usage over a 2 week loggin interval
* Flow analysis
- TCP connection durations
- Number of simultaneous flows in the network over time
- Plot of bandwidth used per IP source address in the network - check if there are clear dominant users
- # Retransmissions
- TCP round trip times for ACKS
* Breakdown of application classes using port numbers
- Peer-to-peer traffic
- Web
- video streaming ... protocols like RTSP have there own port - also IP based for flash sites like youtube
- VoIP ... prototocols like SIP and some known Skype ports - Skype is a challenge if using port 80
- Instant messaging ... could catch things like IRC but maybe IP addresses based again
- tunneling

== Useful links ==

=== Monitoring tools ===
* tshark

=== Scraping tcpdump files ===
* List of useful tools (http://www.comlab.uni-rostock.de/research/tools.html)
* TCPdump tips - filters can also be used with pcapy (http://www.cs.ucr.edu/~marios/ethereal-tcpdump.pdf)
* Crawdad has tools but mostly looks like wifi specific (http://crawdad.cs.dartmouth.edu/tools.php)
* ipsumdump
* tcpflow
* tcpdstat -d <tracefile>
To analyse packet size distribution:L tcpdstat -w out.log <tracefile>
* Port number assignments (http://www.iana.org/assignments/port-numbers)
* To analyse flow rates trpr (http://pf.itd.nrl.navy.mil/protools/trpr.html)
* Coralreef port analysis (http://learn.caida.org/cds/traffic0202/CoralReef/index.html)
* CAida list of tools (http://www.caida.org/tools/)

=== Understanding port numbers and other networky stuff ===
* Database of known official and unofficial port numbers being used (http://ports.tantalo.net/?q=gnutella)
* TTLs set by various operating systems (http://members.cox.net/~ndav1/self_published/TTL_values.html)
* Meaning of http codes (http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html)
* Mime type descriptions (http://www.feedforall.com/mime-types.htm)
* Why youtube can't be cached (http://tumbleweed.org.za/2009/02/18/fun-squid-and-cdns)

=== DNS analysis ===
* Macha uses dnsmasq (uses a default of only 150 names in the cache)
dnsmasq -c --cache-size=<cachesize>
- if cachesize=0 it disables it

=== Checking reboot commands issued ===
* Check with
last reboot

=== Squid proxy analysis ===
* Description of meaning of all log files (http://www.tenon.com/support/webten/papers/squidlog.shtml)
* Hit rate for proxy server (http://www.squid-cache.org/Scripts/)
* Squid analysis (http://squid-graph.securlogic.com/)
* Meaning inside squid log files (http://www.linofee.org/~jel/proxy/Squid/accesslog.shtml)

=== This to be put in a new section - academic writing guide ===
* Writing style guide (http://elearning.homestead.com/ACADEMIC_WRITING_GUIDE.htm)
* When to italicise (http://www.ehow.com/how_2049063_italicize-properly.html)

Solution for tutorial 2 - receiver

2010-01-12T21:03:45Z

Kingdavid:

define ($DEV ath0)

FromDevice($DEV)
->c :: Classifier(12/0800)
// Strip off all the ethernet, IP and UDP headers
->Strip(42)
->Print(CONTENTS ASCII)
->Discard

Solution for tutorial 2 - sender

2010-01-12T21:02:45Z

Kingdavid: New page: Change the $SRC and $DST to suite the IP addresses of your nodes define($DEV ath0, $SRC 172.30.1.74, $DST 172.30.1.75) InfiniteSource("Goodbye World", 1 ) ->UDPIPEncap($SRC, 4001, ...

Change the $SRC and $DST to suite the IP addresses of your nodes

define($DEV ath0, $SRC 172.30.1.74, $DST 172.30.1.75)

InfiniteSource("Goodbye World", 1 )
->UDPIPEncap($SRC, 4001, $DST, 4002)
->IPPrint("my packet")
->arpq :: ARPQuerier($DEV)
->q :: Queue
->ToDevice($DEV)

FromDevice($DEV)
-> c :: Classifier(12/0806 20/0002)
-> [1] arpq

arpq[1] -> q

Solution for Original Tutorial 1

2010-01-12T21:01:40Z

Kingdavid: New page: define($DEV ath0) FromDevice($DEV) -> c :: Classifier(12/0800, 12/0806 20/0002) -> CheckIPHeader(14) -> IPMirror -> StripToNetworkHeader -> GetIPAddress(16) // Paint the dest...

define($DEV ath0)

FromDevice($DEV)
-> c :: Classifier(12/0800, 12/0806 20/0002)
-> CheckIPHeader(14)
-> IPMirror
-> StripToNetworkHeader
-> GetIPAddress(16) // Paint the destination IP address
-> arpq :: ARPQuerier($DEV)
-> IPPrint
-> q :: Queue
-> ToDevice($DEV)

arpq[1] -> q;
c[1] -> [1] arpq;

VO Requirements & Functionality Questions

2009-10-15T07:13:59Z

Kroux:

== Understanding the bigger picture ==

The development on this project is done in phases. New functionalities/features are added with each phase until we reach our target system. All development work of every phase should be aligned with our end system in mind. In order for developers to make the right design choices that will not limit future functionality, they need to have a clear end target that they are working towards.

This page is here to help the developers to get a better understanding of the bigger picture and to make sure the target we design for is clearly defined and does not move around.

Please add any project related questions/thoughts on this page so that we can sort them and see what information is still missing.

=== VO (village operator) in action ===

Lets assume the the new VO has all the kit installed. The gateway (WISP) box is connected to the Internet and he is now ready to commission his new business. His first customers are three schools, the local Municipality and two owners of a local Spaza shop who wants a network connection at the shop and at both their homes. The two shop owners are very excited and they want to put a web server at their shop and they are thinking about opening a little Internet cafe very soon.

=== Definitions ===
* User - An accounting entity with a user name and password. Typically represents an individual or an organisation.
* Edge device - A device that is part of the local wireless network (e.g. talk mesh protocol) and allows other devices to connect through the wireless network. HPNs and 'Basic Nodes' are edge devices.
* HPN - High Performance Node
* BN - Basic Node

=== Random questions / thoughts ===

* What Internet contract packages are available from the VO - prepaid, contract, local, gov sites only ???
* Is data traffic contained on the local wireless network free / flat fee / pay per usage
* must a user do some login before he can use the local network
* must the network be able to log all user traffic between local users
* can a user use the local network without buying prepaid/account from the VO
* How must a prepaid package be activated / deactivated
* Are there any bandwidth limits tied to a Internet package
* Are there any data caps associated with the different package
* Must a prepaid user login before he can use the Internet
* Must a prepaid user login before he can send data on the local network (no Internet)
* How about a gaming server on the local network, does it need to login some where
* Can more than one user share the same prepaid account, and at the same time
* If a school uses prepaid, do they share the same account, or each user in school
* How will someone running an Internet cafe resell his prepaid bandwidth
* Is a prepaid account tied to certain time duration
* How does a person get onto the network?
* If login is necessary, how do peripherals like printers and ATAs authenticate?
* What devices should be 'approved' for use on the network

=== The VO ===
A VO is a person or an entity whose primary objective is running (at a profit) a wireless network. To this end, the following describes the needs or requirements of the VO to meet his objective.

The VO needs:

1. A healthy and working network

2. Diagnostic tools for the network

3. To manage users or subscribers to the network

4. Service management of the network

5. A view of the usage of the network

6. To manage (activate/deactivate) edge devices on the network

1. A healthy and working network

-> Each network element MUST handle all routing and communications other network elements.

-> Each network element MUST require of the VO as minimum as possible configuration without compromising its usability or effectiveness.

2. Diagnostic tools for the network

At a minimum, the VO is provided with a web based diagnostic tools.

-> The diagnostic tools are enough for the VO to establish any problem that may be there.

-> The diagnostic tools can interface with other systems, e.g SMS, email.

3. To manage users or subscribers to the network

-> VO will be able create and modify user accounts

-> VO provides a number of service options for different users

-> VO can monitor usage of the internet connection

-> VO can monitor usage of the local network

-> VO can block access to the network for certain users

4. Service management of the network

-> The VO is able to prioritise certain services within the network

-> The VO is able to block or enable certain services

5. A view of the usage of the network

-> The VO is able to see the usage of the whole network

-> The VO is able to see the usage of each network element

6. To manage (activate/deactivate) edge devices in the network

-> VO can allow (new) edge devices to join the network

-> VO can block edge devices from accessing the local network

Music Coach

2009-10-14T08:59:18Z

Kingdavid: /* Academic references */

== Requirements ==

Submit (via email) a tentative project proposal, including: team member names, brief description of project (a paragraph or so), the main technical challenges, milestones/tasks to accomplish this, the primary obstacles/concerns, how you propose to evaluate the outcome.

== Team Memebers ==
* David Johnson
* Dianna Han

==Idea description==

Use the Nokia N900 as a music coach which can check if you are playing a note at the correct pitch, check if a note is played at the correct time and optionally may allow you to display sheet music from midi files as well as control the pace of performance with movements of the body. It will act both as a real time feedback mechanism alerting you of pitch and rhythm inaccuracies as you play the notes and give you an overall evaluation at the end of your performance. The pitch evaluation is especially useful for instruments which don't have discrete notes such as a cello, violin, trombone or even human voice whereas the rhythm evaluation is suitable for all instruments. Other additional features such as an instrument tuner and a metronome could be added with trivial software modification. It should also be possible to evaluate only pitch with no pre-loaded music score by checking how close a note is to its nearest discrete note. This would assume an advanced student that is playing notes with less than a quarter tone of inaccuracy or else it would rounding to a note which is different to the original intended note.

Different feedback mechanisms will be explored. Visual feedback could use a compass like display where North and South represent a pitch that is above or below the correct note and the length of the arrow represents the fractional difference between the notes. East and West represents whether you played the note too early or too late and the length of a second arrow also represents the fractional difference. Audio feedback could also be explored by playing the correct note on the speaker in the case of a pre-loaded score and using beats in the note which get closer together the further the note is away from the correct note. This would be ideal as a training module for someone learning scales on a cello, for example.

One problem with a pre-loaded score is making allowances for musical expressiveness. This comes in the form of slowing down and speeding up your performance in sections of the music as well as adding vibrato to a note in the case of stringed instruments and the human voice. A vibrato note is an oscillation which falls slightly below and slightly above a note. The accelerometer could be used to sense rhythm by monitoring swaying movements on the body, if the phone was strapped onto a person. For example if the phone was strapped to the head, faster rocking movements in the head would speed up the performance and slower rocking movements would slow the performance down. Allowing for vibrato can be done by calculating the average pitch of the performed note over a specific time window.

==Main challenges==
# Real-time processing of the audio input to recognise the frequency of a note.
# Different instruments have a range of timbre with transient harmonics which can dominate the fundamental frequency momentarily - the challenge will be to decide on an optimal time window to use for the pitch recognition. This challenge may simply limit the scope of instruments to ones which have a dominant fundamental from the attack to the decay of the note.
# Real-time feedback display. You want the performer to get feedback as quickly as possible for it to be useful but if this is too soon - it may be inaccurate, whereas if its too late it will be more accurate but less useful.
# If time permits, inferring rhythm from a trace of accelerometer inputs should not require the user to make movements of the body that would be unnatural. A gentle rocking of the head should be all that's required to alter the pace at which the pre-loaded music progresses.

==Milestones==

# Test some FFT code on a PC which can take as input - sample rate, number of bins and display the frequency of a single note on the screen
# Run some experiments which compare time from note attack to display vs note accuracy.
# Make use of some existing software which can read in a midi file and display the notes and reformat this for a Nokia N900 screen size.
# Experiment with comparing a midi file to a live performance in terms of pitch accuracy - find optimal time windows for calculating average pitch of performed note.
# Evaluate the accuracy of the attack and release times of the live performance with the start and stop times of the notes in the midi file. Make use of a metronome on the phone for the user to synchronise their performance.
# Write a graphical interface on the Maemo simulator which creates feedback on the pitch and timing accuracy of the performance using a compass like display - (other more creative graphical displays may be developed). Also display momentary Pitch and timing accuracy as well as cumulative accuracy as a percentage.
# If both the compass display and the midi file are to be displayed, then a blend of these two could be explored
# Test the FFT code for pitch recognition on the N900
# Test the midi file input on the N900
# Test the graphical feedback display on the N900
# Time permitting add the option to change the speed of the performance using the accelerometer

==Primary obstacles==
* Processing capability to carry out a real-time FFT. If the on-board CPU is not sufficient we might need to use the DSP which adds an extra level of complexity to the problem
* Allowing for a certain level of musical expressiveness in a performance and not seeing this as pitch or rhythm inaccuracy.
* Inferring rhythm for small subtle movements of the phone when attached to the body depends on accuracy of accelerometer.

==Evaluation==
* The real-time pitch recognition software on the N900 can be compared against some more accurate non-real time pitch recognition on a standard PC.
* Musician's feedback on whether the device was useful in improving accuracy of performance.

==Using pulseaudio==
* possible command
pacat --record | sox -t raw -r 44100 -s -L -b 16 -c 2 - "output.wav"

==Code that can be used==
* A metronome with visual and audio feedback will be very useful for a player to know the pace that the application is expecting you to play the music - some code segments of the [http://www.antcom.de/gtick/ GTick] open source software can be used for this.

[[Image:Gtick.jpg]]

* To display notes on the screen a very well written qt application called [http://www.musescore.org/ Musescore] could provide some useful code segments

[[Image:Musescore.jpg]]

== Useful links ==

=== Pitch recognition ===

* http://www.nicholson.com/rhn/dsp.html
* http://www.hotpaw.com/intuna/singintuna.readme.html
* http://cs.ucsb.edu/~davidj/Files/thesis1995.pdf

=== FFT ===
* FFT on a symbian phone (http://wiki.forum.nokia.com/index.php/FFT_algorithm)
* Most stable FFT system - fftw (http://www.fftw.org/)

=== Multimedia system Maemo 5 ===
* Maemo 5 multimedia domain (http://wiki.maemo.org/Documentation/Maemo_5_Developer_Guide/Architecture/Multimedia_Domain#Audio_Subsystem)
* Maemo 5 uses pulseaudio (http://www.pulseaudio.org)
* Examples of using pulseaudio - parec (http://grangerx.wordpress.coExamplem/2009/08/03/fedora-11-recording-audio-from-pulseaudio-using-parec-and-sox/)

=== Accelerometer ===
* Using accelerometer trace to detect shaking (http://msdn.microsoft.com/en-us/magazine/ee413721.aspx)
* Smart phones as instruments (http://www.cnn.com/2009/TECH/10/15/iphone.music.zoozbeat/index.html)
* Zoozbeat (http://www.youtube.com/watch?v=YEsC-lklOA4)

=== MIDI importing ===
* C++ MIDI Library (http://www.jdkoftinoff.com/main/Free_Projects/C++_MIDI_Library/)
* The Computer Music Project Software (http://www.cs.cmu.edu/~music/music.software.html)
* Very Simple MIDI parsing code (http://cap-lore.com/EnglishSuites/code/code.html)

=== MIDI file format explanation ===
* David's MIDI Spec (http://www.srm.com/qtma/davidsmidispec.html)

=== Notation software ===
* GUIDOLib qt music notation library (http://guidolib.sourceforge.net/)

=== Companies that have done music reconigtion ===
* WIDI 1998-2009 Recognition System (79$ stadard, $159 professional) (http://www.widisoft.com/english/mp3-midi-products.html)
* Shazam iphone song reconigtion (http://www.shazam.com/music/web/pages/getshazam.html)

=== Academic references ===
* Google scholar results for "transcription of musical sound" (http://scholar.google.com/scholar?hl=en&q=transcription+of+musical+sound&btnG=Search&as_sdt=2000&as_ylo=&as_vis=0)
* Bibliography on automatic music transcription (http://www.recognisoft.com/cgi-bin/main.cgi?id=62&lan=en&n=&p=&d=&o=&r=y&s=)

===Instrument Ranges ===
* Of all orchestral instruments (http://www.orchestralibrary.com/reftables/rang.html)

WISP in a Box Development

2009-08-04T11:43:04Z

Ajay: /* Development */

{|align="right"
| style="height:100%;width:20em;border:1px solid #D9D9D9;background-color:#F2F2F2;" valign="top" |
*[[WISP_in_a_Box_Development|'''WISP in a Box Development Home''']]
|}

== Developer Guide ==

WISP in a Box development is organized as an Open-Source or [http://en.wikipedia.org/wiki/Free_software Libré] software project.

In order to understand what this means and be able to work together in this way we need to agree on the definitions of the following five words:

* '''Upstream''' - We use the term ''Upstream'' to refer to projects which our project makes use of. For example [http://net-snmp.sourceforge.net/ net-snmp] and [http://code.google.com/p/afrimesh afrimesh] are both ''upstream'' projects of the WispBox.

* '''Downstream''' - ''Downstream'' are projects who use our code. For example, if someone were to create a linux distribution which focused on adding VOIP services to a WISP and which used [http://code.google.com/p/wispinabox WispBox] as a dependency then that project would be considered to be ''downstream'' from our project.

* '''Users''' - We live and breathe for these special individuals as, without them, none of us would have paying jobs. Their contributions to the project usually take the form of running the software, reporting bugs on the mailing list and requesting features.

* '''Contributors''' - ''Contributors'' are usually also users but they have both the ability and desire to contribute code patches, documentation and their expertise to project development discussions. Often, future developers on a project are drawn from the pool of contributors. Contributors are a necessary condition to the long-term sustainability of any Libré project. For this reason all new developers are first required to spend some time as a contributor to the project before they are "handed the keys to the house" and gain full access to the project infrastructure.

* '''Developers''' - ''Developers'' perform the bulk of the development work and have full access to the project source repositories, servers and other infrastructure.

[[Image:libre_software_cycle.png|left|frame|The Libré Software Development Cycle]]

<br clear=all>

=== Upstream Projects ===

WISP in a Box has many upstream dependencies. The following list are the ones you will see referred to most commonly:

* [http://code.google.com/p/afrimesh '''Afrimesh'''] - the WispBox component integration toolkit and the core management dashboard.
* [http://uck.sourceforge.net/ '''Ubuntu Customization Kit'''] - A set of scripts used by our build system to generate the WispBox livecd images.

=== Setup Checklist ===

{| cellpadding="5" cellspacing="0" border="1"
! Task !! Description
|- valign="top"
| Development Workstation ||
* x86
* FreeBSD or Ubuntu
* 1x Edimax 7711UM USB WiFi network card
|- valign="top"
| WISP Server ||
* AMD ION
* 1GB RAM
* 80GB Hard Drive
* D-Link USB Ethernet
|- valign="top"
| Development Mesh Node ||
* D-Link DIR-300
|- valign="top"
| Deployment Mesh Node ||
* HPN
|- valign="top"
| Subscribe to mailing lists || TODO
|- valign="top"
| Join IRC || Install an IRC client and configure it for irc.freenode.net, #afrimesh (ask George if you need help)
|- valign="top"
| Configure WispBox Development Environment ||
Read: http://code.google.com/p/wispinabox/wiki/DevelopmentEnvironment
|- valign="top"
| Configure Afrimesh Development Environment ||
Read: http://code.google.com/p/afrimesh/wiki/DevelopmentEnvironment
|-
|}

=== Core Project Infrastructure ===

{| cellpadding="5" cellspacing="0" border="1"
! Resource !! URL !! Description
|-
| Mailing List - User Support || TO COME || Users, Contributors and Developers should subscribe to and follow this list.
|-
| Mailing List - Development || TO COME || Contributors and Developers should subscribe to and follow this list.
|-
| Issue tracker || http://code.google.com/p/wispinabox/issues/list || Bug reports and feature requests submitted to the Issue tracker are more likely to receive attention!
|-
| IRC || http://webchat.freenode.net/?channels=afrimesh || For IRC client configuration, use: irc.freenode.net, #afrimesh
|-
| Wiki - Main || http://wirelessafrica.meraka.org.za/wiki/index.php?title=WISP_in_a_Box_Development ||
|-
| Wiki - WispBox || http://code.google.com/p/wispinabox/w || Content hosted here still needs to be migrated to the main wiki
|-
| Wiki - Afrimesh || http://code.google.com/p/afrimesh/w || Afrimesh services multiple projects and as such maintains a separate wiki
|-
| Version Control - WispBox || http://code.google.com/p/wispinabox/source/checkout ||
|-
| Version Control - Afrimesh || http://code.google.com/p/afrimesh/source/checkout ||
|-
| Package Repository || https://launchpad.net/~wispinabox ||
|-
| Build Server || TO COME ||
|-
|}

=== Resources ===

* [[Media:Community_barriers_to_entry_checklist.pdf|'''Community Building : Barriers to Entry''']] - Neary Consulting [http://www.neary-consulting.com]

== Development ==

=== [[VO Requirements & Functionality Questions]] ===