Difference between revisions of "Macha Monitoring"

From WirelessAfrica
Jump to: navigation, search
(This to be put in a new section - academic writing guide)
 
(15 intermediate revisions by the same user not shown)
Line 36: Line 36:
 
  python test.py
 
  python test.py
  
 +
* Examples using pypcap
 +
* Pyscan logger (http://code.activestate.com/recipes/576690/)
 +
* Packet monitoring with dpkt (http://code.activestate.com/recipes/576678/)
  
 
   
 
   
Line 68: Line 71:
  
 
== Useful links ==
 
== Useful links ==
 +
 +
=== Monitoring tools ===
 +
* tshark
  
 
=== Scraping tcpdump files ===
 
=== Scraping tcpdump files ===
 
 
* List of useful tools (http://www.comlab.uni-rostock.de/research/tools.html)
 
* List of useful tools (http://www.comlab.uni-rostock.de/research/tools.html)
 
* TCPdump tips - filters can also be used with pcapy (http://www.cs.ucr.edu/~marios/ethereal-tcpdump.pdf)
 
* TCPdump tips - filters can also be used with pcapy (http://www.cs.ucr.edu/~marios/ethereal-tcpdump.pdf)
 
* Crawdad has tools but mostly looks like wifi specific (http://crawdad.cs.dartmouth.edu/tools.php)
 
* Crawdad has tools but mostly looks like wifi specific (http://crawdad.cs.dartmouth.edu/tools.php)
 +
* ipsumdump
 +
* tcpflow
 +
* tcpdstat -d <tracefile>
 +
To analyse packet size distribution:L tcpdstat -w out.log  <tracefile>
 +
* Port number assignments (http://www.iana.org/assignments/port-numbers)
 +
* To analyse flow rates trpr (http://pf.itd.nrl.navy.mil/protools/trpr.html)
 +
* Coralreef port analysis (http://learn.caida.org/cds/traffic0202/CoralReef/index.html)
 +
* CAida list of tools (http://www.caida.org/tools/)
 +
 +
=== Understanding port numbers and other networky stuff ===
 +
* Database of known official and unofficial port numbers being used (http://ports.tantalo.net/?q=gnutella)
 +
* TTLs set by various operating systems (http://members.cox.net/~ndav1/self_published/TTL_values.html)
 +
* Meaning of http codes (http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html)
 +
* Mime type descriptions (http://www.feedforall.com/mime-types.htm)
 +
* Why youtube can't be cached (http://tumbleweed.org.za/2009/02/18/fun-squid-and-cdns)
 +
 +
 +
=== DNS analysis ===
 +
* Macha uses dnsmasq (uses a default of only 150 names in the cache)
 +
dnsmasq -c --cache-size=<cachesize>
 +
- if cachesize=0 it disables it
 +
 +
=== Checking reboot commands issued ===
 +
* Check with
 +
last reboot
 +
 +
 +
 +
=== Squid proxy analysis ===
 +
* Description of meaning of all log files (http://www.tenon.com/support/webten/papers/squidlog.shtml)
 +
* Hit rate for proxy server (http://www.squid-cache.org/Scripts/)
 +
* Squid analysis (http://squid-graph.securlogic.com/)
 +
* Meaning inside squid log files (http://www.linofee.org/~jel/proxy/Squid/accesslog.shtml)
 +
 +
=== This to be put in a new section - academic writing guide ===
 +
* Writing style guide (http://elearning.homestead.com/ACADEMIC_WRITING_GUIDE.htm)
 +
* When to italicise (http://www.ehow.com/how_2049063_italicize-properly.html)

Latest revision as of 02:20, 10 April 2010

Purpose

Rural wireless networks in developing regions often depend on slow satellite links for their Internet access. This paper will try to quanitify the traffic patterns and usage of a slow satellite internet link in a rural wireless network and compare this to recent studies of Internet uisage patterns in the developed world to understand key challenges. This will be used as a starting point for further work to try and optimize Internet usage over slow satellite links in developing regions.

Team Members

  • David Johnson
  • Elizabeth Belding
  • Kevin Almeroth
  • Gertjan van Stam


Idea description

Main challenges

Milestones

Changes made on machines in Macha to enable monitoring

Tools installed on my machine for analysis

Python tools fro analysing tcpdump files or live interface

  • pypcap
sudo apt-get install python-pypcap
dpkg --listfiles python-pypcap ... to see files that it installes
svn checkout http://dpkt.googlecode.com/svn/trunk/ dpkt-read-only
cd dpkt-read-only
sudo make install
cd /usr/share/doc/python-pypcap/examples
python test.py




Primary obstacles

Evaluation

Metrics that will be measured

  • Percentage of local traffic
  • Caching efficiency (ratio of incoming traffic to external destination to outgoing traffic to external destination)
  • DNS hit rate
  • DNS delays (can be quite significant) , % of DNS hit on local DNS
  • Up/Down Traffic usage over a 2 week loggin interval
  • Flow analysis
- TCP connection durations
- Number of simultaneous flows in the network over time
- Plot of bandwidth used per IP source address in the network - check if there are clear dominant users
- # Retransmissions
- TCP round trip times for ACKS
  • Breakdown of application classes using port numbers
- Peer-to-peer traffic
- Web
- video streaming ... protocols like RTSP have there own port - also IP based for flash sites like youtube
- VoIP ... prototocols like SIP and some known Skype ports - Skype is a challenge if using port 80
- Instant messaging ... could catch things like IRC but maybe IP addresses based again
- tunneling

Useful links

Monitoring tools

  • tshark

Scraping tcpdump files

To analyse packet size distribution:L tcpdstat -w out.log  <tracefile>

Understanding port numbers and other networky stuff


DNS analysis

  • Macha uses dnsmasq (uses a default of only 150 names in the cache)
dnsmasq -c --cache-size=<cachesize> 
- if cachesize=0 it disables it

Checking reboot commands issued

  • Check with
last reboot


Squid proxy analysis

This to be put in a new section - academic writing guide