[Commotion-dev] Python-based testing framework across Ubuntu/Debian and Android (and maybe OSX)?

[Ben West]

Hi All,

I've created a repo of a few basic tests implemented as unittests in Python
2.6+.  This repo contains just the Python scripts themselves, so that it
may be included later as a submodule into other repos containing
platform-specific packaging stuff.
https://github.com/westbywest/commotion-tests-core

Due please note I expect this repo to change, and even its location to
maybe switch to OTI's github account at some point soon.

To serve as hopefully demonstrative examples of the test coverage possible,
the scripts I just checked in test for the following:

   - ping localhost
   - ping gateway IP (either default gateway or the one assigned by OLSRd
   smartgateway)
   - ping Google DNS (aka do we have Internet?)
   - presence of an active olsrd process
   - olsrd responds to jsoninfo requests
   - link quality of the next hop is above a specified threshold

These scripts should run under the following Python versions / platforms:

   - Python v2.7+ under Debian/Ubuntu
   - Python for Android (Py4A) app r5+ / Scripting Layer for Android (SL4A)
   app r6+
   - Python-mini v2.7+ under OpenWRT 12.09+

... to test the following Commotion implementations, respectively (where
each is enabled by the user, not by the Python script):

   - commotion-mesh-applet
   - Mesh Tether app
   - Commotion-OpenWRT DR1, with *python*, *python-json*, and *
   olsrd-mod-jsoninfo* modules installed

Questions about how to proceed:

   - Whether to proceed with Python-based testing framework?  I tried to
   heavily leverage the cross-platform compatibility, but is it worthwhile?
   - The Python for Android implementation chosen, Py4A, is only Python
   v2.6.  Is having Python v2.7 on Android worth possibly compiling it into a
   custom APK?
   - Many of the desired test vectors, e.g. throughput testing, require the
   test run simultaneously on at least 2 nodes.  It's pretty easy to write a
   crude server in Python to function as one half of a throughput test, but
   does the complexity of running different Python scripts on different nodes
   simultaneously become unreasonable?
   - The python-mini module for OpenWRT chews up 1.5Mbytes of flash, and it
   doesn't include unittest by default.  This unfortunately appears to be *too
   much already*; it wouldn't fit on a Nanostation flash w/o removing lots
   of stuff.  Maybe this size could be trimmed down by modifying the
   python-mini package's Makefile, or by distributing test scripts in bytecode
   form to OpenWRT nodes.



On Fri, Mar 15, 2013 at 10:14 PM, Andrew Reynolds <
andrew at opentechinstitute.org> wrote:

> It sounds reasonable. Could you sketch out what you have in mind? How
> much of the network setup were you thinking of building into the test
> framework vs. simply triggering and testing?
>
> -andrew
>
> On 03/15/2013 02:48 PM, Ben West wrote:
> > Hi All,
> >
> > In lieu of recent progress towards getting Commotion to a working state
> on
> > Android, Ubuntu/Debian. and now possibly OSX, what thoughts about
> building
> > a simple and (to whatever degree feasible) cross-platform testing
> framework
> > in Python?
> >
> > The general idea is that python scripts could be used to start hitting
> the
> > test vectors listed here (note the server appears to be really slow):
> >
> >
> https://code.commotionwireless.net/projects/commotion/wiki/Testing#Mesh-Routing-Tech-Evaluations
> >
> https://code.commotionwireless.net/projects/commotion/wiki/Testing#Testbed-Requirements-based-on-test-suite-defined-above
> >
> https://code.commotionwireless.net/projects/commotion/wiki/Testing#Release-Candidate-Test-Regimen
> >
> > That is, assuming Ubuntu/Debian/OSX's python support as a starting point,
> > could these lighweight python implementations allow for some unified test
> > scripts across platforms?
> >
> > http://qpython.com/ (for Android)
> > https://dev.openwrt.org/browser/packages/lang/python/Makefile (for
> OpenWRT,
> > to be compiled as module)
> >
> > Has anyone on the list had good experience with these Python
> > implementations?
> >
> > My original thought for such testing scripts was to do them in shell
> > scripting, but I'm guessing Python would be easier and more powerful.
> >
> >
> >
> > _______________________________________________
> > Commotion-dev mailing list
> > Commotion-dev at lists.chambana.net
> > https://lists.chambana.net/mailman/listinfo/commotion-dev
> >
>
>
>
> _______________________________________________
> Commotion-dev mailing list
> Commotion-dev at lists.chambana.net
> https://lists.chambana.net/mailman/listinfo/commotion-dev
>
>


-- 
Ben West
http://gowasabi.net
ben at gowasabi.net
314-246-9434
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.chambana.net/pipermail/commotion-dev/attachments/20130402/ca2a8417/attachment.html>