<p dir="ltr">I'll also chime in late (just landed in dc). My plan so far is to develop the Windows client in python sharing as much code as possible with the linux client. I'll also throw my backing behind python testing as I'll be doing a certain amount of that as I work on the client, trying to be as TDD as possible. </p>
<p dir="ltr">I also have no real kua knowhow other than haing read over some tutorials. </p>
<p dir="ltr">Josh</p>
<div class="gmail_quote">On Apr 24, 2013 5:58 PM, "Hans of Guardian" <<a href="mailto:hans@guardianproject.info">hans@guardianproject.info</a>> wrote:<br type="attribution"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div style="word-wrap:break-word"><div><br></div><div>Part of what makes python so productive is the vast amount of libraries available. If you can find Lua libs for everything, then that would be a starting place. From my personal perspective, I've never done any Lua, so I'd be unlikely to contribute much to Lua-based frameworks.</div>
<div><br></div><div>.hc</div><br><div><div>On Apr 24, 2013, at 5:48 PM, Dan Staples wrote:</div><br><blockquote type="cite">
<div bgcolor="#FFFFFF" text="#000000">
What about porting some of this to Lua, since OpenWRT nodes already
have that interpreter? Would a lot be lost in the translation?<br>
<br>
Also, we can look at the tests run at the latest BattleMesh last
week. I believe Seamus knows the repo where those are...I can't find
them at the moment.<br>
<br>
<div>On 04/24/2013 05:17 PM, Hans of
Guardian wrote:<br>
</div>
<blockquote type="cite">
<div><br>
</div>
<div>Sorry to be late in weighing in this thread. I'm on the
train to DC and diving back into a Commotion state of mind.</div>
<div><br>
</div>
<div>I like having python test framework a lot, even if it doesn't
run on all OpenWRT devices. To cover OpenWRT, you could set up
OpenWRT on any device that has a lot of disk space, then run all
the python stuff there. That would then give you a place to run
the full test suite on OpenWRT. Then on the Ubuquiti hardware
could be handled via some of the other ideas here, like shell
scripts, or remote scraping via JSON, etc.</div>
<div><br>
</div>
<div>Using python is just so vastly more productive that this
approach will definitely be less work even though there would
have to be some special cases for certain OpenWRT devices</div>
<div><br>
</div>
<div>This reminds me, would this also then run in the VMs that you
have been setting up? I think having some random box somewhere
running like 20 or more various VMs with olsrd on them could be
very nice as an automated testbed. This would not test the real
world wireless conditions, but it would provide a way to run
automated tests on olsrd meshes in a way that could be plugged
into Jenkins builds. This kind of testbed would have caught the
commit that broke meshing on ARM and amd64 that showed up in the
more aggressive optimization of Ubuntu/quantal's compiler.</div>
<div><br>
</div>
<div>.hc</div>
<br>
<div>
<div>On Apr 2, 2013, at 10:31 AM, Ben West wrote:</div>
<br>
<blockquote type="cite">
<div dir="ltr">
<div class="gmail_extra">Hi Seamus and Dan,<br>
<br>
</div>
<div class="gmail_extra">Replying about having Python on
(and off) OpenWRT:<br>
</div>
<div class="gmail_extra"><br>
The cross-platform nature seems like it would be very
helpful to maintain a single collection of tests vectors
to apply as Commotion finds its way onto more platforms
(Windows, OSX). But, the obvious tradeoff here is the
size of the Python interpreter.<br>
<br>
</div>
<div class="gmail_extra">Also, the test scripts that query a
node's OLSR instance via jsoninfo could also run <i>on
the client</i>, rather than directly on the node. The
node would just need to open up the jsoninfo port 9090 on
its firewall behind the public AP interface (or have the
client use its private AP). I can tweak the Python
scripts (tho very little needed) to accommodate this
option.<br>
</div>
<div class="gmail_extra"><br>
</div>
<div class="gmail_extra">Besides that, I think I may be able
to tweak the Python Makefile provided with OpenWRT, so
that the compiled package is much smaller than its current
1.5MByte footprint. The Python v2.6 implementation
bundled into Py4A (which runs all these tests fine) is <i>only
150kB</i>, by comparison. Still, admitted that
installing another language on what is already disk- and
RAM-bound devices is less than ideal.<br>
<br>
</div>
<div class="gmail_extra">Finally, sorry for using the term
'bytecode.' That's not really what we need; compiled
Python bytecode is neither smaller than source nor
stand-alone executable. What I meant is maybe compile the
Python testsuite into a single standalone executable using
a tool like these:<br>
<a href="http://cx-freeze.sourceforge.net/" target="_blank">http://cx-freeze.sourceforge.net/</a><br>
<a href="http://www.pyinstaller.org/" target="_blank">http://www.pyinstaller.org/</a><br>
</div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Tue, Apr 2, 2013 at 8:58 AM,
seamus tuohy <span dir="ltr"><<a href="mailto:s2e@opentechinstitute.org" target="_blank">s2e@opentechinstitute.org</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000">
<div>Hey,
<div><br>
<br>
On 04/02/2013 04:20 AM, Ben West wrote:<br>
</div>
</div>
<blockquote type="cite">
<div dir="ltr">
<div>
<div>
<div>Hi All,<br>
<br>
<div> I've created a repo of a
few basic tests implemented as unittests
in Python 2.6+. This repo contains just
the Python scripts themselves, so that
it may be included later as a submodule
into other repos containing
platform-specific packaging stuff.<br>
<a href="https://github.com/westbywest/commotion-tests-core" target="_blank">https://github.com/westbywest/commotion-tests-core</a><br>
</div>
</div>
<div> <br>
Due please note I expect this repo to
change, and even its location to maybe
switch to OTI's github account at some
point soon.<br>
<br>
</div>
</div>
<div> To serve as hopefully
demonstrative examples of the test coverage
possible, the scripts I just checked in test
for the following:<br>
</div>
</div>
<div>
<div>
<ul>
<li>ping localhost</li>
<li>ping gateway IP (either default
gateway or the one assigned by OLSRd
smartgateway)</li>
<li>ping Google DNS (aka do we have
Internet?)</li>
<li>presence of an active olsrd process</li>
<li>olsrd responds to jsoninfo requests</li>
<li>link quality of the next hop is above
a specified threshold</li>
</ul>
</div>
<div> These scripts should run under the
following Python versions / platforms:<br>
<ul>
<li>Python v2.7+ under Debian/Ubuntu</li>
<li>Python for Android (Py4A) app r5+ /
Scripting Layer for Android (SL4A) app
r6+</li>
<li>Python-mini v2.7+ under OpenWRT 12.09+<br>
</li>
</ul>
</div>
<div>... to test the following Commotion
implementations, respectively (where each is
enabled by the user, not by the Python
script):<br>
<ul>
<li>commotion-mesh-applet<br>
</li>
<li>Mesh Tether app<br>
</li>
<li> Commotion-OpenWRT DR1, with <b>python</b>,
<b>python-json</b>, and <b>olsrd-mod-jsoninfo</b>
modules installed<br>
</li>
</ul>
</div>
<div>Questions about how to proceed:<br>
<ul>
<li>Whether to proceed with Python-based
testing framework? I tried to heavily
leverage the cross-platform
compatibility, but is it worthwhile?</li>
</ul>
</div>
</div>
</div>
</blockquote>
<div>
<blockquote type="cite">
<div dir="ltr">
<div>
<ul>
<li>The Python for Android implementation
chosen, Py4A, is only Python v2.6. Is
having Python v2.7 on Android worth
possibly compiling it into a custom APK?</li>
<li>Many of the desired test vectors, e.g.
throughput testing, require the test run
simultaneously on at least 2 nodes.
It's pretty easy to write a crude server
in Python to function as one half of a
throughput test, but does the complexity
of running different Python scripts on
different nodes simultaneously become
unreasonable</li>
</ul>
</div>
</div>
</blockquote>
</div>
Not at all, I think that it is necessary to have
different nodes running tests simultaneously. In
most of the openWRT testing situations we run
currently, we have client devices running
simultaneously to elucidate information about the
mesh nodes. <br>
<div>
<blockquote type="cite">
<div dir="ltr">
<div>
<ul>
<li>The python-mini module for OpenWRT
chews up 1.5Mbytes of flash, and it
doesn't include unittest by default.
This unfortunately appears to be <i>too
much already</i>; it wouldn't fit on a
Nanostation flash w/o removing lots of
stuff. Maybe this size could be trimmed
down by modifying the python-mini
package's Makefile, or by distributing
test scripts in bytecode form to OpenWRT
nodes.<br>
</li>
</ul>
</div>
</div>
</blockquote>
</div>
I think that loading python on to openWRT nodes is
impractical. Having client devices running tests
over the mesh with possible on-node analytics
running for later collection will give us a good
idea of QOS without having to load down the OpenWRT
nodes, and as such, change their performance. We
will not get the same level of detail as the android
or desktop nodes, but it is really the best we can
do while maintaining an accurate image of the nodes
performance.
<div>
<div><br>
<br>
<blockquote type="cite">
<div dir="ltr">
<div>
<ul>
<li> <br>
</li>
</ul>
</div>
</div>
<div class="gmail_extra"><br>
<br>
<div class="gmail_quote">On Fri, Mar 15,
2013 at 10:14 PM, Andrew Reynolds <span dir="ltr"><<a href="mailto:andrew@opentechinstitute.org" target="_blank">andrew@opentechinstitute.org</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">It
sounds reasonable. Could you sketch out
what you have in mind? How<br>
much of the network setup were you
thinking of building into the test<br>
framework vs. simply triggering and
testing?<br>
<span><font color="#888888"><br>
-andrew<br>
</font></span>
<div>
<div><br>
On 03/15/2013 02:48 PM, Ben West
wrote:<br>
> Hi All,<br>
><br>
> In lieu of recent progress
towards getting Commotion to a
working state on<br>
> Android, Ubuntu/Debian. and now
possibly OSX, what thoughts about
building<br>
> a simple and (to whatever
degree feasible) cross-platform
testing framework<br>
> in Python?<br>
><br>
> The general idea is that python
scripts could be used to start
hitting the<br>
> test vectors listed here (note
the server appears to be really
slow):<br>
><br>
> <a href="https://code.commotionwireless.net/projects/commotion/wiki/Testing#Mesh-Routing-Tech-Evaluations" target="_blank">https://code.commotionwireless.net/projects/commotion/wiki/Testing#Mesh-Routing-Tech-Evaluations</a><br>
> <a href="https://code.commotionwireless.net/projects/commotion/wiki/Testing#Testbed-Requirements-based-on-test-suite-defined-above" target="_blank">https://code.commotionwireless.net/projects/commotion/wiki/Testing#Testbed-Requirements-based-on-test-suite-defined-above</a><br>
> <a href="https://code.commotionwireless.net/projects/commotion/wiki/Testing#Release-Candidate-Test-Regimen" target="_blank">https://code.commotionwireless.net/projects/commotion/wiki/Testing#Release-Candidate-Test-Regimen</a><br>
><br>
> That is, assuming
Ubuntu/Debian/OSX's python support
as a starting point,<br>
> could these lighweight python
implementations allow for some
unified test<br>
> scripts across platforms?<br>
><br>
> <a href="http://qpython.com/" target="_blank">http://qpython.com/</a>
(for Android)<br>
> <a href="https://dev.openwrt.org/browser/packages/lang/python/Makefile" target="_blank">https://dev.openwrt.org/browser/packages/lang/python/Makefile</a>
(for OpenWRT,<br>
> to be compiled as module)<br>
><br>
> Has anyone on the list had good
experience with these Python<br>
> implementations?<br>
><br>
> My original thought for such
testing scripts was to do them in
shell<br>
> scripting, but I'm guessing
Python would be easier and more
powerful.<br>
><br>
><br>
><br>
</div>
</div>
<div>
<div>>
_______________________________________________<br>
> Commotion-dev mailing list<br>
> <a href="mailto:Commotion-dev@lists.chambana.net" target="_blank">Commotion-dev@lists.chambana.net</a><br>
> <a href="https://lists.chambana.net/mailman/listinfo/commotion-dev" target="_blank">https://lists.chambana.net/mailman/listinfo/commotion-dev</a><br>
><br>
<br>
<br>
</div>
</div>
<br>
_______________________________________________<br>
Commotion-dev mailing list<br>
<a href="mailto:Commotion-dev@lists.chambana.net" target="_blank">Commotion-dev@lists.chambana.net</a><br>
<a href="https://lists.chambana.net/mailman/listinfo/commotion-dev" target="_blank">https://lists.chambana.net/mailman/listinfo/commotion-dev</a><br>
<br>
</blockquote>
</div>
<br>
<br clear="all">
<br>
-- <br>
Ben West
<div><a href="http://gowasabi.net/" target="_blank">http://gowasabi.net</a><br>
<a href="mailto:ben@gowasabi.net" target="_blank">ben@gowasabi.net</a><br>
<a href="tel:314-246-9434" value="+13142469434" target="_blank">314-246-9434</a><br>
</div>
</div>
<br>
<fieldset></fieldset>
<br>
<pre>_______________________________________________
Commotion-dev mailing list
<a href="mailto:Commotion-dev@lists.chambana.net" target="_blank">Commotion-dev@lists.chambana.net</a>
<a href="https://lists.chambana.net/mailman/listinfo/commotion-dev" target="_blank">https://lists.chambana.net/mailman/listinfo/commotion-dev</a>
</pre>
</blockquote>
<br>
</div>
</div>
</div>
<br>
_______________________________________________<br>
Commotion-dev mailing list<br>
<a href="mailto:Commotion-dev@lists.chambana.net" target="_blank">Commotion-dev@lists.chambana.net</a><br>
<a href="https://lists.chambana.net/mailman/listinfo/commotion-dev" target="_blank">https://lists.chambana.net/mailman/listinfo/commotion-dev</a><br>
<br>
</blockquote>
</div>
<br>
<br clear="all">
<br>
-- <br>
Ben West
<div><a href="http://gowasabi.net/" target="_blank">http://gowasabi.net</a><br>
<a href="mailto:ben@gowasabi.net" target="_blank">ben@gowasabi.net</a><br>
<a href="tel:314-246-9434" value="+13142469434" target="_blank">314-246-9434</a><br>
</div>
</div>
</div>
_______________________________________________<br>
Commotion-dev mailing list<br>
<a href="mailto:Commotion-dev@lists.chambana.net" target="_blank">Commotion-dev@lists.chambana.net</a><br>
<a href="https://lists.chambana.net/mailman/listinfo/commotion-dev" target="_blank">https://lists.chambana.net/mailman/listinfo/commotion-dev</a><br>
</blockquote>
</div>
<br>
<br>
<fieldset></fieldset>
<br>
<pre>_______________________________________________
Commotion-dev mailing list
<a href="mailto:Commotion-dev@lists.chambana.net" target="_blank">Commotion-dev@lists.chambana.net</a>
<a href="https://lists.chambana.net/mailman/listinfo/commotion-dev" target="_blank">https://lists.chambana.net/mailman/listinfo/commotion-dev</a>
</pre>
</blockquote>
<br>
<pre cols="72">--
Dan Staples
Open Technology Institute
<a href="https://commotionwireless.net/" target="_blank">https://commotionwireless.net</a></pre>
</div>
_______________________________________________<br>Commotion-dev mailing list<br><a href="mailto:Commotion-dev@lists.chambana.net" target="_blank">Commotion-dev@lists.chambana.net</a><br><a href="https://lists.chambana.net/mailman/listinfo/commotion-dev" target="_blank">https://lists.chambana.net/mailman/listinfo/commotion-dev</a><br>
</blockquote></div><br></div><br>_______________________________________________<br>
Commotion-dev mailing list<br>
<a href="mailto:Commotion-dev@lists.chambana.net">Commotion-dev@lists.chambana.net</a><br>
<a href="https://lists.chambana.net/mailman/listinfo/commotion-dev" target="_blank">https://lists.chambana.net/mailman/listinfo/commotion-dev</a><br>
<br></blockquote></div>