[Commotion-dev] Stress test results

[Will Hawkins]

On Tuesday, November 12, 2013 00:00 EST, Paul Gardner-Stephen <paul at servalproject.org> wrote: 
 
> Hi Will,
> 
> Have you tried it with the recent servald update that fixed a few memory
> leaks?
> 
> As for high load generating malloc() activity in servald, it would need to
> be activity that touches servald in some way (obvious and direct or
> otherwise).  It will also be interesting to know if it is in servald or
> libserval being used by another application.  Are you able to repeat the
> test with servald not running, and see whether it still happens?

Paul,

Thanks for the response. I am planning on rerunning these tests tomorrow and had already planned on doing the tests that you suggest. I think that your suggestions will help us get to the bottom of what we are seeing here.

That said, it is my *hunch* that this crash has very little to do with any particular application. I think that it is more the result of the device simply running out of memory and having very little recourse but to restart.

I will post here with additional results. Thanks again for your response.

Will

> 
> Paul.
> 
> 
> On Tue, Nov 12, 2013 at 11:23 AM, Will Hawkins <
> hawkinsw at opentechinstitute.org> wrote:
> 
> > Using go (yes, that's right!) I was able to create a test program that
> > opened enough simultaneous HTTP connections to force a crash.
> >
> > Thanks to the fact that we were running a serial console that was
> > logging Pico station console output, we were able to capture the crash
> > information. I am attaching that here.
> >
> > Overall, it looks like the node simply runs out of memory. The first
> > errors are when malloc()s in servald fail Then, when things get really
> > bad, there are errors from the wireless driver saying that it cannot
> > allocate buffer space.
> >
> > Obviously the failures from the wireless driver are bad. They are
> > probably ultimately what causes the node to reboot.
> >
> > I wonder, though, about the servald malloc() failures. I'm not sure if
> > they are pure symptom (i.e, servald just happens to be the application
> > most commonly allocating memory space when the crash happens and so its
> > malloc()s fail first), or if it is part of the problem (i.e, servald
> > causes memory usage to skyrocket under heavy load and *then* these
> > larger memory problems start to occur).
> >
> > In any event, we got some logs, which is a good first step!
> >
> > Will
> >
> > _______________________________________________
> > Commotion-dev mailing list
> > Commotion-dev at lists.chambana.net
> > https://lists.chambana.net/mailman/listinfo/commotion-dev
> >
> >