[IMC-Tech] Re: rewrite rules

[Mike Lehman]

I want to also express my appreciation for the work being done by 
everyone on this big, but very much needed project.

Here's a few suggestions from the user perspective.

We've already agreed that the new site needs to be up, due to spam, etc. 
So if the old Dada site needs to go down for the time being, as long as 
the new site goes up in its place at www.ucimc.org, that's cool. And 
very much needed. There are lots of things going on in our community 
right now, so we badly need to get something usable up where people are 
used to finding it.

It will be good to be able to access the old site, sooner rather than 
later, so if Dan can get it going on an isolated server where it won't 
continue to cause problems and can be easily diagnosed, that sounds like 
a plan. In the meantime, we can manage without it, as long as people 
find the new site in its place.

AFAIK, the patches on Dada that were made by Clint were due to the 
legacy system (SFActive?) needing to be patched into Dada. Is it 
possible to just pull from the Dada database since then and do a 
relatively clean and easy update to whatever the latest build of Dada is 
with just that part of the database? If so, then we can leave the oldest 
part of ucimc.org -- the one that was patched in and has most of the 
potential issues that might snag a Dada update -- for last. I think 
we're talking about just the first two years, as I think the last four 
years were since the update to Dada was done.

But maybe I don't know WTF I'm talking about with that.

Anyway, I'm just saying that people are badly itching for a usable site 
ASAP -- and the rest can come along when it's figured out. No one is 
going to mind if archives of older stuff aren't available right away.
Regards,
Mike Lehman

dan blah wrote:
> On 10/1/06, Zachary C. Miller <zach at zarfmouse.us> wrote:
>> The reason I harp on this so much is that it is the ROOT cause of all
>> the resource problems that imsahp has. System response too slow for
>> you? Mail gets backed up on the first of the month? Hourly downtime
>> annoying? Some web requests hang forever? These are all rooted in the
>> apache hangs. Once we solve the problem of the apache hangs there will
>> be a TON of available CPU power available and a definite increase in
>> speed and stability of all services on imsahp.
>>
>> Anyone who is inspired to pour a ton of effort into helping fix the
>> problems that imsahp has really needs to pour their efforts into
>> solving this problem first and foremost. Moving to Drupal is a start.
>> Moving everything to Apache2 is a start. But as long as we're
>> continuing to run Dada in any form (or on the off chance that we
>> discover that Dada is not the cause) we have to be prepared to
>> continue to deal with the issue of apache hangs. If we can isolate
>> various potential causes (upgrade to Apache2 so we can't blame old
>> apache, upgrade to php5 so we can't blame old php, move Dada off the
>> system entirely so we can't blame Dada, stop even loading mod_perl so
>> we can't blame mod_perl, etc) we will figure out what the real cause
>> is. We're on the right track, I just want to make sure we're all on
>> the same page about what the track is.
>>
> my plan is to do just this but after we get the old dada site working
> on the new server.  as there are no resource issues at all on that
> server with most all of hosted sites working its a fairly safe
> assumption that apache issues on imsahp are related to the current
> ucimc dada installation.  provided we can get the old dada install
> work on the new server w/o causing any resource issues that pretty
> much solves our problems.  from there i will start going through the
> steps you have have listed just for the sake of knowing (i am curious
> exactly what the problem is).
>
>> Another backwards compatibility issue we have to consider: When Clint
>> ported us to Dada, he made sure that old URLs to the original "IMC
>> active" software still worked. I don't know if he did that by leaving
>> a crippled install of active in place or by porting all the old
>> articles into Dada and putting in a forwarding script, I've never
>> looked into it. We'll need to figure that out (or maybe I'm just
>> imagining that he ever even did it) and make sure those old old URLs
>> also remain valid if possible.
>>
> as for the rewite script, awesome.  i cped your pub key on imsahp to
> zeco.chambana.net (new web server).
>> On 10/1/06, Zachary C. Miller <zach at zarfmouse.us> wrote:
>> > Note to all: please understand this well: read this all and don't skim
>> > it...I feel like I've explained this problem in detail before and
>> > people seem to still be making other theories about what is happening
>> > without having addressed what I know to be the facts of the problem.
>> > I'm not trying to be snarky, I'm EXTREMELY grateful that so many
>> > people (especially David and Dan!!!! Thanks!!!!) are stepping forward
>> > to take care of imsahp, I don't have time to do it and you guys are
>> > the lifeline that the system needs. I'm just trying to make sure that
>> > we REALLY understand the problem that needs to be solved, rather than
>> > taking wild guesses about what will fix it.
>> >
>> > The primary bog down that Dada seems to be causing (it might not be
>> > Dada, it might be something else) on imsahp is that apache processes
>> > become STUCK in a loop that consumes 100% of the available CPU and
>> > never times out. The stuck processes build up until the server is so
>> > bogged down that it is unusable. Nothing gets logged when apache
>> > becomes stuck in this manner so it is impossible to know for sure
>> > whether Dada is causing the problem but a lot of circumstantial clues
>> > suggest Dada is the cause.
>> >
>> > PHP is embedded in Apache so I believe the stuck processes are due to
>> > Dada exposing some bug in PHP. Even badly written PHP code should time
>> > out eventually unless there is a bug in PHP. This is why, even though
>> > I believe the problem is caused by Dada, I've advocated in the past
>> > for an upgrade of PHP and of Apache (as well as an upgrade of Dada) to
>> > see if the bug has already been fixed in a newer version. These
>> > upgrades are neccessary anyway for security and stability and
>> > supportability purposes.
>> >
>> > I suspect (this is WILD speculation) that the problem might be related
>> > to memory allocation.  Some OJC people have told me they've seen
>> > similar problems with older version of mod_perl and that they were
>> > memory allocation related. (And honestly, if someone could check all
>> > the websites we host and confirm that none of them are using mod_perl,
>> > I'd suggest disabling mod_perl and see if that helps).
>> >
>> > I do not believe the problem is related to rebuilding of indexes
>> > because it is Apache and not MySQL that sucks up the CPU time. The
>> > stuck apache processes do not respond to normal kill signals (they
>> > must kill -9'ed) so we're really talking about the exposure of a BUG
>> > and not just some rogue code that takes a lot of power to run.
>> >
>> > Note that there are 3 such stuck processes on imsahp at this very
>> > moment as I write this email. They'll go away as soon as the hourly
>> > restart of apache happens. Then they'll inevitably come back over the
>> > next hour. If you guys have stopped posting on Dada, it clearly has
>> > NOT stopped the stuck apache process problem. It will continue to
>> > plague us on a system that is running Dada in "archive" mode.
>> >
>> > I am pretty sure that I have seen the stuck process thing happen while
>> > I was making a web request to simply read the index page (not posting
>> > an article) of ucimc.
>> >
>> > Curently to address this problem there is a root cronjob that restarts
>> > apache every hour. This causes about 5 minutes of downtime for every
>> > site on the server every hour at 51 minutes past the hour. This is a
>> > terrible hackish stop gap that has allowed the server to survive but
>> > the server is still constantly a bit bogged down by a few of these
>> > rogue apache processes. Performance on the server will skyrocket when
>> > this fundamental apache problem is solved.
>> > 51 * * * * /home/wolfgang/bin/apache-restart >> 
>> /var/log/apache-restart.log 2>&1
>> >
>> > If we're going to have an "archive" version of dada, it needs to run
>> > on it's own separate dedicated server. Then we can isolate Dada and
>> > confirm that it is the culprit causing the apache hangs and if it is
>> > we can rest assured that it won't take the rest of imsahp down with
>> > it. We might learn from this experiment that Dada has nothing to do
>> > with causing the Apache hangs...but it'd be nice to learn that. We
>> > need to use a process of elimination to figure out what is triggering
>> > the apache hangs.
>> >
>> > I can help configure rewrite rules if someone can set up the 
>> archive server.
>> >
>> > On 9/27/06, David Gehrig <gehrigspamtrap at gmail.com> wrote:
>> > > It seems to me that post-less dada isn't bogging down the system 
>> the way it
>> > > usually does -- that dada's problem might be something to do with
>> > > regenerating indexes or something else cycle-sucking every time 
>> someone
>> > > posts, and that the spamming was therefore bringing it -- and 
>> everything
>> > > else on imsahp -- to its knees.
>> > >
>> > > If there's agreement on that, then it's time to switch to the new 
>> site.
>> > >
>> > > I am not the master of rewrite rules, though.  Dan, could you set up
>> > > rewrites to do what I've described in step (3) below?   The idea 
>> is that any
>> > > URL that goes to www.ucimc.org/newswire/.... goes instead to
>> > > archive.ucimc.org/newswire... etc.
>> > >
>> > >
>> > > On 9/26/06, David Gehrig <gehrigspamtrap at gmail.com> wrote:
>> > > > All right, here's my so-called thoughts.
>> > > >
>> > > > (1) I'm really very reluctant to try to find a way to graunch
>> > > > the dada data into drupal.  I don't think it can be done well,
>> > > > and I don't have the time to waste doing it badly.
>> > > >
>> > > > (2) Let's watch the lobotomized dada site for a few more
>> > > > bounces -- if the drain on the system was related to the
>> > > > process of posting to dada, then we should see better
>> > > > response all around now that it no longer is.  Early signs
>> > > > look good.
>> > > >
>> > > > (3) Here's a little more detail of what I'm suggesting.
>> > > >
>> > > > If a URL request comes in for
>> > > >
>> > > > www.ucimc.org/newswire/....
>> > > > www.ucimc.org/mod/info/...
>> > > > www.ucimc.org/editor/...
>> > > > www.ucimc.org/library/...
>> > > >
>> > > > then redirect it to archive.ucimc.org.
>> > > >
>> > > > Otherwise have it go to the drupal site.
>> > > >
>> > > > I'm assuming this would have to be done in the main
>> > > > apache config file. If this passes a basic sanity check, let's 
>> try it.
>> > > >
>> > > > @%<
>> > > >
>> > >
>> > >
>> > > _______________________________________________
>> > > IMC-Tech mailing list
>> > > IMC-Tech at lists.ucimc.org
>> > > http://lists.chambana.net/cgi-bin/listinfo/imc-tech
>> > >
>> > >
>> > >
>> >
>> >
>> > --
>> > Zachary C. Miller - @= - http://zach.chambana.net/
>> > IMSA 1995 - UIUC 2000 - Just Another Leftist Muppet - Ya Basta!
>> >  Social Justice, Community, Nonviolence, Decentralization, Feminism,
>> >  Sustainability, Responsibility, Diversity, Democracy, Ecology
>> >
>>
>>
>> -- 
>> Zachary C. Miller - @= - http://zach.chambana.net/
>> IMSA 1995 - UIUC 2000 - Just Another Leftist Muppet - Ya Basta!
>>  Social Justice, Community, Nonviolence, Decentralization, Feminism,
>>  Sustainability, Responsibility, Diversity, Democracy, Ecology
>> _______________________________________________
>> IMC-Tech mailing list
>> IMC-Tech at lists.ucimc.org
>> http://lists.chambana.net/cgi-bin/listinfo/imc-tech
>>
>
>