[IMC-Tech] Re: rewrite rules

Tue Oct 3 10:27:18 CDT 2006

(0) Sorry for being incommunicado over Yom Kippur.

(1) Thanks to Zach for the badly needed two-by-four between my eyes.

(2) A dada-to-drupal script would be terrific.  There was some discussion
of this on the imc-drupal-dev list, but at the time it looked like it was still
being written, if I remember right.

(3) A wiser head than my own has suggested that if we're going to be
disabling posting on dada, then we might as well -- once we're sure
we've got a bulletproof backup of the directory and the databases, in
case we ever need to stand it up again for some reason -- crawl the site
one last time, with something like wget, converting it to flat HTML, and
then have that flat, frozen site be our archive.  This would then allow
us to utter those magic words, "Bye bye dada" and remove the need
for having the archive site on a separate server.

@%<

On 10/2/06, dan blah <dan.blah at gmail.com> wrote:
> On 10/1/06, Mike Lehman <rebelmike at earthlink.net> wrote:
> > I want to also express my appreciation for the work being done by
> > everyone on this big, but very much needed project.
> >
> > Here's a few suggestions from the user perspective.
> >
> > We've already agreed that the new site needs to be up, due to spam, etc.
> > So if the old Dada site needs to go down for the time being, as long as
> > the new site goes up in its place at www.ucimc.org, that's cool. And
> > very much needed. There are lots of things going on in our community
> > right now, so we badly need to get something usable up where people are
> > used to finding it.
> >
> > It will be good to be able to access the old site, sooner rather than
> > later, so if Dan can get it going on an isolated server where it won't
> > continue to cause problems and can be easily diagnosed, that sounds like
> > a plan. In the meantime, we can manage without it, as long as people
> > find the new site in its place.
> >
> i have been trolling around the indymedia global server and have found
> there is a dada2drupal script being written to import old dada info.
> i havent found this yet but am digging.
> > AFAIK, the patches on Dada that were made by Clint were due to the
> > legacy system (SFActive?) needing to be patched into Dada. Is it
> > possible to just pull from the Dada database since then and do a
> > relatively clean and easy update to whatever the latest build of Dada is
> > with just that part of the database? If so, then we can leave the oldest
> > part of ucimc.org -- the one that was patched in and has most of the
> > potential issues that might snag a Dada update -- for last. I think
> > we're talking about just the first two years, as I think the last four
> > years were since the update to Dada was done.
> >
> > But maybe I don't know WTF I'm talking about with that.
> >
> > Anyway, I'm just saying that people are badly itching for a usable site
> > ASAP -- and the rest can come along when it's figured out. No one is
> > going to mind if archives of older stuff aren't available right away.
> > Regards,
> > Mike Lehman
> >
> > dan blah wrote:
> > > On 10/1/06, Zachary C. Miller <zach at zarfmouse.us> wrote:
> > >> The reason I harp on this so much is that it is the ROOT cause of all
> > >> the resource problems that imsahp has. System response too slow for
> > >> you? Mail gets backed up on the first of the month? Hourly downtime
> > >> annoying? Some web requests hang forever? These are all rooted in the
> > >> apache hangs. Once we solve the problem of the apache hangs there will
> > >> be a TON of available CPU power available and a definite increase in
> > >> speed and stability of all services on imsahp.
> > >>
> > >> Anyone who is inspired to pour a ton of effort into helping fix the
> > >> problems that imsahp has really needs to pour their efforts into
> > >> solving this problem first and foremost. Moving to Drupal is a start.
> > >> Moving everything to Apache2 is a start. But as long as we're
> > >> continuing to run Dada in any form (or on the off chance that we
> > >> discover that Dada is not the cause) we have to be prepared to
> > >> continue to deal with the issue of apache hangs. If we can isolate
> > >> various potential causes (upgrade to Apache2 so we can't blame old
> > >> apache, upgrade to php5 so we can't blame old php, move Dada off the
> > >> system entirely so we can't blame Dada, stop even loading mod_perl so
> > >> we can't blame mod_perl, etc) we will figure out what the real cause
> > >> is. We're on the right track, I just want to make sure we're all on
> > >> the same page about what the track is.
> > >>
> > > my plan is to do just this but after we get the old dada site working
> > > on the new server.  as there are no resource issues at all on that
> > > server with most all of hosted sites working its a fairly safe
> > > assumption that apache issues on imsahp are related to the current
> > > ucimc dada installation.  provided we can get the old dada install
> > > work on the new server w/o causing any resource issues that pretty
> > > much solves our problems.  from there i will start going through the
> > > steps you have have listed just for the sake of knowing (i am curious
> > > exactly what the problem is).
> > >
> > >> Another backwards compatibility issue we have to consider: When Clint
> > >> ported us to Dada, he made sure that old URLs to the original "IMC
> > >> active" software still worked. I don't know if he did that by leaving
> > >> a crippled install of active in place or by porting all the old
> > >> articles into Dada and putting in a forwarding script, I've never
> > >> looked into it. We'll need to figure that out (or maybe I'm just
> > >> imagining that he ever even did it) and make sure those old old URLs
> > >> also remain valid if possible.
> > >>
> > > as for the rewite script, awesome.  i cped your pub key on imsahp to
> > > zeco.chambana.net (new web server).
> > >> On 10/1/06, Zachary C. Miller <zach at zarfmouse.us> wrote:
> > >> > Note to all: please understand this well: read this all and don't skim
> > >> > it...I feel like I've explained this problem in detail before and
> > >> > people seem to still be making other theories about what is happening
> > >> > without having addressed what I know to be the facts of the problem.
> > >> > I'm not trying to be snarky, I'm EXTREMELY grateful that so many
> > >> > people (especially David and Dan!!!! Thanks!!!!) are stepping forward
> > >> > to take care of imsahp, I don't have time to do it and you guys are
> > >> > the lifeline that the system needs. I'm just trying to make sure that
> > >> > we REALLY understand the problem that needs to be solved, rather than
> > >> > taking wild guesses about what will fix it.
> > >> >
> > >> > The primary bog down that Dada seems to be causing (it might not be
> > >> > Dada, it might be something else) on imsahp is that apache processes
> > >> > become STUCK in a loop that consumes 100% of the available CPU and
> > >> > never times out. The stuck processes build up until the server is so
> > >> > bogged down that it is unusable. Nothing gets logged when apache
> > >> > becomes stuck in this manner so it is impossible to know for sure
> > >> > whether Dada is causing the problem but a lot of circumstantial clues
> > >> > suggest Dada is the cause.
> > >> >
> > >> > PHP is embedded in Apache so I believe the stuck processes are due to
> > >> > Dada exposing some bug in PHP. Even badly written PHP code should time
> > >> > out eventually unless there is a bug in PHP. This is why, even though
> > >> > I believe the problem is caused by Dada, I've advocated in the past
> > >> > for an upgrade of PHP and of Apache (as well as an upgrade of Dada) to
> > >> > see if the bug has already been fixed in a newer version. These
> > >> > upgrades are neccessary anyway for security and stability and
> > >> > supportability purposes.
> > >> >
> > >> > I suspect (this is WILD speculation) that the problem might be related
> > >> > to memory allocation.  Some OJC people have told me they've seen
> > >> > similar problems with older version of mod_perl and that they were
> > >> > memory allocation related. (And honestly, if someone could check all
> > >> > the websites we host and confirm that none of them are using mod_perl,
> > >> > I'd suggest disabling mod_perl and see if that helps).
> > >> >
> > >> > I do not believe the problem is related to rebuilding of indexes
> > >> > because it is Apache and not MySQL that sucks up the CPU time. The
> > >> > stuck apache processes do not respond to normal kill signals (they
> > >> > must kill -9'ed) so we're really talking about the exposure of a BUG
> > >> > and not just some rogue code that takes a lot of power to run.
> > >> >
> > >> > Note that there are 3 such stuck processes on imsahp at this very
> > >> > moment as I write this email. They'll go away as soon as the hourly
> > >> > restart of apache happens. Then they'll inevitably come back over the
> > >> > next hour. If you guys have stopped posting on Dada, it clearly has
> > >> > NOT stopped the stuck apache process problem. It will continue to
> > >> > plague us on a system that is running Dada in "archive" mode.
> > >> >
> > >> > I am pretty sure that I have seen the stuck process thing happen while
> > >> > I was making a web request to simply read the index page (not posting
> > >> > an article) of ucimc.
> > >> >
> > >> > Curently to address this problem there is a root cronjob that restarts
> > >> > apache every hour. This causes about 5 minutes of downtime for every
> > >> > site on the server every hour at 51 minutes past the hour. This is a
> > >> > terrible hackish stop gap that has allowed the server to survive but
> > >> > the server is still constantly a bit bogged down by a few of these
> > >> > rogue apache processes. Performance on the server will skyrocket when
> > >> > this fundamental apache problem is solved.
> > >> > 51 * * * * /home/wolfgang/bin/apache-restart >>
> > >> /var/log/apache-restart.log 2>&1
> > >> >
> > >> > If we're going to have an "archive" version of dada, it needs to run
> > >> > on it's own separate dedicated server. Then we can isolate Dada and
> > >> > confirm that it is the culprit causing the apache hangs and if it is
> > >> > we can rest assured that it won't take the rest of imsahp down with
> > >> > it. We might learn from this experiment that Dada has nothing to do
> > >> > with causing the Apache hangs...but it'd be nice to learn that. We
> > >> > need to use a process of elimination to figure out what is triggering
> > >> > the apache hangs.
> > >> >
> > >> > I can help configure rewrite rules if someone can set up the
> > >> archive server.
> > >> >
> > >> > On 9/27/06, David Gehrig <gehrigspamtrap at gmail.com> wrote:
> > >> > > It seems to me that post-less dada isn't bogging down the system
> > >> the way it
> > >> > > usually does -- that dada's problem might be something to do with
> > >> > > regenerating indexes or something else cycle-sucking every time
> > >> someone
> > >> > > posts, and that the spamming was therefore bringing it -- and
> > >> everything
> > >> > > else on imsahp -- to its knees.
> > >> > >
> > >> > > If there's agreement on that, then it's time to switch to the new
> > >> site.
> > >> > >
> > >> > > I am not the master of rewrite rules, though.  Dan, could you set up
> > >> > > rewrites to do what I've described in step (3) below?   The idea
> > >> is that any
> > >> > > URL that goes to www.ucimc.org/newswire/.... goes instead to
> > >> > > archive.ucimc.org/newswire... etc.
> > >> > >
> > >> > >
> > >> > > On 9/26/06, David Gehrig <gehrigspamtrap at gmail.com> wrote:
> > >> > > > All right, here's my so-called thoughts.
> > >> > > >
> > >> > > > (1) I'm really very reluctant to try to find a way to graunch
> > >> > > > the dada data into drupal.  I don't think it can be done well,
> > >> > > > and I don't have the time to waste doing it badly.
> > >> > > >
> > >> > > > (2) Let's watch the lobotomized dada site for a few more
> > >> > > > bounces -- if the drain on the system was related to the
> > >> > > > process of posting to dada, then we should see better
> > >> > > > response all around now that it no longer is.  Early signs
> > >> > > > look good.
> > >> > > >
> > >> > > > (3) Here's a little more detail of what I'm suggesting.
> > >> > > >
> > >> > > > If a URL request comes in for
> > >> > > >
> > >> > > > www.ucimc.org/newswire/....
> > >> > > > www.ucimc.org/mod/info/...
> > >> > > > www.ucimc.org/editor/...
> > >> > > > www.ucimc.org/library/...
> > >> > > >
> > >> > > > then redirect it to archive.ucimc.org.
> > >> > > >
> > >> > > > Otherwise have it go to the drupal site.
> > >> > > >
> > >> > > > I'm assuming this would have to be done in the main
> > >> > > > apache config file. If this passes a basic sanity check, let's
> > >> try it.
> > >> > > >
> > >> > > > @%<
> > >> > > >
> > >> > >
> > >> > >
> > >> > > _______________________________________________
> > >> > > IMC-Tech mailing list
> > >> > > IMC-Tech at lists.ucimc.org
> > >> > > http://lists.chambana.net/cgi-bin/listinfo/imc-tech
> > >> > >
> > >> > >
> > >> > >
> > >> >
> > >> >
> > >> > --
> > >> > Zachary C. Miller - @= - http://zach.chambana.net/
> > >> > IMSA 1995 - UIUC 2000 - Just Another Leftist Muppet - Ya Basta!
> > >> >  Social Justice, Community, Nonviolence, Decentralization, Feminism,
> > >> >  Sustainability, Responsibility, Diversity, Democracy, Ecology
> > >> >
> > >>
> > >>
> > >> --
> > >> Zachary C. Miller - @= - http://zach.chambana.net/
> > >> IMSA 1995 - UIUC 2000 - Just Another Leftist Muppet - Ya Basta!
> > >>  Social Justice, Community, Nonviolence, Decentralization, Feminism,
> > >>  Sustainability, Responsibility, Diversity, Democracy, Ecology
> > >> _______________________________________________
> > >> IMC-Tech mailing list
> > >> IMC-Tech at lists.ucimc.org
> > >> http://lists.chambana.net/cgi-bin/listinfo/imc-tech
> > >>
> > >
> > >
> >
> > _______________________________________________
> > IMC-Tech mailing list
> > IMC-Tech at lists.ucimc.org
> > http://lists.chambana.net/cgi-bin/listinfo/imc-tech
> >
>
>
> --
> Daniel
> _______________________________________________
> IMC-Tech mailing list
> IMC-Tech at lists.ucimc.org
> http://lists.chambana.net/cgi-bin/listinfo/imc-tech
>