[Imc-tech] Subject Guide

[Mike Perry]

Well, this is the letter I'm going to send to the global imc-tech people
about our subject guide ideas. I think it pretty much covers everything
we discussed at the meeting, no?

I can write a second one that covers more how the system will work on
from a nontechnical vantage point, focusing on how to write or submit
sources/suggested reading, why it's important, etc, that we can put up on
our website if you want, possibly as a request for articles to be
written while we await completion.

=============

Greetings. At the Indymedia site in Urbana we were considering building
a Subject Guide to be a warehouse for information on social and
progressive topics, to serve as a forum for people to communicate,
organize, and as a source for information for neophytes and expert
activists alike.

Though only the rudiments have been implemented so far, the plans we
have drawn up are quite extensive, and seem to be of a nature that
would not only lend itself well to a more global guide, but would be
actually most benefitted by a large diverse community of contributors.

So the basic idea is that we have a hierarchical message board system
where users would write about a subject, and then link it to related
subjects. Readers of the system could add links if they wanted, and then
each piece of information would have a set of in-links and out-links, in
addition to the initially imposed and browsable subject hierarchy. These
links could then be scored based on how often they are traversed (the
whole system will be in an SQL database, so it will be easy to track
queries), and individual nodes could be given a score based on a system
similar to google's page rank (see http://www9.org/w9cdrom/251/251.html),
taking link weights into account as well.

So, for the theoretically savvy and to satisfy the existential,
essentially you create a group consciousness that can be modeled in a
neural net, and you can then generate new hierarchy relations on the
data set based on page rank and link ranks. Or simply augment the
existing imposed hierarchy with better organization and relational
information. Surprisingly, the amount of work to implement this is not
as much as it would seem, nor is it all that complicated to at least get
something off the ground.

This sort of thing has been attempted before (google search for wiki's).
everything2.com is probably the biggest example of one of these systems,
but if you ask me is a horrible failure, not because of the number of
users or the amount of information, but because of the type of
information this "group mind" seems to produce. It's disorganized, and
leaps from one idea to another without really developing anything. Try
it out. You'll get this horrible feel of disorganization before you
eventually learn to think like the noise (and once you do that, god help
you.. insanity is quite addictive :).

I've thought about why this is, and I think it basically comes down to
the mentality of the user base. The everything2 net has 4 basic flaws.

1. No outside links. One of the rules of everything2 is that users NOT
link to outside resources. If you ask me, this in effect cuts the group
off from external reality, and all you end up with is self-consistent
nonsense. If you look at the database, real factual information is
viewed as secondary to short jabber.

2. No goals. The "mind" basically just wanders all over the place,
touching on idea after idea, without really generating any lengthy or
coherent information, or without discussing any real problems. The whole
database does have a very uniform feel in this respect.. The feel of
disorder.

3. Too low granularity. Part of the feel of disorganization stems from
the fact that this is everything, they have links and definitions for
the most trivial information.

4. No hierarchy. It's very hard to find a category of subject to focus
on. This may stem from 2 and 3.. One of these systems with a goal may
end up generating its own hierarchy with page rank, but everything2 does
not employ that either.

I think that a system like this can be used as a really top notch
discussion forum/subject guide when implemented in a way to overcome
these pitfalls. The Independent Media Center in Urbana mentioned that
they need a subject guide, and I was thinking what better goal for what
is essentially a social structure to have than to discuss social issues.
In order to avoid the chaos of everything2, we'll require that actual
data submissions be complete essays with at least one primary source,
and further outside references (which will also be tracked with
associations in a separate database, with a blurb on each).

Readers can then start discussion boards on a paragraph by paragraph
basis (like "reply to this paragraph", but tentatively called simply
"graffiti") or respond to an issue with another complete essay. When two
essays show up, the authors would be encouraged to converse via email
and write a third, or post revisions of their respective essays.

Furthermore, a specialized message board will be devoted to posts about
what action can be taken on a particular essay. This will be modeled in
such a way in the SQL database that it can be easily imported into local
IMC calendar software (if it exists.. The Urbana IMC is working on a
calendar tool as well).

More complete design notes and of course fully GPL'ed code will follow,
but most of us are tied down with finals and whatnot until mid-May. As I
have said, the project itself will be very easy to bring into a working
state to allow posting of essays, messages, and begin collecting
association data before the end of May for sure (some of the more
ambitious members say within a week after finals :)

The theoretical work to build relations may take a longer amount of
time, depending how accurate of a model we choose to use (simple bean
counting and maximization can also be implemented in about a day, but to
do transitivity and cycle detection may take longer). Also, if we wish
to display these relations in a graphical concept map, as well as just a
list of related topics at the bottom of each essay, we will be able to
do so, but this will of course take time. But having a map would just be
too damn cool a thing to pass up doing :)

Of course, all this technical information will be hidden from the user.
Submitting essays will simply be a matter of uploading a plain text
paper, and filling in a webform form for each bibliographical source not
present in the database. (Sources already present can simply be selected
from a list, or search function).

So this is just a preview of what's to come from the fine land of
Urbana, Illinois.

-- 
Mike Perry
Hacker in Chief
fscked.org media conglomerate