<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta http-equiv="content-type" content="text/html;
charset=ISO-8859-1">
</head>
<body bgcolor="#ffffff" text="#000000">
I think that this type of project would have the most effect if it
included special-case optimizations for four critical services, in
order of importance:<br>
<ol>
<li>One-to-many updates (RSS including twitter and facebook;
inbound)</li>
<li>One-to-many updates (updating twitter, facebook, etc;
outbound)</li>
<li>Short one-to-one messages (IM, SMS)</li>
<li>Medium-length one-to-one messages (email)</li>
</ol>
The first item could be implemented as follows:<br>
<br>
<b>Caching Client-Server RSS Proxy</b> (CSSRSSP)<br>
In net-accessible areas, servers run a daemon that accepts incoming
SSH connections (for compression as much as security), over which
the following protocol is run:<br>
<ul>
<li>client sends monitor<space>N<cr><lf> (for
int N) to indicate that it is monitoring N feeds. N should be
10 digits, 0-padded on the left.<br>
</li>
<li>client sends N monitor notifications, consisting of
A<space>B<space>C<space>D<cr><lf>,
where A is the number of users monitoring the feed, B is the
64-bit UTC timestamp of the last update received for this feed,
and C is the number of characters in the feed's URL, and D is
that URL. Numbers are 10, 20, and 5 digits, respectively,
0-padded on the left, to make read() as easy as possible</li>
<li>server begins to monitor the feed urls, polling each feed
(with higher frequency, down to the feeds minimum update time)
in a round-robin fashion weighted by # of monitoring users</li>
<li>When a feed is updated, server sends
A<space>B<space>C<cr><lf>, where A is
the timestamp of the update (20 digits), B is 0 if this is the
full feed, the length of the update (in bytes; 10 digits)
otherwise, and C is the diff of the RSS feed from the timestamp
last reported to be cached by this client</li>
<li>Client periodically updates server with new "monitor N" info.
A user count of 0 removes the feed from the client's list of
monitored feeds. Otherwise, an entry with an already-seen URL
updates the timestamp and user count on record for that client</li>
<li>Server caches the most recent list of monitored feeds and
metadata in a file, with name == MD5 checksum of the file</li>
<li>Client can resume a cached state (eg, after disconnection) by
sending resume<space>A<cr><lf> with the A ==
the md5sum of a previous state, to which the server should reply
either ok<cr><lf> if it has such a state or
no<cr><lf> if it does not<br>
</li>
</ul>
On the client side, this general daemon can be used to implement a
caching twitter, facebook, and general RSS-feed proxy (each one
would require some custom code).<br>
<br>
<br>
The second item could be implemented with custom code per updateable
service in the client-side interface to the proxy. That might be
preferable to using standard HTTP interface because those short
updates could be prioritized above general HTTP traffic.<br>
<br>
<br>
The third item is only hard if you actually need to proxy the
sessions to disallow or reprioritize file transfers (which should go
via the same priority as HTTP, for example). That amount of
prioritization might come for free with classification by port with
per-stream bandwidth throttling.<br>
<br>
<br>
The 4th item is basically a normal SMTP-hijacking mail server that
is configured to be pretty much an open relay, and deal well with
being disconnected (maybe have some batch-transfer mechanism like
UUCP). Extra points for cacheing the login credentials and original
target of an SMTP request, so you can forward a message to its
intended destination and avoid relaying issues...<br>
<br>
Some thoughts,<br>
Peter<br>
<br>
<br>
</body>
</html>