RFC: Subscriptions harmonizer

Here's a new Web Service for people who use aggregators and have more than one computer. The problem: I subscribe to a feed at home but my aggregator at work doesn't know about it, and vice versa. This document is an RFC, and a roadmap for deployment.

Goals 

1. The solution should work equally well for any aggregator. It should be possible, for example to use NetNewsWire on a Mac at home; Radio UserLand on a Windows machine at work; or AmphetaDesk on a Linux laptop. The subscriptions of each should be harmonized without the harmonizer knowing what app is talking to it.

2. It should rely as little as possible on the centralized component. I will operate a prototype service at Harvard, but it will be limited to 1000 users, and each user will only be able to harmonize 100 times per day. The protocol will be openly specified and clonable. I'm only trying to solve the technological problem, not the economic one.

Overview 

The user establishes membership with the harmonizer app through a Web browser interface.

In the aggregator, a new configuration option, to enter the domain name, port, path for the harmonizer application. (My test server is at blogs.law.harvard.edu, port 80, path /RPC2.)

The first time the aggregator runs after this setup (it could be the first time it runs) it sends an XML-RPC message to the server containing the username, password and an array of URLs of feeds the user is subscribed to.

When the user subscribes to a new feed, a different message is sent to the server, indicating that the feed has been subscribed to. Similarly when the user unsubscribes.

When the aggregator starts up, it sends a message requesting an array of all the feeds the user has subscribed to. The aggregator loops over the array, and for each new feed, it sets up the local aggregator so that it will check the feed in its hourly scan.

Entry-points 

subsHarmonizer.setup (username, password, array) returns boolean

subsHarmonizer.subscribe (username, password, array) returns boolean

subsHarmonizer.unsubscribe (username, password, array) returns boolean

subsHarmonizer.startup (username, password) returns array

Radio implementation 

I've got an implementation of the aggregator-side that runs in Radio UserLand. The configuration is not browser-based. This is the very first step in a bootstrap, please only give it a try if you're experienced working in Radio's object database. Support will be limited if not non-existant.

1. First become a member of this Manila site. Make note of the email address (username) you use to sign up and your password. You'll need it in the next step.

2. Next, download apps.subsHarmonizer.fttb. Bring the Radio app to the front and open it from the File menu. Click on OK to all confirmations. Run subsHarmonizer.init. Jump to user.subsHarmonizer.prefs (Cmd-J on Mac, Control-J on Windows), and enter your username and password for the Harmonizer website. Of course since you've only installed the software on one computer, nothing magical can happen yet.

3. Repeat the instructions in the previous paragraph for another computer running Radio. After a couple of hourly scans the subscription lists of both computers should be in harmony. And then the magic kicks in. When you subscribe to a new feed, Murphy-willing, the other computer will hear about it (not immediately, when it does its next hourly scan). And if you unsubscribe a feed, the other computer will forget about it. Pray hard. I'm about to install the software on a second computer myself. Wish me luck! (It seemed to work.)

Bug fix -- 7/3/03 by DW 

I fixed a bug in the Radio aggregator-side implementation. It now unsubs from any feed that is not present in the array it receives from the server on startup. Without this change, unsubs would not percolate. To install the fix, download and install apps.subsHarmonizer.fttb, per the instruction in step 2 above.

New feature -- 8/17/03 by DW 

A new entry-point, allows the client to send information about the feeds back to the harmonizer, at the end of a scan. This allows the harmonizer to know the names of the channels, without having to read them.

subsHarmonizer.setChannelInfo (username, password, struct) returns boolean

The struct is a struct of struct. The name of each sub-struct is the URL of the feed. Each struct contains the following fields: title, link and description. It may contain other information, which the harmonizer may or may not store.

Other implementations 

As developers of other aggregators or their users produce kits that work with the harmonizer, I'll record them here. Obviously this is an open protocol, anyone can implement it. And as noted above, I don't plan to operate a server for everyone on the Internet. Other servers will obviously be needed if this is to work.

Phillip Pearson: Python.

Philip Miseldine has a subs harmonizer running in .NET.

Andrew Pearson did a subs harmonizer in PHP.