Weblogs.Com for RSS

Updated 10/30/02 DW

Weblogs.Com is a service that tells you, both in HTML and XML, which weblogs have changed in the last three hours. Now, for the first time, that service has been extended to RSS news feeds. You can easily find out which feeds have changed in the last three hours. This could be the key to improving the efficiency of news aggregators, as the market grows.

Read the rest of this howto for details on how to participate.

How and when to ping 

Suppose you have an RSS feed and want to inform interested parties that it has updated. Here's how to do it.

Call an XML-RPC procedure named rssUpdate on rssrpc.weblogs.com, port 80, path /RPC2. It's also available over SOAP, but for now we're only documenting the XML-RPC interface.

rssUpdate takes two parameters, both strings. The first is the name of the RSS feed that has changed and the second is the URL of the feed. It returns a struct indicating success or failure. The struct contains two elements, 1. flerror, a boolean that indicates if there was an error; and 2. message, a string, which explains what happened. It may be non-empty even if flerror is false.

The feed must be a valid RSS 0.9x, 1.0 or 2.0 file; or scriptingNews format.

changes.xml for RSS 

The end-result is a changes.xml file, exactly like the one for weblogs.com, except, instead of being for weblogs, it keeps track of changes in RSS feeds.

http://www.weblogs.com/rssUpdates/changes.xml

To get things started, I've seeded it with changes to the RSS feeds I subscribe to. This weekend we released an update for Radio that automatically pings this way when Radio-generated RSS files change. And at the same time, we're spreading the word to developers of other tools that this service is available and encourage them to support it.

Example 

Here's a one-line script that runs in Radio that notifies weblogs.com that InfoWorld's RSS feed has updated.

["xmlrpc://rssrpc.weblogs.com/RPC2"].rssUpdate ("InfoWorld", "http://www.infoworld.com/rss/news.rdf")

If you produce a content tool 

There are two things to do:

1. When you update an RSS feed, ping Weblogs.Com, as described above.

2. Include a channel-level element in the RSS feed, whose value is rssChanges and domain is the URL of the changes.xml file that's carrying your changes. At first we expect everyone will use Weblogs.Com, as explained above, and the domain will point to our changes.xml, but over time that should broaden, there should be a number of changes.xml's, maybe a large number. You may want to manage your own. That's why we allow for this to be specified by the RSS file itself.

This element tells us one very important thing about your feed -- where to look for changes. The economy comes in when a subscriber has two or more subscribed-to feeds that map to the same changes.xml file. Then one read does the work of two. If three feeds map to the same changes file, then the improvement is three-fold. Doing a little (optional) concentration and centralization results in better use of bandwidth and faster performance for users. Everyone wins.

If you produce an aggregator or news reader 

There's one thing to do:

1. Support .

The algorithm is simple. When you read a feed, if it contains a element, note that. The next time you scan, read the changes file first, and if the feed's url is contained in the changes file, read the file and process the new items. If it doesn't appear, you don't need to read it, you know it hasn't changed.

Note that changes.xml contains three hours of changes. The when attribute for each weblog item is the number of seconds since it's updated. The base time is the "updated" attribute on the top-level weblogUpdates element. Changes.xml is not guaranteed to contain three hours for all time, it may have just one hour at some point, if we need to adjust for bandwidth consumption. This is a free service provided by UserLand to the RSS community, including our competitors. We want to see the whole market do well.

If you use a content tool or aggregator 

Let your favorite developers know that you want the increased efficiency of this new method of aggregating RSS content!

Changes 10/30/02 by DW 

1. The server changed to rssrpc.weblogs.com. It's quite a bit faster and carries a lighter load than the first server we used.

2. Instead of using the element of the blogChannel module, we are recommending use of a element at the level to link the feed to the changes.xml file that broadcasts its updates. There are aggregators that don't work well with namespaces. If we are to deploy this feature in Radio, and perhaps in other content tools, we can't use a namespace here. Perhaps in the future. (I've suggested to Sam Ruby and Ben Hammersley that a serious test suite for aggregators and an informal certification process would be a good thing to start now so we can use namespaces with confidence in the future.) For an example of the element, check out the Scripting News RSS feed.

3. For Frontier and Radio developers, a new glue script, weblogUpdates.publicRssPing encapsulates the details of pinging a RSS feed. Update to get this part.