rsspipe: Pipe the command line to RSS

It's a general tool, but it may help if I give you a specific usage example. I use it to keep track of recent referrers. I have a long-running command like this:

tail -c 1000000000 -f access_log
|./weblog_parse -quiet referer
|./xuniq
|python rsspipe.py --title 'rentzsch.com referrers' referrers-rentzsch.xml

That's one line -- I inserted line breaks on the pipes to make it easier to follow. Let's walk through it command-by-command:

• First I have tail read my entire log file (-c 1000000000) and continue reading it forever (-f).
• Those log lines are fed to a slightly modified version of the excellent weblog_parse, which extracts the referrer from the log line.
• The referrer lines are fed to xuniq, which is a lot like uniq except its input doesn't need to be sorted first. It only outputs unique lines (lines it hasn't seen before).
• Those unique referrers are transformed into a rolling RSS file for ongoing consumption. You can see it in action here.

The only things it requires is input from stdin and an output file path. It does support a few options, however:

• --title: the feed's title. Defaults to "rsspipe".
• --link: the feed's link. Defaults to "http://rentzsch.com/code/rsspipe".
• --description: the feed's description. Defaults to "rsspipe".
• --limit: number of lines to export as RSS items. Defaults to 100.

Monday, November 01, 2004
06:54 PM