Tuesday, October 16, 2012

Simple example of Yahoo Pipes usage


I've been using Yahoo Pipes (or YP for short) since the very beginning. This is a very powerful and easy to use tool to collect and process data from various sources for free. With its simple user interface one doesn't have to be a programmer or IT professional to create a pipe. Unfortunately it looks like that YP is very much forgotten today. Maybe it is related to the Yahoo financial and marketing problems they are facing at the moment. Anyway, I'd like to show a very simple use case and how easy it could be solved using YP. If you're familiar with the YP you may skip the further reading. If not - continue under the cut.

I've been enjoying the NASA's Astronomy Picture Of the Day rss feed for years. The item's description tag contains a small icon of the image with the textual description. The NASA's Image Of the Day rss feed on the other hand contains no image in the description. The link to a media file is stored in the item's enclosure tag and it is displayed just as a link in my rss reader. Nobody reads books with no pictures, right? So, as far as all needed information is stored inside of the feed it is only needed to reformat item's description tag. And here YP comes into play. It serves as a proxy between the feed source and a consumer.

YP has very rich number of modules for data processing one can use. Indeed I've seen different solution for this particular problem involving different modules for strings manipulation but actually only one module is needed - the Regex module. I agree, regular expressions aren't easy to understand for a regular user but the concept is very powerful and it is worth getting yourself familiar with it.

The resulting pipe flow is very simple: 
  • fetch the feed source
  • process items with the Regex module
  • redirect the result to the output
  • profit
Below is the graphical presentation of the flow:


The Regex module simply inserts an img HTML tag with its url taken from the enclosure.url before the item's descrition text:
<div align="center"><a href="${link}"><img src="${enclosure.url}"/></a></div><br/>$1

That's all. Now all items contain an image in their description. You may clone this pipe or use the pipe's rss feed directly.

No comments: