Here is an enormous Yahoo Pipe. You can see it in it’s full size by clicking here Giant Yahoo Pipe.
I’m going to use it to explain how to perform many different data manipulations using Yahoo Pipes. Just so you know, Yahoo Pipes is used to pull data from websites, manipulate that data, create an RSS feed with that data, and then send it where ever you would like. For more information on Yahoo Pipes see my tutorial Yahoo Pipes Tutorial.
In this article I’ll show you:
And, a bunch more!
Remember you can open the full screen version of the Yahoo Pipe to follow along.
Using the Yahoo Search Module in Yahoo Pipes
Here I’m using the 2 User Input modules (URL Input & Text Input) to pass text to the Yahoo Search Module. I then am filtering those results, to only return articles that contain the word star. I pull the full article from the website and place that in the feed description. To finish it all off, I truncate the results to the top 5 results. This is what it looks like.
Yahoo Search & User Input Modules
The Yahoo Search Module takes 2 inputs. The URL you want to restrict the search to. (This is optional) It also excepts the keywords you want to search for. It will then output the results, sorted in the order of relevance to the search term.
I’m allowing the Yahoo Pipe user to enter the chosen URL to search through, with the URL Input Module. You can define the order in which it is listed by typing a value into the Position field. I also chose the Huffington Post as the default URL to search.
Similarly I used the User Text Input Module to enter the chosen keyword, or I used the word “Fed” as a default.
Yahoo Pipe Filter Module
I then defined that I want to block any articles in which the description contains the word “star”. I could also only permit articles with the word “star”. Searches can be further refined by adding additional rules and demanding that descriptions contain all of the words I list. You can filter based off of whether the words lie in numerous locations such as the title (item.title), description (item.description), link to original article (item.link), etc.
Yahoo Pipe Truncate Module
After all of the articles are gathered I then use the Truncate Module to eliminate all of the articles except for the top 5 relevant ones. You do this with the Truncate module.
Fetch a Feed & Fetch a Foreign Feed & Translate it
The next Yahoo Pipe modules will be used to:
This is what it would look like:
What Am I Doing Here?
RSS Feeds are great, but not great for posting to your automated website. To get the whole article, instead of a mini description, just combine the Fetch Feed, Loop and Sub String Modules. Here is what to do:
Wrapping Up the Giant Yahoo Pipe
The Unique Module
After every feed is combined into one, I send them all to the Unique Module. This module will in this case check for duplicate titles. That is probably the only option, of those available, that will eliminate duplicate articles in the feed.
The Sort Module
I use the Sort Module, to sort all of the feeds based off of publication date, in ascending order. You can sort based off of other factors, but I believe this is the most common option chosen.
The Sub String Module
The Sub String Module, will trim the articles to a maximum length of 200 characters, in this example. You could choose to make the length of the description larger or smaller with it.
The String Builder Module
Here I’m creating a Read More link at the end of the article that will point back to the original article. This is considered good web etiquette and is required by most sites online. The String Builder Module allows you to build custom strings on the fly. To create the Read More tag I just enter:
That’s All Folks
There is a giant Yahoo Pipe that demonstrates how to use most of the modules and how they are often used. If you have any questions leave them in the comment section below.
Till Next Time