One more post on topic of creating RSS feeds for sites without any. This time instead of relying on external services I used native sites’ capabilities and bit of Yahoo Pipes magic.
XML sitemap as source of data
Lots of sites have XML sitemaps . Basically it’s a list of internal links for search engines to look through. Good search engine practice – translating to human language “Google said jump and we do”. :)
See my sitemap for example http://www.rarst.net/sitemap.xml
Not much to look at, but what got me interested is that every link has time stamp. And what can be better for tracking site updates than list of links with time of last change?
Converting to comfortable form
Having to visit sitemap is hardly improvement over visiting site itself. So data needs to be converted in more suitable format for tracking updates – RSS of course. It could make epic coding evening suggestion for techies but I already had mine and present you Sitemap2RSS pipe.
Link http://pipes.yahoo.com/rarst/sitemap2rss
How it works
Go to pipe > type in site URL > Run Pipe > get preview of RSS > More Options > Get as RSS
Simple, isn’t it? There are some sitemap-related limitations like no nesting support (Yahoo server becomes unhappy when I process 17000+ links from nested Lifehacker sitemap).
Overall
It may not be most informative updates tracking ever but it’s extremely precise and uses information from site instead of scheduled external checks . Interesting side-effect is tracking changes to old pages. If page is edited date is often automatically changed in sitemap (even if not visible elsewhere).
WePC.com – seeks crowd wisdom for PC design
Is blogging new gaming?
Empowering feed with Media RSS via Yahoo Pipes
What RSS feed really is
FeedDemon and Google Reader – best feed reading ever?
Scrape headings into RSS with Yahoo Query Language
Follow tags on Delicious with cleaned up RSS feeds
Follow-up on Interfacelift and their witch hunt
Nice post since nothing in the hole web sugest about it.
But pipes only displayed here 15 articles.
What about more?
And full rss showed? Cause pipes blocks my show full rss feed script!
Thanks
The ideia explained in last comment it’s to have hole website in rss since webmaster do not provide it!
@Ricardo
You can adjust number of posts to display in Pipe’s code (you can click “Clone” at pipe’s page and get your own copy to play with, only need Yahoo account).
Tinkering to including content in such feed is possible, but very likely bad practice. RSS is provided as single file with multiple pieces of content. So is XML sitemap.
But getting content will require requesting multiple pages and generate much higher load on target site (which owner might be less than happy about).
For a service that is bit more suited for getting actual content see post on Dapper http://www.rarst.net/web/dapper/
Hi, I tried your pipe but i only get
No pipe results for Sitemap to RSS.
And my site has a xml sitemap… dont understand the error. Thanks for any help you can provide.
@fher98
I fixed a bug there just now – Pipes had been moved to new version and it was likely caused by that. Should work in a bit once caches clear.
btw I am not sure why you want this for your site, you seem to be running WordPress and it does feeds pretty good. :) Or just playing?