feedparser
feedparser gem - web feed parser and normalizer (Atom, RSS, JSON Feed, HTML h-entry, etc.)
What's News?
October/2017: Added support for attachments / media enclosures in RSS and Atom.
June/2017: Added support for reading feeds in HTML with Microformats incl.
h-entry
,
h-feed
and others.
All feed with test assertions for easy (re)use and browsing moved
to its own repo, that is, /feeds
.
May/2017: Added support for reading feeds in the new JSON Feed format in - surprise, surprise - JSON.
What's a Web Feed?
See the Awesome Feeds page ».
Usage
Structs
Feed • Item • Author • Tag • Attachment • Generator

Feed
Struct
Mappings
Note: uses question mark (?
) for optional elements (otherwise assume required elements)
Title 'n' Summary
Note: The Feed parser will remove all html tags and attributes from the title (RSS 2.0+Atom),
description (RSS 2.0) and subtitle (Atom) content and will unescape HTML entities e.g. &
becomes & and so on - always
resulting in plain vanilla text.
feed.title | title | plain text | title | plain text | title | plain text |
feed.summary | description | plain text | subtitle ? | plain text | description ? | plain text |
Dates
feed.updated | lastBuildDate ? | RFC-822 format | updated | ISO 801 format |
feed.published | pubDate ? | RFC-822 format | - | |
Note: Check - for RSS 2.0 set feed.updated to pubDate or lastBuildDate if only one present? if both present - map as above.
RFC-822 date format e.g. Wed, 14 Jan 2015 19:48:57 +0100
ISO-801 date format e.g. 2015-01-11T09:30:16Z
class Feed
attr_accessor :format
attr_accessor :title
attr_accessor :url
attr_accessor :items
attr_accessor :summary
attr_accessor :updated
attr_accessor :published
attr_accessor :authors
attr_accessor :tags
attr_accessor :generator
end
(Source: lib/feedparser/feed.rb
)
Item
Struct
Title 'n' Summary
Note: The Feed parser will remove all html tags and attributes from the title (RSS 2.0+Atom),
description (RSS 2.0) and summary (Atom) content
and will unescape HTML entities e.g. &
becomes & and so on - always
resulting in plain vanilla text.
Note: In plain vanilla RSS 2.0 there's no difference between (full) content and summary - everything is wrapped
in a description element; however, best practice is using the content "module" from RSS 1.0 inside RSS 2.0.
If there's no content module present the feed parser will "clone" the description and use one version for item.summary
and
the clone for item.content
.
Note: In JSON Feed the title is not required (that is, is optional). The idea is to support "modern" micro blog postings (tweets, toots, etc.) that have no titles. The JSON Feed supports content_html
and/or content_text
. At least one version must be present.
Note: The content element will assume html content.
item.title | title | plain text | title | plain text | title ? | plain text |
item.summary | description | plain text | summary ? | plain text | -tbd- | |
item.content | content ? | html | content ? | html | content_html (*) | html |
Dates
item.updated | pubDate ? | RFC-822 format | updated | ISO 801 format |
item.published | - | RFC-822 format | published ? | ISO 801 format |
Note: In plain vanilla RSS 2.0 there's only one pubDate
for items, thus, it's not possible to differeniate between published and updated dates for items; note - the item.pubDate
will get mapped to item.updated
. To set the published date in RSS 2.0 use the dublin core module e.g dc:created
, for example.
class Item
attr_accessor :title
attr_accessor :url
attr_accessor :content
attr_accessor :content_type
attr_accessor :summary
attr_accessor :updated
attr_accessor :published
attr_accessor :guid
end
(Source: lib/feedparser/item.rb
)
Author
Struct
(Source: lib/feedparser/author.rb
)
Tag
Struct
(Source: lib/feedparser/tag.rb
)
Attachment
Struct
Also known as Media Enclosure
(Source: lib/feedparser/attachment.rb
)
Generator
Struct
(Source: lib/feedparser/generator.rb
)
Read Feed Example
require 'open-uri'
require 'feedparser'
txt = open( 'http://openfootball.github.io/feed.xml' ).read
feed = FeedParser::Parser.parse( txt )
puts feed.title
puts feed.url
puts feed.items[0].title
puts feed.items[0].url
puts feed.items[0].updated
puts feed.items[0].content
...
or reading a feed in the new JSON Feed format in - surprise, surprise - JSON;
note: nothing changes :-)
txt = open( 'http://openfootball.github.io/feed.json' ).read
feed = FeedParser::Parser.parse( txt )
puts feed.title
puts feed.url
puts feed.items[0].title
puts feed.items[0].url
puts feed.items[0].updated
puts feed.items[0].content_text
...
Microformats
Microformats let you mark up feeds and posts in HTML with
h-entry
,
h-feed
,
and friends.
Note: Microformats support in feedparser is optional.
Install and require the the microformats gem to read
feeds in HTML with Microformats.
require 'microformats'
text =<<HTML
<article class="h-entry">
<h1 class="p-name">Microformats are amazing</h1>
<p>Published by
<a class="p-author h-card" href="http://example.com">W. Developer</a>
on <time class="dt-published" datetime="2013-06-13 12:00:00">13<sup>th</sup> June 2013</time>
<p class="p-summary">In which I extoll the virtues of using microformats.</p>
<div class="e-content">
<p>Blah blah blah</p>
</div>
</article>
HTML
feed = FeedParser::Parser.parse( text )
puts feed.format
puts feed.items.size
puts feed.items[0].authors.size
puts feed.items[0].content_html
puts feed.items[0].content_text
puts feed.items[0].title
puts feed.items[0].summary
puts feed.items[0].published
puts feed.items[0].authors[0].name
...
Samples
Feed Reader
Planet Feed Reader in 20 Lines of Ruby
planet.rb
:
require 'open-uri'
require 'feedparser'
require 'erb'
FEED_URLS = [
'http://vienna-rb.at/atom.xml',
'http://weblog.rubyonrails.org/feed/atom.xml',
'http://www.ruby-lang.org/en/feeds/news.rss',
'http://openfootball.github.io/feed.json',
]
items = []
FEED_URLS.each do |url|
feed = FeedParser::Parser.parse( open( url ).read )
items += feed.items
end
FEED_ITEM_TEMPLATE = <<EOS
<% items.each do |item| %>
<div class="item">
<h2><a href="<%= item.url %>"><%= item.title %></a></h2>
<div><%= item.content %></div>
</div>
<% end %>
EOS
puts ERB.new( FEED_ITEM_TEMPLATE ).result
Run the script:
$ ruby ./planet.rb
Prints:
<div class="item">
<h2><a href="http://vienna-rb.at/blog/2017/11/06/picks/">Picks / what the vienna.rb team thinks is worth sharing this week</a></h2>
<div>
<h3>6/11 Picks!!</h3>
<p>In a series on this website we'll entertain YOU with our picks...
...
Real World Usage
See the Planet Pluto feed reader family:
Add your tools, scripts, apps here! Let us know.
Install
Just install the gem:
$ gem install feedparser
License

The feedparser
scripts are dedicated to the public domain.
Use it as you please with no restrictions whatsoever.
Send them along to the wwwmake Forum/Mailing List.
Thanks!