rss2yaml

rss2yaml

Standalone command-line tool to fetch RSS/Atom feeds and output structured YAML.

Install

Global (permanent, available everywhere):

npm install -g rss2yaml

Run without installing (via npx):

npx -y rss2yaml --urls "https://techcrunch.com/feed/"

From source (clone the repo):

git clone https://github.com/YOUR_USER/rss2yaml.git
cd rss2yaml
npm install
node rss2yaml.js --urls "https://hnrss.org/frontpage" --limit 5

Usage

npx -y rss2yaml --urls "https://techcrunch.com/feed/"

rss2yaml --urls "https://techcrunch.com/feed/" --fields title,description,url,date --limit 10 --sort newest

One-liner examples (copy-paste friendly)

Fetch Reddit AI news, save to file:

rss2yaml --urls "https://www.reddit.com/search.rss?q=ai+models+news&sort=hot" --fields title,description,url,date --limit 10 --sort newest --out feed-news.yaml

Multiple feeds, filter by keyword:

rss2yaml --urls "https://techcrunch.com/feed/,https://hnrss.org/frontpage" --fields title,url,date --limit 5 --sort newest --filter "openai"

Print to stdout (pipe to another tool):

rss2yaml --urls "https://hnrss.org/frontpage" --fields title,url --limit 20 --no-dedupe

Google News

Google News RSS feeds return redirect URLs instead of the real article links. rss2yaml automatically decodes them to the original website URL using google-news-url-decoder.

rss2yaml --urls "https://news.google.com/rss/search?q=openai" --fields title,url,date --limit 5

Output will contain the real article URLs (e.g. https://techcrunch.com/...) instead of https://news.google.com/rss/articles/... redirects.

Google News search tips:

Pattern	Example
Simple keyword	https://news.google.com/rss/search?q=openai
Multiple keywords	https://news.google.com/rss/search?q=ai+openai
Restrict to a site	https://news.google.com/rss/search?q=ai+openai+site:youtube.com
Exclude a site	https://news.google.com/rss/search?q=openai+-site:reddit.com
Exact phrase	https://news.google.com/rss/search?q="artificial+intelligence"
Language/region (France)	https://news.google.com/rss/search?q=openai&hl=fr&gl=FR&ceid=FR:fr
Language/region (Canada, French)	https://news.google.com/rss/search?q=openai&hl=fr&gl=CA&ceid=CA:fr
Language/region (Belgium, French)	https://news.google.com/rss/search?q=openai&hl=fr&gl=BE&ceid=BE:fr

hl sets the language, gl sets the country, and ceid combines them as COUNTRY:language — include all three for consistent results (Google sometimes mixes languages if ceid is missing). Without these params, Google News defaults to hl=en-US&gl=US&ceid=US:en (US English).

Reddit

Reddit exposes RSS feeds for searches, subreddits, and user profiles.

rss2yaml --urls "https://www.reddit.com/search.rss?q=ai+openai&sort=hot" --limit 10

Rate limit: Reddit allows roughly 1 RSS request per minute from the same IP. If you get HTTP 429 errors, wait a minute before retrying. The built-in cache (5 min) helps avoid repeated hits.

Pattern	Example
Search	https://www.reddit.com/search.rss?q=ai+models+news&sort=hot
Subreddit (hot)	https://www.reddit.com/r/ClaudeCode/hot/.rss
Subreddit (top today)	https://www.reddit.com/r/ClaudeCode/top/.rss?t=day
Subreddit (new)	https://www.reddit.com/r/ClaudeCode/new/.rss

Filter tips

The --filter flag does a simple substring match (case-insensitive). For short keywords like "AI", add spaces around it to avoid matching words like "email", "daily", "train", etc.:

# Bad — matches "email", "daily", "omain", etc.
rss2yaml --urls "https://hnrss.org/frontpage" --filter "ai"

# Good — matches only " AI " as a standalone word
rss2yaml --urls "https://hnrss.org/frontpage" --filter " AI "

Find - Discovery

Don't know the feed URL for a website? --find discovers RSS/Atom feeds automatically, so you don't have to dig through a site's source code by hand.

rss2yaml --find "https://techcrunch.com" --limit 3

How it works

Discovery runs through several passes and stops once it has verified feeds:

• rss-url-finder package — parses the page and checks for known RSS MIME types.
• HTML <head> scan — looks for <link rel="alternate" type="application/rss+xml"> (and Atom/RDF equivalents).
• HTML <body> scan — looks for <a> tags pointing to feed-like URLs or labeled "RSS", "Atom feed", "flux RSS", etc.
• Common path probing — tries well-known paths on the domain: /rss, /rss/, /feed, /feeds, /rss-feeds, /syndication.
• Verification pass — every candidate URL found above is fetched and checked for real RSS/Atom XML content before being included in the result. Non-feed URLs (sitemaps, manifests, favicons, images, etc.) are filtered out.

Special cases:

• Reddit URLs are detected and short-circuited straight to /r/<sub>/hot/.rss, /top/.rss, /new/.rss (or /user/<name>/.rss) without scanning HTML.
• Google redirect URLs (google.com/url?q=...) are unwrapped before scanning.

Output

JSON (default) — an object with the resolved URL, the list of feeds, and a count:

rss2yaml --find "https://techcrunch.com"

{
  "url": "https://techcrunch.com/",
  "feeds": [
    "https://techcrunch.com/feed/"
  ],
  "count": 1
}

Plain text — one feed URL per line, handy for piping straight into --urls:

rss2yaml --find "https://techcrunch.com" --format plain

https://techcrunch.com/feed/

Limit and output file

--limit with --find caps the number of feeds returned, not items per feed (that only applies in normal fetch mode without --find):

rss2yaml --find "https://techcrunch.com" --limit 3

Write the result to a file instead of stdout:

rss2yaml --find "https://techcrunch.com" --format plain --out feeds.txt

Chaining find → fetch

Since --format plain prints one URL per line, you can discover feeds and feed them straight into a normal fetch (shell example, adjust for your shell):

FEEDS=$(rss2yaml --find "https://techcrunch.com" --format plain | tr '\n' ',')
rss2yaml --urls "$FEEDS" --limit 5

Limitations

• Discovery does network requests per candidate URL (page fetch + common paths + verification), so it's slower than fetching a known feed directly.
• Some sites block scraping or bot User-Agents, which can result in zero feeds found even if a feed exists — in that case, use the site's documented RSS URL directly.
• If no valid feed is found, the command exits with an error message and a non-zero exit code.

Flags

Flag	Description	Default
--urls, --url	Comma-separated feed URLs (required, unless --find is used)	—
--find <url>	Discover RSS/Atom feeds on a website instead of fetching items	—
--format	Output format for --find: json, plain	json
--limit	With --find: max feeds returned. Otherwise: max items per feed	no limit (find) / 10 (fetch)
--fields	Fields to extract: title, description, url, date, author	title,description,url,date
--sort	Sort order: newest, oldest, none	newest
--dedupe	Dedup strategy: url, title, url_or_title, none	url_or_title
--no-dedupe	Disable deduplication	dedupe is ON
--filter	Keep only items matching this keyword	off
--keep-html	Keep HTML in text fields	HTML stripped
--json	Output JSON instead of YAML	YAML
--cache <sec>	Cache TTL in seconds	300 (5 min)
--no-cache	Skip cache, force fresh fetch	cache is ON
--out	Write output to file (omit to print to stdout)	stdout
--help	Show help message	—

Output format

total_items: 5
items:
  - source_feed: TechCrunch
    title: Some Article
    description: Brief summary here
    url: https://example.com/article
    date: '2025-07-30T12:00:00.000Z'

Notes

• Works with node (v18+) or bun.
• Google News URLs are automatically decoded to the real article URL (via google-news-url-decoder).
• Reddit feeds use a specific User-Agent to avoid rate limiting.
• HTML tags are stripped from text fields by default. Use --keep-html to preserve them.
• Feeds are cached locally (.rss-cache/) for 5 minutes by default. Use --no-cache to force a fresh fetch, or --cache 600 to change the TTL.