datahtml
datahtml is a library for crawling and extraction of data from html and xml content.
Datahtml lets you:
- Extract ld+json data from html
- Extract frequently used meta tags from html (those that are used for SEO and social media, between others)
- Extract Article data from a html, usually from Newspaper sites
- Parse RSS feeds from sites
- Crawl some specific social media sites like google and youtube
Under the hood datahtml uses libraries like BeautifoulSoup, Newspaper2k, feedparser between others, but
datahtml takes an opinionated approach for crawling based on our expriencies doing so.
Quickstart
pip install datahtml
from datahtml import web, crawler
c = crawler.LocalCrawler()
w = web.download("https://www.infobae.com", crawler=c)
w.links()
License
datahtml
is distributed under the terms of the MPL-2.0 license.