pafer

Features

• Parse HTML into structured tag / text / metadata results
• CSS-style selectors (#id, .class, tag name)
• Whitelist-based sanitization
• Minification with comment stripping
• Link and table extraction
• LRU result caching with TTL
• RSS 2.0 feed parsing
• JSON / CSV export
• CLI tool
• Standard library only — no third-party dependencies

Installation

pip install pafer

Quickstart

from pafer import HTMLPafer

html = '<div class="post"><h1>Title</h1><a href="/about">About</a></div>'

p = HTMLPafer()
result = p.parse(html)

result['tags']        # list of {name, attributes, content, rows?}
result['metadata']    # {'title': ...}
result['doctype']     # '<!DOCTYPE ...>' or None

p.select(html, '.post')    # match by #id, .class or tag name
p.sanitize(html)           # keep only allowed tags
p.minify(html)             # collapse whitespace, strip comments
p.extract_links(html)      # [{'url': ..., 'text': ..., 'attributes': ...}]

Enable the parsed-result cache with HTMLPafer(use_cache=True).

Configuration

load_config(path) loads an exact file and raises ConfigLoadError (or a subclass) when it is missing, invalid, or corrupted. Omitting path auto-finds the caller's default.config — module directory first, then the package directory; find_default_config() exposes that lookup directly. Config() (or Config.load_default()) uses built-in defaults without touching the filesystem. save_config auto-finds its target the same way when path is omitted.

File format

raw:   [8-byte MD5 checksum][4-byte size][pickle payload]
gzip:  [8-byte MD5 checksum][4-byte size][gzip.compress(pickle payload)]
xz:    [8-byte MD5 checksum][4-byte size][lzma.compress(pickle payload)]
zlib:  [8-byte MD5 checksum][4-byte size][zlib.compress(pickle payload)]

from pafer import load_config, save_config

settings = {'strip_whitespace': False, 'cache_size': 100}
save_config(settings, 'my.config')                   # raw
save_config(settings, 'my.config.gz', compression='gzip')
save_config(settings, 'my.config.xz', compression='xz')
cfg = load_config('my.config.gz')                    # format auto-detected

Default settings

Key	Default	Purpose
parse_tables	True	extract table rows while parsing
strip_whitespace	True	collapse whitespace
encoding	utf-8	decode bytes input
max_depth	100	maximum element nesting
default_tag	div	fallback tag
preserve_comments	False	keep comments in the tree
strict_mode	False	raise on malformed HTML
remove_comments	True	strip comments when minifying
allowed_tags	p, br, b, i, a, ul, ol, li, div, span	sanitize whitelist
max_file_size_mb	10	reject oversized input
cache_size	50	cache capacity

Caching

pafer.cache.LRUCache is an in-memory LRU cache with TTL expiry:

from pafer.cache import LRUCache

cache = LRUCache(max_size=100, ttl_seconds=300)
cache.set('key', value)
cache.get('key')

RSS feeds

from pafer.feed import FeedPafer

items = FeedPafer().parse_rss(xml_content)
items[0]['title']       # 'title', 'link', 'description', 'pubDate'

Exporting

from pafer.exporters import DataExporter

DataExporter.to_json(data)    # JSON string with 2-space indent
DataExporter.to_csv(rows)     # CSV from a list of rows

Exceptions

HTMLPaferError (base), SanitizationError, SelectorSyntaxError, ConfigLoadError.

LICENSE

MIT