apple-news-to-sqlite

apple-news-to-sqlite

Export Apple News Saved Stories to SQLite

Install

pip install apple-news-to-sqlite

Source code

apple-news-to-sqlite

Usage

apple-news-to-sqlite articles.db

apple-news-to-sqlite --dump

Your Terminal app will require full disk access in order to access the saved article database in the Apple News app sandbox.

CLI help

Usage: apple-news-to-sqlite [OPTIONS] [DB_PATH]

  Export your Apple News saved stories/articles to a SQLite database

  Example usage:

      apple-news-to-sqlite articles.db

  This will populate articles.db with an "articles" table containing information
  about your saved articles.

  Notes:

  The contents of the articles themselves are not stored in the database, only
  metadata about the article such as title, author, url, etc.

  The date the article was saved is in GMT.

Options:
  --version       Show the version and exit.
  --dump, --json  Print saved stories to standard output as JSON.
  --all           Process all saved articles; if not specified, only saved
                  articles that have not previously been stored in the database
                  are processed.
  --schema        Create database schema and exit.
  --help          Show this message and exit.

Using apple-news-to-sqlite in your own Python code

get_saved_articles() returns a list of dictionaries, each representing a saved article with the following keys:

* id: str
* date: datetime.datetime
* url: str
* title: str
* description: str
* image: str
* author: str

>>> from apple_news_to_sqlite import get_saved_articles
>>> articles = get_saved_articles()

How it works

Through reverse engineering, it was determined that the Apple News app stores saved articles in a file called reading-list in the following directory:

~/Library/Containers/com.apple.news/Data/Library/Application Support/com.apple.news/com.apple.news.public-com.apple.news.private-production/

This format of this file is unknown but it is a binary file that contains two embedded binary plist files. The first contains an NSKeyedArchiver object which I have not yet inspected. The second bplist contains a list of saved article IDs along with the date they were saved. The article IDs are used to look up the article data on Apple's News site and the article data is extracted with Beautiful Soup.

Testing

The code is at the "it works on my machine" stage of testing. (M1 Mini, macOS Ventura 13.1)

I've also tested this on macOS Catalina 10.15.7 and it appears to work correctly.

If it doesn't work for you, please open an issue!

Contributing

Contributions of all types are welcome! Fork the repo, make a branch, and submit a PR.

See README_DEV.md for developer notes.

Thanks

Thanks to Simon Willison who inspired this project with his excellent "everything-to-sqlite" dogsheep project.

Thanks Simon also for the excellent tools sqlite-utils and Datasette.

Thanks also to Dave Bullock who inspired this project and helped tremendously with the reverse engineering and initial code.

License

MIT License