notion-news-crawler

Notion-News-Crawler

Notion News Crawler is a Python library that collects news from Notion and uploads it to a database.

News can be collected using Naver Search API, so only Naver and Notion API keys can be used by putting them in environment variables.

Table of Contents

1. Install
2. Supported Python Version
3. APIs
   • Environment Parameters
   • Notion API
   • Naver API
4. Result

Install

$ pip3 install notion-news-crawler

Supported Python version

Library	Supported Version
Notion News Crawler	3.8, 3.9, 3.10, 3.11, 3.12

Quick Start

from notion_news_crawler import NaverAPI, UploadToDataBase, ResetDatabase


if __name__=='__main__':
    reset_database = ResetDatabase()
    reset_database.delete_all_pages()

    for subject in ['Economy', 'Science', 'Society', 'Politics', 'Stock']:
        naver_api = NaverAPI(subject, 100)
        news_data = naver_api.parse_data(naver_api.get_news())

        upload_to_database = UploadToDataBase(news_data)
        upload_to_database.add_to_notion()

APIs

To use this library, Notion and Naver API are required.

Just enter the two API Key values ​​in the environment variables.

Environment parameters

You can create an environment variable .env file and use it by entering the following.

.env file format

# Notion
NOTION_TOKEN='YOUR NOTION TOKEN'
NOTION_DATABASE_ID='YOUR NOTION DATABASE ID'

# Naver
X_NAVER_CLIENT_ID='YOUR NAVER CLIENT ID'
X_NAVER_SECRET='YOUR NAVER SECRET KEY'

Notion API

The Notion API can be found at: Notion API
Enter the Notion application token value in the environment variable. Create a Notion application, register the Notion application in the database you will use, and then use it.

Naver API

You can create a Naver application and obtain the API key value.
More detail on NAVER API docs

The Naver API result values ​​are as follows, and when uploading to Notion, you can refine and use the data as you wish.

{
    "title": "article title",
    "originallink": "original news link",
    "link": "naver news link",
    "description": "article description",
    "pubDate": "published date"
}