techcrunch-api
Advanced tools
Comparing version 1.0.1 to 1.0.2
{ | ||
"name": "techcrunch-api", | ||
"version": "1.0.1", | ||
"description": "TechCrunch Scraper is a powerful Node.js package designed to facilitate the extraction of article data from TechCrunch based on specific categories or tags", | ||
"version": "1.0.2", | ||
"description": "TechCrunch API is a powerful Node.js package designed to facilitate the extraction of article data from TechCrunch based on specific categories or tags", | ||
"main": "index.js", | ||
@@ -6,0 +6,0 @@ "scripts": { |
@@ -1,19 +0,19 @@ | ||
# TechCrunch Scraper | ||
# TechCrunch API 🧑💻 | ||
TechCrunch Scraper is a Node.js package that allows you to scrape articles from TechCrunch based on categories or tags. This package is designed for systems using Ubuntu or other Debian-based distributions that support `sudo` commands, leveraging Puppeteer to navigate and scrape content from a headless Chromium environment. | ||
TechCrunch API is a Node.js package that allows you to scrape articles from TechCrunch based on categories or tags. This package is designed for systems using Ubuntu or other Debian-based distributions that support `sudo` commands, leveraging Puppeteer to navigate and scrape content from a headless Chromium environment. 🌐 | ||
## Features | ||
## Features 🚀 | ||
- **Scrape by Category:** Automatically retrieve all articles under a specified category. | ||
- **Scrape by Tag:** Collect articles that are tagged with a specific keyword. | ||
- **Headless Browser Support:** Runs Chromium in headless mode to scrape dynamic content. | ||
- **Optimized for Ubuntu:** Includes installation instructions specifically for Ubuntu, but compatible with other Linux distributions. | ||
- **Scrape by Category:** Automatically retrieve all articles under a specified category. 📂 | ||
- **Scrape by Tag:** Collect articles that are tagged with a specific keyword. 🏷️ | ||
- **Headless Browser Support:** Runs Chromium in headless mode to scrape dynamic content. 👻 | ||
- **Optimized for Ubuntu:** Includes installation instructions specifically for Ubuntu, but compatible with other Linux distributions. 🐧 | ||
## Prerequisites | ||
## Prerequisites 📋 | ||
Before installing the TechCrunch Scraper, you need to ensure your system has the following dependencies installed: | ||
- Node.js (Version 14 or later recommended) | ||
- Puppeteer | ||
- Dependencies required for Puppeteer and headless Chromium | ||
- Node.js (Version 14 or later recommended) 🟢 | ||
- Puppeteer 🎭 | ||
- Dependencies required for Puppeteer and headless Chromium 🔧 | ||
@@ -36,3 +36,2 @@ ## Installation | ||
### Step 2: Install TechCrunch API Package | ||
@@ -47,17 +46,18 @@ | ||
## Usage | ||
#### After installation, you can use the package in your Node.js scripts as follows: | ||
# ES6 Syntax | ||
```javascript | ||
import { getByCategory, getByTag } from "techcrunch-api"; | ||
import { getByCategory, getByTag } from 'techcrunch-api'; | ||
// Fetch articles by category | ||
getByCategory('artificial-intelligence').then(articles => { | ||
console.log(articles); | ||
getByCategory("artificial-intelligence").then((articles) => { | ||
console.log(articles); | ||
}); | ||
// Fetch articles by tag | ||
getByTag('startups').then(articles => { | ||
console.log(articles); | ||
getByTag("startups").then((articles) => { | ||
console.log(articles); | ||
}); | ||
@@ -67,14 +67,16 @@ ``` | ||
# ES5 Syntax | ||
```javascript | ||
const techCrunchAPI = require("techcrunch-api"); | ||
const techCrunchAPI = require('techcrunch-api'); | ||
// Fetch articles by category | ||
techCrunchAPI.getByCategory('artificial-intelligence').then(function(articles) { | ||
techCrunchAPI | ||
.getByCategory("artificial-intelligence") | ||
.then(function (articles) { | ||
console.log(articles); | ||
}); | ||
}); | ||
// Fetch articles by tag | ||
techCrunchAPI.getByTag('startups').then(function(articles) { | ||
console.log(articles); | ||
techCrunchAPI.getByTag("startups").then(function (articles) { | ||
console.log(articles); | ||
}); | ||
@@ -84,6 +86,5 @@ ``` | ||
## Running the Scraper | ||
```bash | ||
xvfb-run --auto-servernum --server-args='-screen 0 1024x768x24' node your-script.js | ||
``` | ||
4997
87