Useful tool to scrape information from twitter users and tweets.
*In order to use this scraper you need an active twitter account. Scraper will sign in and use session to scrape data
*A {username}.json file with AUTH cookies will be saved in your system TMP folder
Features
- Scraper followers, tweets, favorites from a twitter username(public profile only)
- Scrape retweeted by, liked by, comments from a tweet id
- Results can be saved in to a CSV and JSON files
Followers List
Retweeted By Liest
Limits:
- retweeted by - 97 inputes (web api limit).
- liked by - 100 inputes (web api limit).
- followers, comments, tweets, favorites - unlimited
Possible errors
- If there will be let me know
Installation
Install from NPM
$ npm i -g twitter-buddy
Install from YARN
$ yarn global add twitter-buddy
USAGE
Terminal
$ twitter-buddy --help
Usage: twitter-buddy <command> [options]
Commands:
twitter-buddy retweetedBy scrape retweetedBy data from a tweet. Maximum you can scrape 97 inputs
twitter-buddy likedBy scrape likedBy data from a tweet. Maximum you can scrape 100 inputs
twitter-buddy conversation scrape comments data from a tweet. Maximum you can scrape Unlimited inputs
twitter-buddy followers scrape followers data from a twitter username. Maximum you can scrape Unlimited inputs
twitter-buddy tweets scrape tweets data from a twitter username. Maximum you can scrape Unlimited inputs
twitter-buddy favorites scrape favorites data from a twitter username. Maximum you can scrape Unlimited inputs
Options:
--help, -h help [boolean]
--version Show version number [boolean]
--username, -u Your twitter username. In order to use this scraper you need
to sign in by using your twitter account. This needs to be
done only once [string] [default: ""]
--password, -p Your twitter password. In order to use this scraper you need
to sign in by using your twitter account. This needs to be
done only once [string] [default: ""]
--input, -i Scrape from. For example if you need to scrape followers from
'twitter' then enter 'twitter'. If you need retweetedBy then
enter tweet id '21312312321312313' [string] [default: ""]
--number, -n Number of inputs to scrape [default: 20]
--csv Save to a CSV file? [boolean] [default: true]
--json Save to a JSON file? [boolean] [default: false]
Examples:
twitter-buddy followers -u TWITTER_USERNAME -p TWITTER_PASSWORD -input TWITTER_USERNAME -n 100
twitter-buddy retweetedBy -u TWITTER_USERNAME -p TWITTER_PASSWORD -input TWEET_ID -n 100
Example 1
Scrape 1000 followers from twitter profile 'realDonaldTrump' and save everything to a CSV file
$ twitter-buddy followers -u TWITTER_USERNAME -p TWITTER_PASSWORD -input realDonaldTrump -n 1000
The file will be saved in a folder from which you run the script:
folowers:1552945544582:realDonaldTrump.csv
Example 2
Scrape 1000 tweets from twitter profile 'realDonaldTrump'.
$ twitter-buddy tweets -u TWITTER_USERNAME -p TWITTER_PASSWORD -input realDonaldTrump -n 1000
The file will be saved in a folder from which you run the script:
tweets:1552945544582:realDonaldTrump.csv
Example 3
Scrape 100 users who retweeted this tweet 'https://twitter.com/realDonaldTrump/status/1183796209942695938'.
$ twitter-buddy retweetedBy -u TWITTER_USERNAME -p TWITTER_PASSWORD -input 1183796209942695938 -n 100
The file will be saved in a folder from which you run the script:
retweetedBy:1552945544582:1183796209942695938.csv
Module
const twitterScraper = require('twitter-buddy');
(async() => {
try{
let bot = new twitterScraper({
username:'YOUR_TWITTER_USERNAME',
password:'YOUR_TWITTER_PASSWORD',
//session_path: 'CUSTOM_SESSION_PATH',
//proxy: 'HTTPS PROXY'
});
// Scrape 1000 followers from twitter profile 'realDonaldTrump'
let followers = await bot._input({
type: 'scrape',
action: 'followers',
input: 'realDonaldTrump',
number: 1000,
});
// Scrape 500 tweets from twitter profile 'realDonaldTrump'
let tweets = await bot._input({
type: 'scrape',
action: 'tweets',
input: 'realDonaldTrump',
number: 500,
});
// Scrape 500 favorites from twitter profile 'realDonaldTrump'
let favorites = await bot._input({
type: 'scrape',
action: 'favorites',
input: 'realDonaldTrump',
number: 500,
});
// Scrape 300 comments from tweet https://twitter.com/realDonaldTrump/status/1183796209942695938'
let favorites = await bot._input({
type: 'scrape',
action: 'conversation',
input: '1183796209942695938',
number: 300,
});
// Scrape 100 users who retweeted tweet https://twitter.com/realDonaldTrump/status/1183796209942695938'
let favorites = await bot._input({
type: 'scrape',
action: 'retweetedBy',
input: '1183796209942695938',
number: 300,
});
// Scrape 100 users who liked tweet https://twitter.com/realDonaldTrump/status/1183796209942695938'
let favorites = await bot._input({
type: 'scrape',
action: 'likedBy',
input: '1183796209942695938',
number: 300,
});
}catch(error){
console.log(error);
}
})()
License
MIT
Free Software