FinPull - Financial Data Scraper

FinPull is a comprehensive financial data scraping tool providing multiple interfaces for accessing financial market data. The package includes API, command-line, and graphical user interfaces, making it suitable for various use cases from automated trading systems to interactive data analysis.
Installation
pip install finpull
For API-only usage (lightweight):
pip install finpull-core
Quick Start
GUI Mode (Default)
finpull
Command Line Interface
finpull --interactive
finpull add AAPL GOOGL MSFT
finpull show AAPL --full
finpull export portfolio.xlsx --xlsx
finpull refresh
Programmatic API
from finpull import FinancialDataAPI
api = FinancialDataAPI()
result = api.add_ticker("AAPL")
data = api.get_data("AAPL")
if data['success']:
stock_info = data['data']
print(f"Company: {stock_info['company_name']}")
print(f"Price: ${stock_info['price']}")
print(f"Market Cap: {stock_info['market_cap']}")
Performance
Comprehensive benchmarks on typical hardware with 10 test runs each:
Package Size | 21.9 KB | 27.2 KB | +5.3 KB (+24.2%) |
Installed Size | 134 KB | 188 KB | +54 KB (+40.3%) |
Import Time (cached) | 0.0002s | 0.0002s | No difference |
Dependencies | 3 packages | 4 packages | +openpyxl |
Key Features
- Multiple Interfaces: GUI, CLI, and API access
- Export Options: JSON, CSV, and Excel formats
- Progress Tracking: Real-time operation status
- Cross-Platform: Windows, macOS, and Linux support
Interfaces
Graphical User Interface (GUI)
Launch: finpull
or finpull --gui
Features:
- Data Grid: Complete view of all 27 financial metrics with horizontal/vertical scrolling
- Multi-Selection: Select and manage multiple tickers simultaneously (Ctrl+Click, Shift+Click)
- Smart Sorting: Click column headers to sort by any metric with data type awareness
- Real-time Updates: Progress indicators showing 🔄 loading, ✅ success, ❌ error states
- Export Dialog: Save data to JSON, CSV, or Excel with file browser integration
- Status Bar: Current operation status and ticker count display
- Responsive Layout: Adapts to different screen sizes and resolutions
Command Line Interface (CLI)
Launch: finpull --interactive
or direct commands
Available Commands:
add <tickers>
- Add ticker symbols for tracking
remove <tickers>
- Remove tickers from tracking
show [ticker] [--full]
- Display ticker information
refresh [ticker]
- Update data from sources
export <filename> [--json] [--csv] [--xlsx]
- Save data to files
stats
- Show system statistics and health
clear
- Remove all tracked data (with confirmation)
Features:
- Interactive Mode: Shell-like interface (
finpull>
prompt) for exploration
- Direct Commands: Single-command operations perfect for scripting and automation
- Batch Operations: Process multiple tickers efficiently in one command
- Formatted Output: Beautiful ASCII tables with aligned columns and borders
- Progress Indicators: Real-time status updates for long-running operations
- Auto-completion: Tab completion for commands and options (in interactive mode)
API Interface
Import: from finpull import FinancialDataAPI, FinancialDataScraper
Classes:
- FinancialDataAPI: High-level interface with error handling and validation
- FinancialDataScraper: Low-level scraper for direct data access and control
- FinancialData: Data model with 27+ financial attributes
Features:
- Consistent Responses: All methods return standardized JSON format
- Comprehensive Error Handling: Detailed error codes and descriptive messages
- Type Hints: Full type annotation support for better IDE integration
- Validation: Built-in ticker format validation and data sanitization
- Callback Support: Progress callbacks for batch operations and real-time updates
Data Coverage
Provides 27 financial metrics per ticker across all categories: basic info, valuation ratios, earnings, profitability, growth, financial position, and market data. Uses Finviz and Yahoo Finance with automatic failover for high reliability.
Configuration
Environment Variables
export FINPULL_STORAGE_FILE="/path/to/custom/storage.json"
export FINPULL_RATE_LIMIT="2"
export FINPULL_LOG_LEVEL="INFO"
export FINPULL_GUI_THEME="default"
Storage
Data is persisted locally in JSON format with automatic backups:
- Linux/macOS:
~/.finpull/data.json
- Windows:
%USERPROFILE%\.finpull\data.json
- Backup: Automatic backup before major operations
- Format: Human-readable JSON with proper indentation
Rate Limiting
Built-in intelligent rate limiting prevents API blocks:
- Default: 1 request per second (configurable)
- Adaptive: Automatically increases delays if rate limits detected
- Burst Protection: Prevents accidental rapid-fire requests
- Source-Specific: Different limits for different data sources
Documentation
Web Integration
Supports browser integration via Pyodide and Node.js via CLI commands. See main repository README for complete integration examples and code samples.
Package Architecture
finpull/
├── Core Package (finpull-core)
│ ├── API interface
│ ├── Data scraping engine
│ ├── Storage management
│ └── Utility functions
└── Interface Extensions
├── Command-line interface
├── Graphical user interface
└── Excel export functionality
The full package depends on finpull-core
for core functionality, ensuring:
- No code duplication between packages
- Consistent API across all interfaces
- Modular installation options
- Streamlined maintenance and updates
Examples
Portfolio Management
from finpull import FinancialDataAPI
api = FinancialDataAPI()
tech_stocks = ["AAPL", "GOOGL", "MSFT", "AMZN", "TSLA", "META", "NVDA"]
results = api.batch_add_tickers(tech_stocks)
print(f"Successfully added {results['summary']['added_count']} stocks")
portfolio_data = api.get_data()
for stock in portfolio_data['data']:
print(f"{stock['ticker']}: ${stock['price']} | P/E: {stock['pe_ratio']} | Cap: {stock['market_cap']}")
Automated CLI Workflow
#!/bin/bash
echo "Updating portfolio..."
finpull add AAPL GOOGL MSFT
finpull refresh
finpull export "reports/portfolio_$(date +%Y%m%d)" --json --csv --xlsx
finpull show --full
echo "Portfolio update complete"
Real-time Monitoring
import time
from finpull import FinancialDataAPI
api = FinancialDataAPI()
watchlist = ["AAPL", "GOOGL", "MSFT", "TSLA"]
api.batch_add_tickers(watchlist)
while True:
api.refresh_data()
data = api.get_data()
for stock in data['data']:
change_5y = float(stock['change_5y'].replace('%', ''))
if abs(change_5y) > 10:
print(f"Alert: {stock['ticker']} changed {change_5y}% over 5 years")
time.sleep(300)
GUI Automation
import threading
from finpull import FinancialDataGUI
def setup_automated_gui():
gui = FinancialDataGUI()
gui.scraper.add_ticker("AAPL")
gui.scraper.add_ticker("GOOGL")
gui.refresh_display()
gui_thread = threading.Thread(target=gui.run)
gui_thread.start()
return gui
gui = setup_automated_gui()
Error Handling & Logging
import logging
from finpull import FinancialDataAPI
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)
api = FinancialDataAPI()
def safe_operation(ticker):
try:
result = api.add_ticker(ticker)
if result['success']:
logger.info(f"Successfully added {ticker}")
return True
else:
logger.warning(f"Failed to add {ticker}: {result.get('error')}")
return False
except Exception as e:
logger.error(f"Exception for {ticker}: {e}")
return False
tickers = ["AAPL", "INVALID", "GOOGL", "MSFT"]
successful = [t for t in tickers if safe_operation(t)]
print(f"Successfully processed {len(successful)}/{len(tickers)} tickers")
License
MIT License - see LICENSE file for details.
Links