A Python package for efficient file and directory management, featuring tools for bulk renaming, data handling, and format conversion
filewise is a comprehensive Python toolkit designed to simplify file operations, data manipulation, and scientific data processing. It provides a robust set of tools for file handling, directory management, format conversion, and data analysis, making it an essential utility package for Python developers working with diverse file types and data formats.
File Operations:
Format Converters:
Data Processing:
Automation Scripts:
General Utilities:
Before installing, please ensure the following dependencies are available on your system:
Required Third-Party Libraries:
pip install pandas numpy xarray netcdf4 openpyxl xlsxwriter odfpy
Or via Anaconda (recommended channel: conda-forge):
conda install -c conda-forge pandas numpy xarray netcdf4 openpyxl xlsxwriter odfpy
External Tools (for PDF and email conversion):
# Ubuntu/Debian
sudo apt-get install ghostscript pdftk wkhtmltopdf poppler-utils
# For email conversion
sudo apt-get install libemail-address-xs-perl
Internal Package Dependencies:
pip install paramlib
pip install pygenutils # Core functionality
pip install pygenutils[arrow] # With arrow support (optional)
# Install filewise from PyPI (includes all dependencies)
pip install filewise
Note: The package now includes all dependencies with version constraints, so no additional installation steps are required.
# Clone the repository
git clone https://github.com/EusDancerDev/filewise.git
cd filewise
# Install with development dependencies (includes latest Git versions)
pip install -e .[dev]
# Alternative: Use requirements-dev.txt for explicit Git dependencies
pip install -r requirements-dev.txt
pip install -e .
Benefits of the new approach:
pip install filewise with all dependencies includedTo stay up-to-date with the latest version of this package, simply run:
pip install --upgrade filewise
If you're planning to contribute to the project or work with the source code, follow these setup instructions:
# Clone the repository
git clone https://github.com/EusDancerDev/filewise.git
cd filewise
# Install with development dependencies (includes latest Git versions)
pip install -e .[dev]
Note: The -e flag installs the package in "editable" mode, meaning changes to the source code are immediately reflected without reinstalling. The [dev] flag includes the latest Git versions of interdependent packages.
If you prefer to use the explicit development requirements file:
# Clone the repository
git clone https://github.com/EusDancerDev/filewise.git
cd filewise
# Install development dependencies from requirements-dev.txt
pip install -r requirements-dev.txt
# Install in editable mode
pip install -e .
This approach gives you the latest development versions of all interdependent packages for testing and development.
If you encounter import errors after cloning:
pip install filewise (all dependencies included)pip install -e .[dev] to include development dependenciesTo verify that your installation is working correctly, you can run this quick test:
# Test script to verify installation
try:
import filewise
from pygenutils.arrays_and_lists.data_manipulation import flatten_list
from paramlib.global_parameters import BASIC_OBJECT_TYPES
print("✅ All imports successful!")
print(f"✅ filewise version: {filewise.__version__}")
print("✅ Installation is working correctly.")
except ImportError as e:
print(f"❌ Import error: {e}")
print("💡 For regular users: pip install filewise")
print("💡 For developers: pip install -e .[dev]")
This project implements a dual-approach dependency management system:
pip install filewise with all dependencies includedpip install -e .[dev] for latest Git versions and development toolsfrom filewise.file_operations.path_utils import find_files
from filewise.file_operations.bulk_rename_auto import reorder_objs
# Find all PDF files in a directory
pdf_files = find_files(
patterns="pdf",
search_path="/path/to/documents",
match_type="ext",
top_only=False
)
# Automatically rename files with sequential numbering
reorder_objs(
path="/path/to/documents",
obj_type="file",
extensions2skip="tmp",
starting_number=1,
zero_padding=3
)
from filewise.format_converters.pdf_tools import merge_files, file_compressor
# Merge multiple PDF files
pdf_list = ["document1.pdf", "document2.pdf", "document3.pdf"]
merge_files(
in_path_list=pdf_list,
out_path="merged_document.pdf"
)
# Compress PDF files
file_compressor(
in_path="large_document.pdf",
out_path="compressed_document.pdf"
)
from filewise.pandas_utils.pandas_obj_handler import merge_excel_files, standardise_time_series
from filewise.pandas_utils.data_manipulation import sort_df_values
# Merge multiple Excel files
result = merge_excel_files(
input_file_list=["data1.xlsx", "data2.xlsx"],
output_file_path="merged_data.xlsx",
save_merged_file=True
)
# Standardise time series data
standardised_df = standardise_time_series(
dfs=[df1, df2, df3],
date_value_pairs=[("date", "value1"), ("timestamp", "value2"), ("time", "value3")],
handle_duplicates=True
)
from filewise.xarray_utils.file_utils import scan_ncfiles, ncfile_integrity_status
from filewise.xarray_utils.patterns import get_latlon_bounds
from filewise.xarray_utils.xarray_obj_handler import save2nc
# Scan NetCDF files in directory
file_info = scan_ncfiles("/path/to/netcdf/files")
# Check file integrity
dataset = ncfile_integrity_status("climate_data.nc")
# Extract coordinate bounds
lat_bounds, lon_bounds = get_latlon_bounds(
nc_file="climate_data.nc",
lat_dimension_name="latitude",
lon_dimension_name="longitude"
)
from filewise.json_utils.json_obj_handler import serialise_to_json, deserialise_json_to_df
# Serialise data to JSON with custom formatting
serialise_to_json(
data={"results": [1, 2, 3], "metadata": {"version": "1.0"}},
out_file_path="output.json",
indent=2,
sort_keys=True
)
# Convert JSON to DataFrame
df = deserialise_json_to_df(
json_obj_list=["data1.json", "data2.json"],
orient="records"
)
from filewise.scripts.copy_compress import _execute_copy_compress_workflow
# Execute automated file copy and compression workflow
# (Configure parameters in the script as needed)
_execute_copy_compress_workflow()
The package is organised into specialised sub-packages for different file operations:
filewise/
├── file_operations/
│ ├── bulk_rename_auto.py # Automatic bulk file renaming
│ ├── bulk_rename_manual.py # Manual file renaming templates
│ ├── cat_file_content.py # File content display utilities
│ ├── ops_handler.py # Core file operations (copy, move, sync)
│ ├── path_utils.py # Path searching and pattern matching
│ └── permission_manager.py # File/directory permission management
├── format_converters/
│ └── pdf_tools.py # PDF manipulation and conversion tools
├── pandas_utils/
│ ├── conversions.py # DataFrame format conversions
│ ├── data_manipulation.py # DataFrame operations and analysis
│ └── pandas_obj_handler.py # Excel, CSV, ODS file handling
├── xarray_utils/
│ ├── conversions.py # Climate data format conversion
│ ├── data_manipulation.py # NetCDF data processing
│ ├── file_utils.py # NetCDF file utilities and integrity checks
│ ├── patterns.py # Coordinate and dimension pattern analysis
│ └── xarray_obj_handler.py # NetCDF file creation and manipulation
├── json_utils/
│ ├── json_encoding_operations.py # Custom JSON encoding/decoding
│ └── json_obj_handler.py # JSON file operations and DataFrame integration
├── general/
│ └── introspection_utils.py # Function introspection and debugging
└── scripts/
├── bulk_rename.py # Bulk renaming automation
├── compress_pdf.py # PDF compression automation
├── copy_compress.py # File copy and compression workflow
├── eml2pdf_exec.py # Email to PDF conversion
├── modify_properties.py # File property modification
├── msg2pdf_exec.py # MSG to PDF conversion
└── tweak_pdf.py # PDF page manipulation
find_files() - Advanced file searching with pattern matchingreorder_objs() - Automatic sequential file/directory renamingrsync() - Directory synchronisation with advanced optionsmodify_obj_permissions() - Batch permission modificationmerge_files() - PDF merging with customisable optionsfile_compressor() - PDF compression with quality controleml_to_pdf(), msg_to_pdf() - Email format conversionmerge_excel_files() - Multi-file Excel processingstandardise_time_series() - Time series data normalisationscan_ncfiles() - NetCDF file analysis and cataloguingget_latlon_bounds() - Climate data coordinate extractionserialise_to_json() - Advanced JSON serialisationdeserialise_json_to_df() - JSON to DataFrame conversionCurrent version: 3.10.0
For detailed version history and changelog, see CHANGELOG.md and VERSIONING.md.
Contributions are welcome! Please feel free to submit a Pull Request. For major changes, please open an issue first to discuss what you would like to change.
This project is licensed under the MIT License - see the LICENSE file for details.
For any questions or suggestions, please open an issue on GitHub or contact the maintainers.
This package relies on several high-quality external packages:
pygenutils - General utility functions and data manipulationparamlib - Parameter and configuration managementFAQs
A Python package for efficient file and directory management, featuring tools for bulk renaming, data handling, and format conversion
We found that filewise demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 1 open source maintainer collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
Popular npm packages like eslint-config-prettier were compromised after a phishing attack stole a maintainer’s token, spreading malicious updates.
Security News
/Research
A phishing attack targeted developers using a typosquatted npm domain (npnjs.com) to steal credentials via fake login pages - watch out for similar scams.
Security News
Knip hits 500 releases with v5.62.0, refining TypeScript config detection and updating plugins as monthly npm downloads approach 12M.