By Ryan Gerard Wilson (https://ryangerardwilson.com)
rgwfuncs keeps your Python data science and ML code clean by avoiding types defined in external libraries. Foreign types make debugging a nightmare—rgwfuncs uses functional and procedural paradigms, ensuring your types stay local and under your control.
It wraps complex data science patterns into clear, reusable functions with well-defined, type-casted returns. No wrestling with weird objects you don’t have time to decipher—just simple, predictable outputs that make your life easier. You get the power of Python’s best libraries without the headache of their opaque classes.
Built for Linux, some features (like database queries or Slack integration) require a .rgwfuncsrc file. With rgwfuncs, your code is debug-friendly, maintainable, and refreshingly straightforward—because type-casted functions beat cryptic objects every time.
Install the package using:
pip install rgwfuncs
.rgwfuncsrc FileA .rgwfuncsrc file (located in the current, or any parent directory) is required for MSSQL, CLICKHOUSE, MYSQL, GOOGLE BIG QUERY, SLACK, TELEGRAM, and GMAIL integrations.
{
"db_presets" : [
{
"name": "mssql_db9",
"db_type": "mssql",
"host": "",
"username": "",
"password": "",
"database": ""
},
{
"name": "clickhouse_db7",
"db_type": "clickhouse",
"host": "",
"username": "",
"password": "",
"database": ""
},
{
"name": "mysql_db2",
"db_type": "mysql",
"host": "",
"username": "",
"password": "",
"database": ""
},
{
"name": "bq_db1",
"db_type": "google_big_query",
"json_file_path": "",
"project_id": ""
},
{
"name": "athena_db1",
"db_type": "aws_athena",
"aws_access_key": "",
"aws_secret_key": "",
"aws_region: "",
"database": "logs",
"output_bucket": "s3://bucket-name"
}
],
"vm_presets": [
{
"name": "main_server",
"host": "",
"ssh_user": "",
"ssh_key_path": ""
}
],
"cloud_storage_presets": [
{
"name": "gcs_bucket_name",
"credential_path": "/path/to/your/credentials.json"
}
],
"telegram_bot_presets": [
{
"name": "rgwml-bot",
"chat_id": "",
"bot_token": ""
}
],
"slack_bot_presets": [
{
"name": "labs-channel",
"channel_id": "",
"bot_token": ""
}
],
"gmail_bot_presets": [
{
"name": "info@xyz.com",
"service_account_credentials_path": "/path/to/your/credentials.json"
}
]
}
Import the library:
import rgwfuncs
View available function docstrings in alphabetical order:
rgwfuncs.docs()
View specific docstrings by providing a filter (comma-separated). For example, to display docstrings about "numeric_clean":
rgwfuncs.docs(method_type_filter='numeric_clean')
To display all docstrings, use:
rgwfuncs.docs(method_type_filter='*')
Print a list of available function names in alphabetical order. If a filter is provided, print the docstrings of functions containing the term.
• Parameters:
method_type_filter (str): Optional, comma-separated to select docstring types, or '*' for all.• Example:
from rgwfuncs import docs
docs(method_type_filter='numeric_clean,limit_dataframe')
This section includes functions that facilitate launching an interactive Python shell to inspect and modify local variables within the user's environment.
interactive_shellLaunches an interactive prompt for inspecting and modifying local variables, making all methods in the rgwfuncs library available by default. This REPL (Read-Eval-Print Loop) environment supports command history and autocompletion, making it easier to interact with your Python code. This function is particularly useful for debugging purposes when you want real-time interaction with your program's execution environment.
• Parameters:
local_vars (dict, optional): A dictionary of local variables to be accessible within the interactive shell. If not provided, defaults to an empty dictionary.• Usage:
• Example:
from rgwfuncs import interactive_shell
import pandas as pd
import numpy as np
# Example DataFrame
df = pd.DataFrame({
'id': [1, 2, 3, 4, 5],
'name': ['Alice', 'Bob', 'Charlie', 'David', 'Eva'],
'age': [30, 25, 35, 28, 22],
'city': ['New York', 'Los Angeles', 'Chicago', 'San Francisco', 'Boston']
})
# Launch the interactive shell with local variables
interactive_shell(locals())
Subsequently, in the interactive shell you can use any library in your python file, as well as all rgwfuncs methods (even if they are not imported). Notice, that while pandas and numpy are available in the shell as a result of importing them in the above script, the rgwfuncs method first_n_rows was not imported - yet is available for use.
Welcome to the rgwfuncs interactive shell.
>>> pirst_n_rows(df, 2)
Traceback (most recent call last):
File "<console>", line 1, in <module>
NameError: name 'pirst_n_rows' is not defined. Did you mean: 'first_n_rows'?
>>> first_n_rows(df, 2)
{'age': '30', 'city': 'New York', 'id': '1', 'name': 'Alice'}
{'age': '25', 'city': 'Los Angeles', 'id': '2', 'name': 'Bob'}
>>> print(df)
id name age city
0 1 Alice 30 New York
1 2 Bob 25 Los Angeles
2 3 Charlie 35 Chicago
3 4 David 28 San Francisco
4 5 Eva 22 Boston
>>> arr = np.array([1, 2, 3, 4, 5])
>>> arr
array([1, 2, 3, 4, 5])
Below is a quick reference of available functions, their purpose, and basic usage examples.
numeric_cleanCleans the numeric columns in a DataFrame according to specified treatments.
• Parameters:
column_names (str): A comma-separated string containing the names of the columns to clean.column_type (str): The type to convert the column to (INTEGER or FLOAT).irregular_value_treatment (str): How to treat irregular values (NAN, TO_ZERO, MEAN).• Returns:
• Example:
from rgwfuncs import numeric_clean
import pandas as pd
# Sample DataFrame
df = pd.DataFrame({
'col1': [1, 2, 3, 'x', 4],
'col2': [10.5, 20.1, 'not_a_number', 30.2, 40.8]
})
# Clean numeric columns
df_cleaned = numeric_clean(df, 'col1,col2', 'FLOAT', 'MEAN')
print(df_cleaned)
limit_dataframeLimit the DataFrame to a specified number of rows.
• Parameters:
num_rows (int): The number of rows to retain.• Returns:
• Example:
from rgwfuncs import limit_dataframe
import pandas as pd
df = pd.DataFrame({'A': range(10), 'B': range(10, 20)})
df_limited = limit_dataframe(df, 5)
print(df_limited)
from_raw_dataCreate a DataFrame from raw data.
• Parameters:
• Returns:
• Example:
from rgwfuncs import from_raw_data
headers = ["Name", "Age"]
data = [
["Alice", 30],
["Bob", 25],
["Charlie", 35]
]
df = from_raw_data(headers, data)
print(df)
append_rowsAppend rows to the DataFrame.
• Parameters:
• Returns:
• Example:
from rgwfuncs import append_rows
import pandas as pd
df = pd.DataFrame({'Name': ['Alice'], 'Age': [30]})
new_rows = [
['Bob', 25],
['Charlie', 35]
]
df_appended = append_rows(df, new_rows)
print(df_appended)
append_columnsAppend new columns to the DataFrame with None values.
• Parameters:
col_names (list or comma-separated string): The names of the columns to add.• Returns:
• Example:
from rgwfuncs import append_columns
import pandas as pd
df = pd.DataFrame({'Name': ['Alice', 'Bob'], 'Age': [30, 25]})
df_new = append_columns(df, ['Salary', 'Department'])
print(df_new)
update_rowsUpdate specific rows in the DataFrame based on a condition.
• Parameters:
• Returns:
• Example:
from rgwfuncs import update_rows
import pandas as pd
df = pd.DataFrame({'Name': ['Alice', 'Bob'], 'Age': [30, 25]})
df_updated = update_rows(df, "Name == 'Alice'", {'Age': 31})
print(df_updated)
delete_rowsDelete rows from the DataFrame based on a condition.
• Parameters:
• Returns:
• Example:
from rgwfuncs import delete_rows
import pandas as pd
df = pd.DataFrame({'Name': ['Alice', 'Bob'], 'Age': [30, 25]})
df_deleted = delete_rows(df, "Age < 28")
print(df_deleted)
drop_duplicatesDrop duplicate rows in the DataFrame, retaining the first occurrence.
• Parameters:
• Returns:
• Example:
from rgwfuncs import drop_duplicates
import pandas as pd
df = pd.DataFrame({'A': [1,1,2,2], 'B': [3,3,4,4]})
df_no_dupes = drop_duplicates(df)
print(df_no_dupes)
drop_duplicates_retain_firstDrop duplicate rows based on specified columns, retaining the first occurrence.
• Parameters:
• Returns:
• Example:
from rgwfuncs import drop_duplicates_retain_first
import pandas as pd
df = pd.DataFrame({'A': [1,1,2,2], 'B': [3,3,4,4]})
df_no_dupes = drop_duplicates_retain_first(df, 'A')
print(df_no_dupes)
drop_duplicates_retain_lastDrop duplicate rows based on specified columns, retaining the last occurrence.
• Parameters:
• Returns:
• Example:
from rgwfuncs import drop_duplicates_retain_last
import pandas as pd
df = pd.DataFrame({'A': [1,1,2,2], 'B': [3,3,4,4]})
df_no_dupes = drop_duplicates_retain_last(df, 'A')
print(df_no_dupes)
load_data_from_queryLoad data from a specified database using a SQL query and return the results in a Pandas DataFrame. The database connection configurations are determined by a preset name specified in a configuration file.
db_preset_name (str): The name of the database preset found in the configuration file. This preset determines which database connection details to use.query (str): The SQL query string to be executed on the database.config (Optional[Union[str, dict]], optional): Configuration source. Can be:
pd.DataFrame: Returns a DataFrame that contains the results from the executed SQL query.~/.rgwfuncsrc.name: Name of the database preset.db_type: Type of the database (mssql, mysql, clickhouse, google_big_query, aws_athena).credentials: Necessary credentials such as host, username, password, and potentially others depending on the database type.from rgwfuncs import load_data_from_query
# Load data using a preset configuration
df = load_data_from_query(
db_preset_name="MyDBPreset",
query="SELECT * FROM my_table"
)
print(df)
~/.rgwfuncsrc) is secure and accessible only to authorized users, as it contains sensitive information.pymssql for MSSQL, mysql-connector-python for MySQL, and so on.ValueError if the specified preset name does not exist or if the database type is unsupported. Additional exceptions may arise from network issues or database errors.load_data_from_pathLoad data from a file into a DataFrame based on the file extension.
• Parameters:
file_path (str): The absolute path to the data file.• Returns:
• Example:
from rgwfuncs import load_data_from_path
df = load_data_from_path("/absolute/path/to/data.csv")
print(df)
load_data_from_sqlite_pathExecute a query on a SQLite database file and return the results as a DataFrame.
• Parameters:
sqlite_path (str): The absolute path to the SQLite database file.• Returns:
• Example:
from rgwfuncs import load_data_from_sqlite_path
df = load_data_from_sqlite_path("/path/to/database.db", "SELECT * FROM my_table")
print(df)
first_n_rowsDisplay the first n rows of the DataFrame (prints out in dictionary format).
• Parameters:
• Example:
from rgwfuncs import first_n_rows
import pandas as pd
df = pd.DataFrame({'A': [1,2,3], 'B': [4,5,6]})
first_n_rows(df, 2)
last_n_rowsDisplay the last n rows of the DataFrame (prints out in dictionary format).
• Parameters:
• Example:
from rgwfuncs import last_n_rows
import pandas as pd
df = pd.DataFrame({'A': [1,2,3,4,5], 'B': [6,7,8,9,10]})
last_n_rows(df, 2)
top_n_unique_valuesPrint the top n unique values for specified columns in the DataFrame.
• Parameters:
• Example:
from rgwfuncs import top_n_unique_values
import pandas as pd
df = pd.DataFrame({'Cities': ['NY', 'LA', 'NY', 'SF', 'LA', 'LA']})
top_n_unique_values(df, 2, ['Cities'])
bottom_n_unique_valuesPrint the bottom n unique values for specified columns in the DataFrame.
• Parameters:
• Example:
from rgwfuncs import bottom_n_unique_values
import pandas as pd
df = pd.DataFrame({'Cities': ['NY', 'LA', 'NY', 'SF', 'LA', 'LA']})
bottom_n_unique_values(df, 1, ['Cities'])
print_correlationPrint correlation for multiple pairs of columns in the DataFrame.
• Parameters:
column_pairs (list of tuples): E.g., [('col1','col2'), ('colA','colB')].• Example:
from rgwfuncs import print_correlation
import pandas as pd
df = pd.DataFrame({
'col1': [1,2,3,4,5],
'col2': [2,4,6,8,10],
'colA': [10,9,8,7,6],
'colB': [5,4,3,2,1]
})
pairs = [('col1','col2'), ('colA','colB')]
print_correlation(df, pairs)
print_memory_usagePrint the memory usage of the DataFrame in megabytes.
• Parameters:
• Example:
from rgwfuncs import print_memory_usage
import pandas as pd
df = pd.DataFrame({'A': range(1000)})
print_memory_usage(df)
filter_dataframeReturn a new DataFrame filtered by a given query expression.
• Parameters:
filter_expr (str)• Returns:
• Example:
from rgwfuncs import filter_dataframe
import pandas as pd
df = pd.DataFrame({
'Name': ['Alice', 'Bob', 'Charlie'],
'Age': [30, 20, 25]
})
df_filtered = filter_dataframe(df, "Age > 23")
print(df_filtered)
filter_indian_mobilesFilter and return rows containing valid Indian mobile numbers in the specified column.
• Parameters:
mobile_col (str): The column name with mobile numbers.• Returns:
• Example:
from rgwfuncs import filter_indian_mobiles
import pandas as pd
df = pd.DataFrame({'Phone': ['9876543210', '12345', '7000012345']})
df_indian = filter_indian_mobiles(df, 'Phone')
print(df_indian)
print_dataframePrint the entire DataFrame and its column types. Optionally print a source path.
• Parameters:
• Example:
from rgwfuncs import print_dataframe
import pandas as pd
df = pd.DataFrame({'Name': ['Alice'], 'Age': [30]})
print_dataframe(df, source='SampleData.csv')
send_dataframe_via_telegramSend a DataFrame via Telegram using a specified bot configuration.
• Parameters:
bot_name (str)as_file (bool)remove_after_send (bool)config (Optional[Union[str, dict]], optional): Configuration source. Can be:
• Example:
from rgwfuncs import send_dataframe_via_telegram
# Suppose your bot config is in "rgwml.config" under [TelegramBots] section
df = ... # Some DataFrame
send_dataframe_via_telegram(
df,
bot_name='MyTelegramBot',
message='Hello from RGWFuncs!',
as_file=True,
remove_after_send=True
)
send_data_to_emailSend an email with an optional DataFrame attachment using the Gmail API via a specified preset.
• Parameters:
preset_name (str)to_email (str)as_file (bool)remove_after_send (bool)config (Optional[Union[str, dict]], optional): Configuration source. Can be:
• Example:
from rgwfuncs import send_data_to_email
df = ... # Some DataFrame
send_data_to_email(
df,
preset_name='MyEmailPreset',
to_email='recipient@example.com',
subject='Hello from RGWFuncs',
body='Here is the data you requested.',
as_file=True,
remove_after_send=True
)
send_data_to_slackSend a DataFrame or message to Slack using a specified bot configuration.
• Parameters:
bot_name (str)as_file (bool)remove_after_send (bool)config (Optional[Union[str, dict]], optional): Configuration source. Can be:
• Example:
from rgwfuncs import send_data_to_slack
df = ... # Some DataFrame
send_data_to_slack(
df,
bot_name='MySlackBot',
message='Hello Slack!',
as_file=True,
remove_after_send=True
)
order_columnsReorder the columns of a DataFrame based on a string input.
• Parameters:
column_order_str (str): Comma-separated column order.• Returns:
• Example:
from rgwfuncs import order_columns
import pandas as pd
df = pd.DataFrame({'Name': ['Alice', 'Bob'], 'Age': [30, 25], 'Salary': [1000, 1200]})
df_reordered = order_columns(df, 'Salary,Name,Age')
print(df_reordered)
append_ranged_classification_columnAppend a ranged classification column to the DataFrame.
• Parameters:
target_col (str): The column to classify.new_col_name (str): Name of the new classification column.• Returns:
• Example:
from rgwfuncs import append_ranged_classification_column
import pandas as pd
df = pd.DataFrame({'Scores': [5, 12, 25]})
df_classified = append_ranged_classification_column(df, [0, 10, 20, 30], 'Scores', 'ScoreRange')
print(df_classified)
append_percentile_classification_columnAppend a percentile classification column to the DataFrame.
• Parameters:
target_col (str)new_col_name (str)• Returns:
• Example:
from rgwfuncs import append_percentile_classification_column
import pandas as pd
df = pd.DataFrame({'Values': [10, 20, 30, 40, 50]})
df_classified = append_percentile_classification_column(df, [25, 50, 75], 'Values', 'ValuePercentile')
print(df_classified)
append_ranged_date_classification_columnAppend a ranged date classification column to the DataFrame.
• Parameters:
date_ranges (list[str]): List of date strings in a format pandas can parse (e.g., ['2020-01-01', '2020-06-30', '2020-12-31']).target_col (str)new_col_name (str)• Returns:
• Example:
from rgwfuncs import append_ranged_date_classification_column
import pandas as pd
df = pd.DataFrame({'EventDate': pd.to_datetime(['2020-03-15', '2020-08-10'])})
df_classified = append_ranged_date_classification_column(
df,
['2020-01-01', '2020-06-30', '2020-12-31'],
'EventDate',
'DateRange'
)
print(df_classified)
rename_columnsRename columns in the DataFrame.
• Parameters:
rename_pairs (dict): Mapping old column names to new ones.• Returns:
• Example:
from rgwfuncs import rename_columns
import pandas as pd
df = pd.DataFrame({'OldName': [1,2,3]})
df_renamed = rename_columns(df, {'OldName': 'NewName'})
print(df_renamed)
cascade_sortCascade sort the DataFrame by specified columns and order.
• Parameters:
• Returns:
• Example:
from rgwfuncs import cascade_sort
import pandas as pd
df = pd.DataFrame({
'Name': ['Charlie', 'Alice', 'Bob'],
'Age': [25, 30, 22]
})
sorted_df = cascade_sort(df, ["Name::ASC", "Age::DESC"])
print(sorted_df)
append_xgb_labelsAppend XGB training labels (TRAIN, VALIDATE, TEST) based on a ratio string.
• Parameters:
ratio_str (str): e.g. "8:2", "7:2:1".• Returns:
• Example:
from rgwfuncs import append_xgb_labels
import pandas as pd
df = pd.DataFrame({'A': range(10)})
df_labeled = append_xgb_labels(df, "7:2:1")
print(df_labeled)
append_xgb_regression_predictionsAppend XGB regression predictions to the DataFrame. Requires an XGB_TYPE column for TRAIN/TEST splits.
• Parameters:
target_col (str)feature_cols (str): Comma-separated feature columns.pred_col (str)boosting_rounds (int, optional)model_path (str, optional)• Returns:
• Example:
from rgwfuncs import append_xgb_regression_predictions
import pandas as pd
df = pd.DataFrame({
'XGB_TYPE': ['TRAIN','TRAIN','TEST','TEST'],
'Feature1': [1.2, 2.3, 3.4, 4.5],
'Feature2': [5.6, 6.7, 7.8, 8.9],
'Target': [10, 20, 30, 40]
})
df_pred = append_xgb_regression_predictions(df, 'Target', 'Feature1,Feature2', 'PredictedTarget')
print(df_pred)
append_xgb_logistic_regression_predictionsAppend XGB logistic regression predictions to the DataFrame. Requires an XGB_TYPE column for TRAIN/TEST splits.
• Parameters:
target_col (str)feature_cols (str)pred_col (str)boosting_rounds (int, optional)model_path (str, optional)• Returns:
• Example:
from rgwfuncs import append_xgb_logistic_regression_predictions
import pandas as pd
df = pd.DataFrame({
'XGB_TYPE': ['TRAIN','TRAIN','TEST','TEST'],
'Feature1': [1, 0, 1, 0],
'Feature2': [0.5, 0.2, 0.8, 0.1],
'Target': [1, 0, 1, 0]
})
df_pred = append_xgb_logistic_regression_predictions(df, 'Target', 'Feature1,Feature2', 'PredictedTarget')
print(df_pred)
print_n_frequency_cascadingPrint the cascading frequency of top n values for specified columns.
• Parameters:
order_by (str): ASC, DESC, FREQ_ASC, FREQ_DESC.• Example:
from rgwfuncs import print_n_frequency_cascading
import pandas as pd
df = pd.DataFrame({'City': ['NY','LA','NY','SF','LA','LA']})
print_n_frequency_cascading(df, 2, 'City', 'FREQ_DESC')
print_n_frequency_linearPrints the linear frequency of the top n values for specified columns.
pd.DataFrame): The DataFrame to analyze.int): The number of top values to print for each column.list): A list of column names to be analyzed.str): The order of frequency. The available options are:
"ASC": Sort keys in ascending lexicographical order."DESC": Sort keys in descending lexicographical order."FREQ_ASC": Sort the frequencies in ascending order (least frequent first)."FREQ_DESC": Sort the frequencies in descending order (most frequent first)."BY_KEYS_ASC": Sort keys in ascending order, numerically if possible, handling special strings like 'NaN' as typical entries."BY_KEYS_DESC": Sort keys in descending order, numerically if possible, handling special strings like 'NaN' as typical entries.from rgwfuncs import print_n_frequency_linear
import pandas as pd
df = pd.DataFrame({'City': ['NY', 'LA', 'NY', 'SF', 'LA', 'LA']})
print_n_frequency_linear(df, 2, ['City'], 'FREQ_DESC')
This example analyzes the City column, printing the top 2 most frequent values in descending order of frequency.
retain_columnsRetain specified columns in the DataFrame and drop the others.
• Parameters:
columns_to_retain (list or str)• Returns:
• Example:
from rgwfuncs import retain_columns
import pandas as pd
df = pd.DataFrame({'A': [1,2], 'B': [3,4], 'C': [5,6]})
df_reduced = retain_columns(df, ['A','C'])
print(df_reduced)
mask_against_dataframeRetain only rows with common column values between two DataFrames.
• Parameters:
other_df (pd.DataFrame)column_name (str)• Returns:
• Example:
from rgwfuncs import mask_against_dataframe
import pandas as pd
df1 = pd.DataFrame({'ID': [1,2,3], 'Value': [10,20,30]})
df2 = pd.DataFrame({'ID': [2,3,4], 'Extra': ['X','Y','Z']})
df_masked = mask_against_dataframe(df1, df2, 'ID')
print(df_masked)
mask_against_dataframe_converseRetain only rows with uncommon column values between two DataFrames.
• Parameters:
other_df (pd.DataFrame)column_name (str)• Returns:
• Example:
from rgwfuncs import mask_against_dataframe_converse
import pandas as pd
df1 = pd.DataFrame({'ID': [1,2,3], 'Value': [10,20,30]})
df2 = pd.DataFrame({'ID': [2,3,4], 'Extra': ['X','Y','Z']})
df_uncommon = mask_against_dataframe_converse(df1, df2, 'ID')
print(df_uncommon)
union_joinPerform a union join, concatenating two DataFrames and dropping duplicates.
• Parameters:
df1 (pd.DataFrame): First DataFrame.df2 (pd.DataFrame): Second DataFrame.• Returns:
df1 and df2, without duplicates.• Example:
from rgwfuncs import union_join
import pandas as pd
df1 = pd.DataFrame({'ID': [1, 2, 3], 'Value': [10, 20, 30]})
df2 = pd.DataFrame({'ID': [2, 3, 4], 'Value': [20, 30, 40]})
df_union = union_join(df1, df2)
print(df_union)
bag_union_joinPerform a bag union join, concatenating two DataFrames without dropping duplicates.
• Parameters:
df1 (pd.DataFrame): First DataFrame.df2 (pd.DataFrame): Second DataFrame.• Returns:
df1 and df2.• Example:
from rgwfuncs import bag_union_join
import pandas as pd
df1 = pd.DataFrame({'ID': [1, 2, 3], 'Value': [10, 20, 30]})
df2 = pd.DataFrame({'ID': [2, 3, 4], 'Value': [20, 30, 40]})
df_bag_union = bag_union_join(df1, df2)
print(df_bag_union)
left_joinPerform a left join on two DataFrames.
• Parameters:
df1 (pd.DataFrame): The left DataFrame.df2 (pd.DataFrame): The right DataFrame.left_on (str): Column name in df1 to join on.right_on (str): Column name in df2 to join on.• Returns:
• Example:
from rgwfuncs import left_join
import pandas as pd
df1 = pd.DataFrame({'ID': [1, 2, 3], 'Value': [10, 20, 30]})
df2 = pd.DataFrame({'ID': [2, 3, 4], 'Extra': ['A', 'B', 'C']})
df_left_join = left_join(df1, df2, 'ID', 'ID')
print(df_left_join)
right_joinPerform a right join on two DataFrames.
• Parameters:
df1 (pd.DataFrame): The left DataFrame.df2 (pd.DataFrame): The right DataFrame.left_on (str): Column name in df1 to join on.right_on (str): Column name in df2 to join on.• Returns:
• Example:
from rgwfuncs import right_join
import pandas as pd
df1 = pd.DataFrame({'ID': [1, 2, 3], 'Value': [10, 20, 30]})
df2 = pd.DataFrame({'ID': [2, 3, 4], 'Extra': ['A', 'B', 'C']})
df_right_join = right_join(df1, df2, 'ID', 'ID')
print(df_right_join)
insert_dataframe_in_sqlite_databaseInserts a Pandas DataFrame into a SQLite database table. If the specified table does not exist, it will be created with column types automatically inferred from the DataFrame's data types.
Parameters:
db_path (str): The path to the SQLite database file. If the database does not exist, it will be created.tablename (str): The name of the table in the database. If the table does not exist, it is created with the DataFrame's columns and data types.df (pd.DataFrame): The DataFrame containing the data to be inserted into the database table.Returns:
NoneNotes:
int64 is mapped to INTEGERfloat64 is mapped to REALobject is mapped to TEXTdatetime64[ns] is mapped to TEXT (dates are stored as text)bool is mapped to INTEGER (SQLite does not have a separate Boolean type)Example:
from rgwfuncs import insert_dataframe_in_sqlite_database import pandas as pd
df = pd.DataFrame({ 'ID': [1, 2, 3], 'Name': ['Alice', 'Bob', 'Charlie'], 'Score': [88.5, 92.3, 85.0] })
db_path = 'my_database.db' tablename = 'students'
insert_dataframe_in_sqlite_database(db_path, tablename, df)
sync_dataframe_to_sqlite_databaseProcesses and saves a DataFrame to an SQLite database, adding a timestamp column and replacing the existing table if needed. Creates the table if it does not exist.
• Parameters:
db_path (str): Path to the SQLite database file.tablename (str): The name of the table in the database.df (pd.DataFrame): The DataFrame to be processed and saved.• Returns:
• Example:
from rgwfuncs import sync_dataframe_to_sqlite_database
import pandas as pd
df = pd.DataFrame({'ID': [1, 2, 3], 'Value': [10, 20, 30]})
db_path = 'my_database.db'
tablename = 'my_table'
sync_dataframe_to_sqlite_database(db_path, tablename, df)
load_fresh_data_or_pull_from_cacheRetrieves data from a cache if a recent cache file exists, or fetches fresh data, saves it to the cache, removes older cache files, and returns it. If the cache is too old or doesn’t exist, it uses a fetching function to get new data, which it caches and returns. When fresh data is fetched and saved, all earlier cache files with the same prefix are deleted to keep the cache directory clean. An optional dtype parameter allows specifying column data types when reading from the cache, preventing issues with mixed-type columns.
fetch_func (typing.Callable[[], pd.DataFrame]): A callable function that fetches fresh data and returns it as a pandas DataFrame.cache_dir (str): The directory where cache files are stored.file_prefix (str): The prefix used for cache filenames to identify relevant cache files.cache_cutoff_hours (int): The age in hours beyond which a cache file is considered obsolete.dtype (dict, optional): A dictionary specifying the data types for columns when reading the CSV cache file. Passed to pd.read_csv() to handle mixed-type columns explicitly. Defaults to None, in which case pandas infers the types.pd.DataFrame: The DataFrame containing cached or freshly unmarked data.from rgwfuncs import load_fresh_data_or_pull_from_cache
import pandas as pd
def fetch_data():
# This is your data-fetching logic. Replace with real fetching code.
return pd.DataFrame({'Column1': [1, 2, 3], 'Column2': ['4', '5', '6']})
cache_dir = 'cache_directory'
file_prefix = 'cached_data'
cache_cutoff_hours = 24
# Without dtype (pandas infers types)
df = load_fresh_data_or_pull_from_cache(fetch_data, cache_dir, file_prefix, cache_cutoff_hours)
# With dtype to handle mixed types
df = load_fresh_data_or_pull_from_cache(fetch_data, cache_dir, file_prefix, cache_cutoff_hours, dtype={'Column2': str})
print(df)
For more information, refer to each function’s docstring by calling:
rgwfuncs.docs(method_type_filter='function_name')
or display all docstrings with:
rgwfuncs.docs(method_type_filter='*')
© 2025 Ryan Gerard Wilson. All rights reserved.
FAQs
A functional programming paradigm for mathematical modelling and data science
We found that rgwfuncs demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 1 open source maintainer collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
pnpm 10.12.1 introduces a global virtual store for faster installs and new options for managing dependencies with version catalogs.
Security News
Amaro 1.0 lays the groundwork for stable TypeScript support in Node.js, bringing official .ts loading closer to reality.
Research
A deceptive PyPI package posing as an Instagram growth tool collects user credentials and sends them to third-party bot services.