bqemulatormanager

BiqQueryEmulator Manager

this package is wrapper of bigquery-emulator which provides us BigQuery mock working in local machine.

using this package, you can

• do unit test of your sql
• download the schema of big query, and use it to make test data

usage

1. following instruction, download bigquery-emulator command.

2. install this package.

pip install bqemulatormanager

1. test your sql.

import bqemulatormanager as bqem
import pandas as pd

manager = bqem.Manager(project='test', schema_path='resources/schema_example.yaml')

with manager:
    data = pd.DataFrame([
        {'id': 1, 'name': 'sato'},
        {'id': 2, 'name': 'yamada'}
    ])

    manager.load(data, 'dataset1.table_a')

    sql = 'SELECT id, name FROM `dataset1.table_a`'

    df = manager.query(sql)
print(df)

unit test example is here.

automatically detect schema

When called Manager.load, SchemaManager search correspond table schema from schema_path (default is bqem_master_schema.yaml).

If schema definition cannot be found, SchemaManager request it from BigQuery in production environment and update master schema file.