@google-cloud/bigquery

What is @google-cloud/bigquery?

@google-cloud/bigquery is a Node.js client library for Google BigQuery, a fully-managed, serverless data warehouse that enables scalable analysis over petabytes of data. This package allows you to interact with BigQuery to run queries, manage datasets, tables, and jobs, and perform data manipulations.

What are @google-cloud/bigquery's main functionalities?

Running Queries

This feature allows you to run SQL queries against your BigQuery datasets. The code sample demonstrates how to run a simple query to select names and ages from a table where the age is greater than 30.

const {BigQuery} = require('@google-cloud/bigquery');
const bigquery = new BigQuery();
async function query() {
  const query = 'SELECT name, age FROM `my-dataset.my-table` WHERE age > 30';
  const options = { query: query, location: 'US' };
  const [rows] = await bigquery.query(options);
  console.log('Rows:', rows);
}
query();

Managing Datasets

This feature allows you to create and manage datasets in BigQuery. The code sample demonstrates how to create a new dataset named 'my_new_dataset'.

const {BigQuery} = require('@google-cloud/bigquery');
const bigquery = new BigQuery();
async function createDataset() {
  const [dataset] = await bigquery.createDataset('my_new_dataset');
  console.log(`Dataset ${dataset.id} created.`);
}
createDataset();

Managing Tables

This feature allows you to create and manage tables within datasets. The code sample demonstrates how to create a new table named 'my_new_table' within the 'my_new_dataset' dataset.

const {BigQuery} = require('@google-cloud/bigquery');
const bigquery = new BigQuery();
async function createTable() {
  const dataset = bigquery.dataset('my_new_dataset');
  const [table] = await dataset.createTable('my_new_table');
  console.log(`Table ${table.id} created.`);
}
createTable();

Loading Data

This feature allows you to load data from various sources into BigQuery tables. The code sample demonstrates how to load data from a local CSV file into a table named 'my_new_table' within the 'my_new_dataset' dataset.

const {BigQuery} = require('@google-cloud/bigquery');
const bigquery = new BigQuery();
async function loadData() {
  const dataset = bigquery.dataset('my_new_dataset');
  const table = dataset.table('my_new_table');
  const [job] = await table.load('local-file.csv');
  console.log(`Job ${job.id} completed.`);
}
loadData();

Other packages similar to @google-cloud/bigquery

aws-sdk

The aws-sdk package is the official AWS SDK for JavaScript, which allows you to interact with various AWS services, including Amazon Redshift, a data warehouse service similar to Google BigQuery. While aws-sdk provides a broader range of functionalities across different AWS services, it can be used to perform similar data warehousing tasks as @google-cloud/bigquery.

azure-sdk-for-js

The azure-sdk-for-js package is the official Azure SDK for JavaScript, which allows you to interact with various Azure services, including Azure Synapse Analytics, a data warehousing service similar to Google BigQuery. Like aws-sdk, azure-sdk-for-js offers a wide range of functionalities across different Azure services, including data warehousing capabilities.

snowflake-sdk

The snowflake-sdk package is the official Node.js driver for Snowflake, a cloud-based data warehousing service. It provides functionalities to connect to Snowflake, run queries, and manage data, similar to what @google-cloud/bigquery offers for Google BigQuery.

README

Google Cloud BigQuery: Node.js Client

Google BigQuery Client Library for Node.js

• Google Cloud BigQuery Node.js Client API Reference
• Google Cloud BigQuery Documentation
• github.com/googleapis/nodejs-bigquery

Read more about the client libraries for Cloud APIs, including the older Google APIs Client Libraries, in Client Libraries Explained.

Table of contents:

• Quickstart
  • Before you begin
  • Installing the client library
  • Using the client library
• Samples
• Versioning
• Contributing
• License

Quickstart

Before you begin

1. Select or create a Cloud Platform project.
2. Enable the Google Cloud BigQuery API.
3. Set up authentication with a service account so you can access the API from your local workstation.

Installing the client library

npm install @google-cloud/bigquery

Using the client library

// Imports the Google Cloud client library
const {BigQuery} = require('@google-cloud/bigquery');

async function createDataset() {
  // Creates a client
  const bigqueryClient = new BigQuery();

  // Create the dataset
  const [dataset] = await bigqueryClient.createDataset(datasetName);
  console.log(`Dataset ${dataset.id} created.`);
}
createDataset();

Samples

Samples are in the samples/ directory. The samples' README.md has instructions for running the samples.

Sample	Source Code	Try it
Add Column Load Append	source code
Add Column Query Append	source code
Add Empty Column	source code
Auth View Tutorial	source code
Browse Rows	source code
Cancel Job	source code
Client JSON Credentials	source code
Copy Table	source code
Copy Table Multiple Source	source code
Create Dataset	source code
Create Job	source code
Create Table	source code
Create Table Partitioned	source code
Create Table Range Partitioned	source code
Create View	source code
Ddl Create View	source code
Delete Dataset	source code
Delete Label Dataset	source code
Delete Label Table	source code
Delete Model	source code
Delete Table	source code
Extract Table Compressed	source code
Extract Table JSON	source code
Extract Table To GCS	source code
Get Dataset	source code
Get Dataset Labels	source code
Get Job	source code
BigQuery Get Model	source code
BigQuery Get Table	source code
Get Table Labels	source code
Get View	source code
Insert Rows As Stream	source code
Inserting Data Types	source code
BigQuery Label Dataset	source code
Label Table	source code
List Datasets	source code
List Datasets By Label	source code
List Jobs	source code
BigQuery List Models	source code
BigQuery List Models Streaming	source code
List Tables	source code
Load CSV From GCS	source code
Load CSV From GCS Autodetect	source code
Load CSV From GCS Truncate	source code
Load JSON From GCS	source code
Load JSON From GCS Autodetect	source code
Load JSON From GCS Truncate	source code
Load Local File	source code
Load Orc From GCS Truncate	source code
Load Parquet From GCS Truncate	source code
Load Table GCS Avro	source code
Load Table GCS Avro Truncate	source code
Load Table GCSORC	source code
Load Table GCS Parquet	source code
Load Table Partitioned	source code
Nested Repeated Schema	source code
Query	source code
Query Batch	source code
Query Destination Table	source code
Query Disable Cache	source code
Query Dry Run	source code
Query External GCS Perm	source code
Query Legacy	source code
Query Legacy Large Results	source code
Query Params Arrays	source code
Query Params Named	source code
Query Params Named Types	source code
Query Params Positional	source code
Query Params Positional Types	source code
Query Params Structs	source code
Query Params Timestamps	source code
Query Stack Overflow	source code
Quickstart	source code
Relax Column	source code
Relax Column Load Append	source code
Relax Column Query Append	source code
Undelete Table	source code
Update Dataset Access	source code
Update Dataset Description	source code
Update Dataset Expiration	source code
BigQuery Update Model	source code
Update Table Description	source code
Update Table Expiration	source code
Update View Query	source code

The Google Cloud BigQuery Node.js Client API Reference documentation also contains samples.

Supported Node.js Versions

Our client libraries follow the Node.js release schedule. Libraries are compatible with all current active and maintenance versions of Node.js.

Client libraries targetting some end-of-life versions of Node.js are available, and can be installed via npm dist-tags. The dist-tags follow the naming convention legacy-(version).

Legacy Node.js versions are supported as a best effort:

• Legacy versions will not be tested in continuous integration.
• Some security patches may not be able to be backported.
• Dependencies will not be kept up-to-date, and features will not be backported.

Legacy tags available

• legacy-8: install client libraries from this dist-tag for versions compatible with Node.js 8.

Versioning

This library follows Semantic Versioning.

This library is considered to be General Availability (GA). This means it is stable; the code surface will not change in backwards-incompatible ways unless absolutely necessary (e.g. because of critical security issues) or with an extensive deprecation period. Issues and requests against GA libraries are addressed with the highest priority.

More Information: Google Cloud Platform Launch Stages

Contributing

Contributions welcome! See the Contributing Guide.

Please note that this README.md, the samples/README.md, and a variety of configuration files in this repository (including .nycrc and tsconfig.json) are generated from a central template. To edit one of these files, make an edit to its template in this directory.

License

Apache Version 2.0

See LICENSE

Changelog

5.0.0 (2020-06-19)

⚠ BREAKING CHANGES

• don't return Stream from createLoadJob (#647)
• drop Node 8 from engines field (#662)

Features

• drop Node 8 from engines field (#662) (712b029)
• improved types (40087fa)
• update types.d.ts (#667) (a12b094), closes #662 #662 #647 #647 #640 #640 #647 #661 #661 #658 #658 #8203 #8203 #8203 #8203 #8203 #657 #657 #8203 #8203 #8203 #8203 #8203 #8203 #8203 #8203 #660 #660 #8203 #8203 #8203 #8203 #8203 #8203 #8203 #8203 #8203 #8203 #8203 #8203 #665 #665 #454 #454 #456 #456 #463 #463
• userAgent: allow for optional user agent to be provided (#671) (25aeca8)

Bug Fixes

• add types for hasDrift and seasonalPeriods (#680) (d6c9566)
• deps: update dependency @google-cloud/paginator to v3 (#658) (a09c493)
• apache license URL (#468) (#669) (d3ed602)
• drop dependency on string-format-obj (#698) (cf8f58f)
• load job to a different project ID (#748) (bfb74ad)
• docs: fix link for job configuration load (#678) (ea3d7af)
• selectedFields on getRows not working correctly (#712) (13b7e39)
• deps: update dependency @google-cloud/promisify to v2 (#657) (5d8112c)
• deps: update dependency @google-cloud/storage to v5 (#700) (a2e34ef)
• deps: update dependency google-auth-library to v6 (#660) (3ea642e)
• docs: configuration.copy link (#709) (4a81b1e)
• docs: correct createTablePartitioned sample argument (#701) (9a7520e)
• table: add retries for insert partial failures (#589) (b8639c2), closes #655
• types: drop changes for drift and seasonal (#681) (679d990)

Code Refactoring

• don't return Stream from createLoadJob (#647) (8e26fb5), closes #640 #640