Security News
Research
Data Theft Repackaged: A Case Study in Malicious Wrapper Packages on npm
The Socket Research Team breaks down a malicious wrapper package that uses obfuscation to harvest credentials and exfiltrate sensitive data.
onc is a Python client library that facilitates access to scientific data hosted by Ocean Networks Canada through the Oceans 3.0 API public web services. It can help you explore and download our data, by consuming our discovery, data product download, archive file download, and near real-time data access services.
onc can be installed from PyPI:
pip install onc
A unique Oceans 3.0 API token is required to access our data. To obtain a token, follow the steps below:
Register for an Oceans 3.0 account at https://data.oceannetworks.ca/Registration.
Log into your account at https://data.oceannetworks.ca by clicking the Log In link.
Click the Profile link (top right corner) to access your account profile.
Access the Web Services API tab and click Copy Token.
If you forget your token, you can always find it again in your Oceans 3.0 account profile.
To download ONC data, you need to specify the type of data you require and where in particular (i.e. location, device) it originates from.
In the Oceans 3.0 API, there's a unique code that identifies every location, device, property, data product type, etc. Include these codes in a group of filters (these will be used as URL parameters when making HTTP requests) that determine the data you're interested in.
Discovery methods allow you to explore the hierarchy of the ONC database to obtain the codes for your filters (they work like a "search" function).
The example below uses the getLocations
method to search for locations that include "Burrard" in their name (i.e. "Burrard Inlet"):
from onc import ONC
onc = ONC("YOUR_TOKEN_HERE")
onc.getLocations({"locationName": "Burrard"})
The previous code prints a list with locations that match the search filters provided. Each location in the list includes a dataSearchURL that points to the Data Search Tool, and a locationCode ("BIPP" and "BISS" in this example) that can be used to continue searching "inside" it, as in the following example:
onc.getDeviceCategories({"locationCode": "BIIP"})
onc.getDataProducts({"locationCode": "BIIP", "deviceCategoryCode": "CTD"})
Check more on the discovery methods guide and code examples.
Once you determine the exact dictionary of filters that identifies the data you are interested in, there are multiple methods to download it.
One method is to request the ONC servers to generate a custom data product with the data. This is done through the data product download methods.
The following example downloads two PNG files with plots for 30 seconds of data from a CTD in Campbell River:
params = {
"locationCode": "BIIP",
"deviceCategoryCode": "CTD",
"dataProductCode": "TSSP",
"extension": "png",
"dateFrom": "2019-06-20T00:00:00.000Z",
"dateTo": "2019-06-20T00:30:00.000Z",
"dpo_qualityControl": "1",
"dpo_resample": "none",
}
onc.orderDataProduct(params, includeMetadataFile=False)
The filters above include codes for location, deviceCategory, and dataProduct, as well as the file extension and a time interval (in UTC). They also include a couple of filters to configure this specific data product type (starting with the "dpo_" prefix), which can be obtained from the Data Product Options documentation. You can download more than 120 different types of data products including audio & video.
Check more on the data product download methods guide and code examples.
Another method to obtain ONC data is by directly obtaining a time series of sensor readings (available as soon as they reach our database).
In the following example, we obtain 5 seconds of conductivity readings from the CTD at Burrard Inlet:
params = {
"locationCode": "BIIP",
"deviceCategoryCode": "CTD",
"propertyCode": "conductivity",
"dateFrom": "2019-06-20T00:00:00.000Z",
"dateTo": "2019-06-20T00:00:05.000Z",
}
onc.getDirectByLocation(params)
The result includes matching lists of "values" and "sampleTimes" (increases performance for long time ranges). We also use the property code "conductivity" to limit results to a specific property available in this CTD.
Check more on the near real-time data access methods guide and code examples.
ONC scripts auto-generate and archive data products of different types at set time intervals. You can directly download these data product files from our files archive, as long as you know their unique filename.
In the following example, we get a list of archived files available for a camera at Ridley Island (in a certain time span), and download one of the files:
params = {
"locationCode": "RISS",
"deviceCategoryCode": "VIDEOCAM",
"dateFrom": "2016-12-01T00:00:00.000Z",
"dateTo": "2016-12-01T00:05:00.000Z",
}
result = onc.getListByLocation(params, allPages=True)
# download one of the files from result["files"]
onc.getFile("AXISQ6044PTZACCC8E334C53_20161201T000001.000Z.jpg")
You can use the method getFile()
as above to download individual files or the method getDirectFiles()
to download all the files that match your filters.
Check more on the archive file download methods guide and code examples.
The client library documentation is hosted on GitHub Pages. For documentation and examples about Oceans 3.0 API, visit the wiki and OpenAPI page on the Oceans 3.0 Data Portal website.
We kindly ask users to not use too many threads when using threading/multiprocessing libraries on download tasks. It can cause issues for both server and client and may not appreciably increase download speeds.
All contributions are welcome and appreciated! Please refer to the Contributing guide before submitting any issues or pull requests.
FAQs
Oceans 3.0 API Python Client Library
We found that onc demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 4 open source maintainers collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
Research
The Socket Research Team breaks down a malicious wrapper package that uses obfuscation to harvest credentials and exfiltrate sensitive data.
Research
Security News
Attackers used a malicious npm package typosquatting a popular ESLint plugin to steal sensitive data, execute commands, and exploit developer systems.
Security News
The Ultralytics' PyPI Package was compromised four times in one weekend through GitHub Actions cache poisoning and failure to rotate previously compromised API tokens.