Latest Threat Research:SANDWORM_MODE: Shai-Hulud-Style npm Worm Hijacks CI Workflows and Poisons AI Toolchains.Details →
adamapi
Artifacts are distributable versions of a package. It typically includes the source code and metadata
Advanced tools
adamapi - npm Package Compare versions
+20
| Copyright (c) 2022 MEEO s.r.l. | ||
| Permission is hereby granted, free of charge, to any person obtaining a copy | ||
| of this software and associated documentation files (the "Software"), to deal | ||
| in the Software without restriction, including without limitation the rights | ||
| to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | ||
| copies of the Software, and to permit persons to whom the Software is | ||
| furnished to do so, subject to the following conditions: | ||
| The above copyright notice and this permission notice shall be included in all | ||
| copies or substantial portions of the Software. | ||
| THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | ||
| IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | ||
| FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | ||
| AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER | ||
| LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, | ||
| OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE | ||
| SOFTWARE. | ||
| [build-system] | ||
| requires = [ | ||
| "setuptools>=42", | ||
| "wheel" | ||
| ] | ||
| build-backend = "setuptools.build_meta" |
+334
| # Installation | ||
| ### Versioning | ||
| 1. adamapi==2.0.11 , This package works only with ADAMCORE 1. | ||
| 2. adamapi==2.1.0, This pachage works only with ADAMCORE 2. | ||
| ## Requirements | ||
| ```bash | ||
| sudo apt-get install python3-venv python3-gdal gdal-bin | ||
| ``` | ||
| ## Install with pip | ||
| ```bash | ||
| VENVNAME="adamapi" | ||
| python3 -m venv "${VENVNAME}" | ||
| source "${VENVNAME}/bin/activate"; | ||
| python3 -m pip install --upgrade pip; | ||
| pip install adamapi | ||
| ln -s "/usr/lib/python3/dist-packages/osgeo" "${VENVNAME}/lib/python3.8/site-packages/osgeo" | ||
| ``` | ||
| # API DEFINITIONS | ||
| This document briefly describes the ADMAPI functionalities.<br> | ||
| The ADAMAPI library is divided in 4 modules: | ||
| 1. Auth --> the authorization module | ||
| 2. Datasets --> to get the list of datasets | ||
| 3. Search --> to get the lists of products, including associated metadata (e.g. geometry, cloud cover, orbit, tile, ...) | ||
| 4. GetData --> to retrieve the product(s), including options to subset in space and time in the native data granularity and reduced processing capacity | ||
| ## 1 - Auth | ||
| This module takes care of user authentication and authorization.<br> | ||
| Without instancing an object of this module other components don't work.<br> | ||
| Auth module is based on the ADAMAPI_KEY, a key that uniquelly identifies the user. | ||
| **Class contructor and parameters** | ||
| ```python | ||
| from adamapi import Auth | ||
| a = Auth() | ||
| ``` | ||
| Parameters:<br> | ||
| | position/keyword | mandatory | type | default | description | | ||
| | ------ | ------ | ------ | ------ | ------ | | ||
| | | | | | | | ||
| **Public methods and parameters** | ||
| * **.setKey()** --> To setup the ADAMAPI_KEY<br> | ||
| Parameters: | ||
| | position/keyword | mandatory | type | default | description | | ||
| | ------ | ------ | ------ | ------ | ------ | | ||
| | 0 | True | str | | The ADAMAPI_KEY | | ||
| * **.setAdamCore()** --> To setup the url of the ADAM-CORE endpoint<br> | ||
| Parameters: | ||
| | position/keyword | mandatory | type | default | description | | ||
| | ------ | ------ | ------ | ------ | ------ | | ||
| | 0 | True | str | | The url like https://test.adamplatform.eu | | ||
| * **.authorize()** --> to instanciate an auth object<br> | ||
| Parameters: | ||
| | position/keyword | mandatory | type | default | description | | ||
| | ------ | ------ | ------ | ------ | ------ | | ||
| | | | | | | | ||
| * **.getAuthToken()** --> to get the authorization token<br> | ||
| Parameters: | ||
| | position/keyword | mandatory | type | default | description | | ||
| | ------ | ------ | ------ | ------ | ------ | | ||
| | | | | | | | ||
| ### 1.1 - ADAMAPI_KEY retrieval | ||
| To get the ADAMAPI_KEY, you need to access your ADAM portal and:<br> | ||
| 1. Select the "user icon" on the top right | ||
| 2. Expand / click the "USERNAME" | ||
| 3. Click on the "Api Key" to display your key | ||
| <br> | ||
| *Command-line ADAMAPI_KEY retrieval TBP* | ||
| ### 1.2 - ADAMAPI_KEY setup | ||
| There are three methods to setup the ADAMAPI_KEY and the ADAM-CORE instance: | ||
| 1. use the method setKey() and setAdamCore() | ||
| ```python | ||
| from adamapi import Auth | ||
| a = Auth() | ||
| a.setKey('<ADAMAPI_KEY>') | ||
| a.setAdamCore('https://test.adamplatform.eu') | ||
| ``` | ||
| 2. Export two envars like | ||
| ```bash | ||
| #open a Terminal and type: | ||
| export ADAMAPI_KEY='<ADAMAPI_KEY>' | ||
| export ADAMAPI_URL='https://test.adamplatform.eu' | ||
| ``` | ||
| 3. create a file called **.adamapirc** in the user home directory with the following content | ||
| ```text | ||
| key=<ADAMAPI_KEY> | ||
| url=https://test.adamplatform.eu | ||
| ``` | ||
| ### 1.3 - Examples | ||
| After ADAMAPI_KEY has been set up, an auth instance can be created with: | ||
| ```python | ||
| from adamapi import Auth | ||
| a = Auth() | ||
| a.authorize() | ||
| ``` | ||
| After authorize method you can retrive your autho token: | ||
| ```python | ||
| from adamapi import Auth | ||
| a = Auth() | ||
| a.authorize() | ||
| a.getAuthToken() | ||
| ``` | ||
| ## 2 - Datasets | ||
| This module provides datasets discovery functionality. | ||
| **Class contructor and parameters** | ||
| ```python | ||
| from adamapi import Datasets | ||
| datasets = Datasets( a ) | ||
| ``` | ||
| Parameters:<br> | ||
| | position/keyword | mandatory | type | default | description | | ||
| | ------ | ------ | ------ | ------ | ------ | | ||
| | 0 | True | Auth instance | | The ADAMAPI authorized instance obtained in the previous section | | ||
| **Public methods and parameters** | ||
| * **.getDatasets()** --> To retrieve datasets list <br> | ||
| Parameters: | ||
| | position/keyword | mandatory | type | default | description | | ||
| | ------ | ------ | ------ | ------ | ------ | | ||
| | 0 | False | str | | The datasetId. | | ||
| | page | False | numeric | 0 | Indicats a specific page | | ||
| | maxRecords | False | numeric | 10 | Max number of results in output. | | ||
| This .getDatasets() function can be used to retrive additional filters which are described in the key **filtersEnabled** (if exists). | ||
| ### 2.1 Examples | ||
| This module can be used in 2 different ways. | ||
| 1. To list all available datasets: | ||
| ```python | ||
| datasets = Datasets(a) | ||
| print(datasets.getDatasets()) | ||
| ``` | ||
| 2. To get detailed metadata about a specific dataset | ||
| ```python | ||
| datasets = Datasets(a) | ||
| print( datasets.getDatasets( '{{ID:DATASET}}' , page=0 , maxRecords=10 ) ) | ||
| ``` | ||
| 3. To get filtersEnabled. To use this additional filters see first example in Search section. | ||
| ```python | ||
| datasets = Datasets(a) | ||
| out=datasets.getDatasets("{{ID:DATASET}}") | ||
| print(out["filtersEnabled"]) | ||
| ``` | ||
| ## 3 - Search | ||
| This module provides discovery functionality through the products available on the ADAM instance. | ||
| **Class contructor and parameters** | ||
| ```python | ||
| from adamapi import Search | ||
| search = Search( a ) | ||
| ``` | ||
| Parameters:<br> | ||
| | position/keyword | mandatory | type | default | description | | ||
| | ------ | ------ | ------ | ------ | ------ | | ||
| | 0 | True | Auth instance | | The ADAMAPI authorized instance obtained in section 1-Auth | | ||
| **Public methods and parameters** | ||
| * **.getProducts()** --> To retrieve datasets list and metadata<br> | ||
| Parameters: | ||
| | position/keyword | mandatory | type | default | description | | ||
| | ------ | ------ | ------ | ------ | ------ | | ||
| | 0 | True | str | | The datasetId. | | ||
| | maxRecords | False | int | 10 | number of records | | ||
| | startIndex | False | int | 0 | starting record index | | ||
| | startDate | False | str or [datetime](https://docs.python.org/3/library/datetime.html) | | the start date | | ||
| | endDate | False | str or [datetime](https://docs.python.org/3/library/datetime.html) | | the end date | | ||
| | geometry | True | str or geojson | | GeoJson geometry,[geojson format](https://tools.ietf.org/html/rfc7946) [appendix](#geometry)| | ||
| ### 3.1 Examples | ||
| 1. Example1: | ||
| ```python | ||
| search=Search(a) | ||
| mongo_search=search.getProducts('{{ID:DATASET}}',maxRecords=1,startIndex=0,platform="{{VALUE}}") | ||
| ``` | ||
| 2. Example2: | ||
| ```python | ||
| search=Search(a) | ||
| mongo_search=search.getProducts('{{ID:DATASET}}',maxRecords=1,startIndex=0) | ||
| ``` | ||
| ## 4 - GetData | ||
| This module provides data access of raster, spatial subset, timeseries in the native data granularity and reduced processing capacity. | ||
| **Class contructor and parameters** | ||
| ```python | ||
| from adamapi import GetData | ||
| data=GetData(a) | ||
| ``` | ||
| Parameters:<br> | ||
| | position/keyword | mandatory | type | default | description | | ||
| | ------ | ------ | ------ | ------ | ------ | | ||
| | 0 | True | Auth Instance | | The ADAMAPI authorized instance obtained in the section 1-Auth | | ||
| **Public methods and parameters** | ||
| * **.getData()** --> To retrieve a specific product or a dataset in its native granularity, to get a subset of it, to perform a timeseries or to exec simple processing <br> | ||
| | position/keyword | mandatory | type | default | description | | ||
| | ------ | ------ | ------ | ------ | ------ | | ||
| | 0 | True | str | | The datasetId | | ||
| | 1 | True | str | GetFile | request type. available values: GetFile,GetSubset, GetTimeseries and GetProcessing | | ||
| | asynchronous | False | boolean | False | rappesents how the request will be performed | | ||
| | compress | False | boolean | False | return a zip file | | ||
| | rest | False | boolean | True | perform RESTful order ignoring explorer state on the server and equalization configured using the explorer gui | | ||
| | filters | True | json | {} | json object with filters parameter. startDate and endDate are required inside it | | ||
| | options | False | json | {} | request option | | ||
| | outputDir | False | str | `adamapiresults/` | set a different download directory inside `adamapiresult/` main directory | | ||
| ### 4.1 Examples | ||
| ```python | ||
| data=GetData(a) | ||
| #to retrive a specific product | ||
| image = data.getData('{{ID:DATASET}}',"GetFile",asynchronous=False,compress=False,rest=False,filters={"startDate":'{{STARTDATE}}',"endDate":'{{ENDDATE}}',"productId":'{{PRODUCTID}}'},outputDir='{{OUTPUT_DIR}}') | ||
| #to retrieve a dataset in its native granularity | ||
| data=GetData(self.a) | ||
| image = data.getData('{{ID:DATASET}}',"GetFile",asynchronous=False,compress=False,rest=False,filters={"startDate":'{{STARTDATE}}',"endDate":'{{ENDDATE}}',"geometry":'{{GEOMETRY}}'},outputDir='{{OUTPUT_DIR}}') | ||
| ``` | ||
| For the GetSubset,GetTimeseries and GetProcessing requests you need to add the `options` parameter with these constraints : [output formats](#output-formats) and [functions](#processing-function)(only for processing request) | ||
| ```python | ||
| #subset example | ||
| image = data.getData('{{ID:DATASET}}',"GetSubset",asynchronous=False,compress=False,rest=False,filters={"startDate":'{{STARTDATE}}',"endDate":'{{ENDDATE}}',"geometry":'{{GEOMETRY}}'},options={"format":'{{FORMATS}}'},outputDir='{{OUTPUT_DIR}}') | ||
| #timeseries example | ||
| image = data.getData('{{ID:DATASET}}',"GetTimeseries",asynchronous=False,compress=False,rest=False,filters={"startDate":'{{STARTDATE}}',"endDate":'{{ENDDATE}}',"geometry":'{{GEOMETRY}}'},options={"format":'{{FORMATS}}'},outputDir='{{OUTPUT_DIR}}') | ||
| #processing example | ||
| image = data.getData('{{ID:DATASET}}',"GetProcessing",asynchronous=False,compress=False,rest=False,filters={"startDate":'{{STARTDATE}}',"endDate":'{{ENDDATE}}',"geometry":'{{GEOMETRY}}'},options={"format":'{{FORMAT}}',"function":'{{FUNCTION}}'},outputDir='{{OUTPUT_DIR}}') | ||
| ``` | ||
| ### 4.3 Asyncronous Example | ||
| ```python | ||
| #1. execute the request | ||
| image = data.getData('{{ID:DATASET}}',"GetSubset",asynchronous=False,compress=False,rest=False,filters={"startDate":'{{STARTDATE}}',"endDate":'{{ENDDATE}}',"geometry":'{{GEOMETRY}}'},options={"format":'{{FORMATS}}'},outputDir='{{OUTPUT_DIR}}') | ||
| #2. check the status | ||
| stat=data.getData(datasetId,"GetSubset",asynchronous=True,id=str(image.pk)) | ||
| while stat.status != "completed": | ||
| time.sleep(1) | ||
| stat=data.getData(datasetId,"GetSubset",asynchronous=True,id=str(image.pk)) | ||
| #3. download the zip,unzip it and remove the zip (optional) | ||
| for res in stat.list: | ||
| if res["status"] == "failed": | ||
| print(res["exit_code"]) | ||
| else: | ||
| r=self.a.client(res["download"]["url"],{},"GET") | ||
| with open(str(res["download"]["url"].split("/")[4])+"_"+str(res["download"]["url"].split("/")[5]), 'wb' ) as f: | ||
| f.write( r.content ) | ||
| ``` | ||
| # Appendix 1 - Data format | ||
| ## date and date+time | ||
| Supported string date/date+time format are: | ||
| * '%Y-%m-%dT%H:%M:%S', | ||
| * '%Y-%m-%dT%H:%M:%SZ', | ||
| * '%Y-%m-%d' | ||
| ### GeoJson | ||
| Geometry have to follow the latest geojson standard [rfc7946](https://tools.ietf.org/html/rfc7946)<br> | ||
| In particular Polygons and MultiPolygons should follow the right-hand rule<br> | ||
| ### Geometry | ||
| ```python | ||
| #This geometry will return all the results it has intersected within it | ||
| geometry = { "type": "Polygon", "coordinates": [ [ [ 43.916666667, 15.716666667 ], [ 43.916666667, 15.416666667 ] , [ 44.216666667, 15.416666667 ], [ 44.216666667, 15.716666667 ], [ 43.916666667, 15.716666667 ] ] ] } | ||
| ``` | ||
| ```python | ||
| #This geometry will return all the results it has intersected on its outside | ||
| geometry = { "type": "Polygon", "coordinates": [ [ [ 43.84986877441406,15.925676536359038 ], [ 44.6539306640625,15.950766025306109 ],[ 44.681396484375,15.194084972583916 ], [ 43.8189697265625,15.20998780073036 ], [ 43.84986877441406,15.925676536359038 ] ] ] } | ||
| ``` | ||
| ### Output Formats | ||
| | request | output format | | ||
| | ---- | ---- | | ||
| | GetFile | - | | ||
| | GetSubset | tiff,png | | ||
| | GetTimeseries | json,csv | | ||
| | GetProcessing | tiff,png | | ||
| ### Processing Function | ||
| | type | description | | ||
| | ---- | ---- | | ||
| | average | execute the average from the products | | ||
| | overlap | execute the products overlap without any specific strategy | | ||
| | mosterecent | puts on the top the most recent product | | ||
| | leastrecent | puts on top the least recent product | | ||
| | minvalue | for each pixel, puts on top the minimum value of the pixel | | ||
| | maxvalue | for each pixel, puts on top the maximum value of the pixel | |
| Metadata-Version: 2.1 | ||
| Name: adamapi | ||
| Version: 2.1.0 | ||
| Summary: Python Adam API | ||
| Home-page: https://git.services.meeo.it/das/adamapi | ||
| Author: MEEO s.r.l. | ||
| Author-email: info@meeo.it | ||
| Classifier: Programming Language :: Python :: 3 | ||
| Classifier: License :: OSI Approved :: MIT License | ||
| Classifier: Operating System :: Unix | ||
| Requires-Python: >=3.6 | ||
| Description-Content-Type: text/markdown | ||
| License-File: LICENSE | ||
| # Installation | ||
| ### Versioning | ||
| 1. adamapi==2.0.11 , This package works only with ADAMCORE 1. | ||
| 2. adamapi==2.1.0, This pachage works only with ADAMCORE 2. | ||
| ## Requirements | ||
| ```bash | ||
| sudo apt-get install python3-venv python3-gdal gdal-bin | ||
| ``` | ||
| ## Install with pip | ||
| ```bash | ||
| VENVNAME="adamapi" | ||
| python3 -m venv "${VENVNAME}" | ||
| source "${VENVNAME}/bin/activate"; | ||
| python3 -m pip install --upgrade pip; | ||
| pip install adamapi | ||
| ln -s "/usr/lib/python3/dist-packages/osgeo" "${VENVNAME}/lib/python3.8/site-packages/osgeo" | ||
| ``` | ||
| # API DEFINITIONS | ||
| This document briefly describes the ADMAPI functionalities.<br> | ||
| The ADAMAPI library is divided in 4 modules: | ||
| 1. Auth --> the authorization module | ||
| 2. Datasets --> to get the list of datasets | ||
| 3. Search --> to get the lists of products, including associated metadata (e.g. geometry, cloud cover, orbit, tile, ...) | ||
| 4. GetData --> to retrieve the product(s), including options to subset in space and time in the native data granularity and reduced processing capacity | ||
| ## 1 - Auth | ||
| This module takes care of user authentication and authorization.<br> | ||
| Without instancing an object of this module other components don't work.<br> | ||
| Auth module is based on the ADAMAPI_KEY, a key that uniquelly identifies the user. | ||
| **Class contructor and parameters** | ||
| ```python | ||
| from adamapi import Auth | ||
| a = Auth() | ||
| ``` | ||
| Parameters:<br> | ||
| | position/keyword | mandatory | type | default | description | | ||
| | ------ | ------ | ------ | ------ | ------ | | ||
| | | | | | | | ||
| **Public methods and parameters** | ||
| * **.setKey()** --> To setup the ADAMAPI_KEY<br> | ||
| Parameters: | ||
| | position/keyword | mandatory | type | default | description | | ||
| | ------ | ------ | ------ | ------ | ------ | | ||
| | 0 | True | str | | The ADAMAPI_KEY | | ||
| * **.setAdamCore()** --> To setup the url of the ADAM-CORE endpoint<br> | ||
| Parameters: | ||
| | position/keyword | mandatory | type | default | description | | ||
| | ------ | ------ | ------ | ------ | ------ | | ||
| | 0 | True | str | | The url like https://test.adamplatform.eu | | ||
| * **.authorize()** --> to instanciate an auth object<br> | ||
| Parameters: | ||
| | position/keyword | mandatory | type | default | description | | ||
| | ------ | ------ | ------ | ------ | ------ | | ||
| | | | | | | | ||
| * **.getAuthToken()** --> to get the authorization token<br> | ||
| Parameters: | ||
| | position/keyword | mandatory | type | default | description | | ||
| | ------ | ------ | ------ | ------ | ------ | | ||
| | | | | | | | ||
| ### 1.1 - ADAMAPI_KEY retrieval | ||
| To get the ADAMAPI_KEY, you need to access your ADAM portal and:<br> | ||
| 1. Select the "user icon" on the top right | ||
| 2. Expand / click the "USERNAME" | ||
| 3. Click on the "Api Key" to display your key | ||
| <br> | ||
| *Command-line ADAMAPI_KEY retrieval TBP* | ||
| ### 1.2 - ADAMAPI_KEY setup | ||
| There are three methods to setup the ADAMAPI_KEY and the ADAM-CORE instance: | ||
| 1. use the method setKey() and setAdamCore() | ||
| ```python | ||
| from adamapi import Auth | ||
| a = Auth() | ||
| a.setKey('<ADAMAPI_KEY>') | ||
| a.setAdamCore('https://test.adamplatform.eu') | ||
| ``` | ||
| 2. Export two envars like | ||
| ```bash | ||
| #open a Terminal and type: | ||
| export ADAMAPI_KEY='<ADAMAPI_KEY>' | ||
| export ADAMAPI_URL='https://test.adamplatform.eu' | ||
| ``` | ||
| 3. create a file called **.adamapirc** in the user home directory with the following content | ||
| ```text | ||
| key=<ADAMAPI_KEY> | ||
| url=https://test.adamplatform.eu | ||
| ``` | ||
| ### 1.3 - Examples | ||
| After ADAMAPI_KEY has been set up, an auth instance can be created with: | ||
| ```python | ||
| from adamapi import Auth | ||
| a = Auth() | ||
| a.authorize() | ||
| ``` | ||
| After authorize method you can retrive your autho token: | ||
| ```python | ||
| from adamapi import Auth | ||
| a = Auth() | ||
| a.authorize() | ||
| a.getAuthToken() | ||
| ``` | ||
| ## 2 - Datasets | ||
| This module provides datasets discovery functionality. | ||
| **Class contructor and parameters** | ||
| ```python | ||
| from adamapi import Datasets | ||
| datasets = Datasets( a ) | ||
| ``` | ||
| Parameters:<br> | ||
| | position/keyword | mandatory | type | default | description | | ||
| | ------ | ------ | ------ | ------ | ------ | | ||
| | 0 | True | Auth instance | | The ADAMAPI authorized instance obtained in the previous section | | ||
| **Public methods and parameters** | ||
| * **.getDatasets()** --> To retrieve datasets list <br> | ||
| Parameters: | ||
| | position/keyword | mandatory | type | default | description | | ||
| | ------ | ------ | ------ | ------ | ------ | | ||
| | 0 | False | str | | The datasetId. | | ||
| | page | False | numeric | 0 | Indicats a specific page | | ||
| | maxRecords | False | numeric | 10 | Max number of results in output. | | ||
| This .getDatasets() function can be used to retrive additional filters which are described in the key **filtersEnabled** (if exists). | ||
| ### 2.1 Examples | ||
| This module can be used in 2 different ways. | ||
| 1. To list all available datasets: | ||
| ```python | ||
| datasets = Datasets(a) | ||
| print(datasets.getDatasets()) | ||
| ``` | ||
| 2. To get detailed metadata about a specific dataset | ||
| ```python | ||
| datasets = Datasets(a) | ||
| print( datasets.getDatasets( '{{ID:DATASET}}' , page=0 , maxRecords=10 ) ) | ||
| ``` | ||
| 3. To get filtersEnabled. To use this additional filters see first example in Search section. | ||
| ```python | ||
| datasets = Datasets(a) | ||
| out=datasets.getDatasets("{{ID:DATASET}}") | ||
| print(out["filtersEnabled"]) | ||
| ``` | ||
| ## 3 - Search | ||
| This module provides discovery functionality through the products available on the ADAM instance. | ||
| **Class contructor and parameters** | ||
| ```python | ||
| from adamapi import Search | ||
| search = Search( a ) | ||
| ``` | ||
| Parameters:<br> | ||
| | position/keyword | mandatory | type | default | description | | ||
| | ------ | ------ | ------ | ------ | ------ | | ||
| | 0 | True | Auth instance | | The ADAMAPI authorized instance obtained in section 1-Auth | | ||
| **Public methods and parameters** | ||
| * **.getProducts()** --> To retrieve datasets list and metadata<br> | ||
| Parameters: | ||
| | position/keyword | mandatory | type | default | description | | ||
| | ------ | ------ | ------ | ------ | ------ | | ||
| | 0 | True | str | | The datasetId. | | ||
| | maxRecords | False | int | 10 | number of records | | ||
| | startIndex | False | int | 0 | starting record index | | ||
| | startDate | False | str or [datetime](https://docs.python.org/3/library/datetime.html) | | the start date | | ||
| | endDate | False | str or [datetime](https://docs.python.org/3/library/datetime.html) | | the end date | | ||
| | geometry | True | str or geojson | | GeoJson geometry,[geojson format](https://tools.ietf.org/html/rfc7946) [appendix](#geometry)| | ||
| ### 3.1 Examples | ||
| 1. Example1: | ||
| ```python | ||
| search=Search(a) | ||
| mongo_search=search.getProducts('{{ID:DATASET}}',maxRecords=1,startIndex=0,platform="{{VALUE}}") | ||
| ``` | ||
| 2. Example2: | ||
| ```python | ||
| search=Search(a) | ||
| mongo_search=search.getProducts('{{ID:DATASET}}',maxRecords=1,startIndex=0) | ||
| ``` | ||
| ## 4 - GetData | ||
| This module provides data access of raster, spatial subset, timeseries in the native data granularity and reduced processing capacity. | ||
| **Class contructor and parameters** | ||
| ```python | ||
| from adamapi import GetData | ||
| data=GetData(a) | ||
| ``` | ||
| Parameters:<br> | ||
| | position/keyword | mandatory | type | default | description | | ||
| | ------ | ------ | ------ | ------ | ------ | | ||
| | 0 | True | Auth Instance | | The ADAMAPI authorized instance obtained in the section 1-Auth | | ||
| **Public methods and parameters** | ||
| * **.getData()** --> To retrieve a specific product or a dataset in its native granularity, to get a subset of it, to perform a timeseries or to exec simple processing <br> | ||
| | position/keyword | mandatory | type | default | description | | ||
| | ------ | ------ | ------ | ------ | ------ | | ||
| | 0 | True | str | | The datasetId | | ||
| | 1 | True | str | GetFile | request type. available values: GetFile,GetSubset, GetTimeseries and GetProcessing | | ||
| | asynchronous | False | boolean | False | rappesents how the request will be performed | | ||
| | compress | False | boolean | False | return a zip file | | ||
| | rest | False | boolean | True | perform RESTful order ignoring explorer state on the server and equalization configured using the explorer gui | | ||
| | filters | True | json | {} | json object with filters parameter. startDate and endDate are required inside it | | ||
| | options | False | json | {} | request option | | ||
| | outputDir | False | str | `adamapiresults/` | set a different download directory inside `adamapiresult/` main directory | | ||
| ### 4.1 Examples | ||
| ```python | ||
| data=GetData(a) | ||
| #to retrive a specific product | ||
| image = data.getData('{{ID:DATASET}}',"GetFile",asynchronous=False,compress=False,rest=False,filters={"startDate":'{{STARTDATE}}',"endDate":'{{ENDDATE}}',"productId":'{{PRODUCTID}}'},outputDir='{{OUTPUT_DIR}}') | ||
| #to retrieve a dataset in its native granularity | ||
| data=GetData(self.a) | ||
| image = data.getData('{{ID:DATASET}}',"GetFile",asynchronous=False,compress=False,rest=False,filters={"startDate":'{{STARTDATE}}',"endDate":'{{ENDDATE}}',"geometry":'{{GEOMETRY}}'},outputDir='{{OUTPUT_DIR}}') | ||
| ``` | ||
| For the GetSubset,GetTimeseries and GetProcessing requests you need to add the `options` parameter with these constraints : [output formats](#output-formats) and [functions](#processing-function)(only for processing request) | ||
| ```python | ||
| #subset example | ||
| image = data.getData('{{ID:DATASET}}',"GetSubset",asynchronous=False,compress=False,rest=False,filters={"startDate":'{{STARTDATE}}',"endDate":'{{ENDDATE}}',"geometry":'{{GEOMETRY}}'},options={"format":'{{FORMATS}}'},outputDir='{{OUTPUT_DIR}}') | ||
| #timeseries example | ||
| image = data.getData('{{ID:DATASET}}',"GetTimeseries",asynchronous=False,compress=False,rest=False,filters={"startDate":'{{STARTDATE}}',"endDate":'{{ENDDATE}}',"geometry":'{{GEOMETRY}}'},options={"format":'{{FORMATS}}'},outputDir='{{OUTPUT_DIR}}') | ||
| #processing example | ||
| image = data.getData('{{ID:DATASET}}',"GetProcessing",asynchronous=False,compress=False,rest=False,filters={"startDate":'{{STARTDATE}}',"endDate":'{{ENDDATE}}',"geometry":'{{GEOMETRY}}'},options={"format":'{{FORMAT}}',"function":'{{FUNCTION}}'},outputDir='{{OUTPUT_DIR}}') | ||
| ``` | ||
| ### 4.3 Asyncronous Example | ||
| ```python | ||
| #1. execute the request | ||
| image = data.getData('{{ID:DATASET}}',"GetSubset",asynchronous=False,compress=False,rest=False,filters={"startDate":'{{STARTDATE}}',"endDate":'{{ENDDATE}}',"geometry":'{{GEOMETRY}}'},options={"format":'{{FORMATS}}'},outputDir='{{OUTPUT_DIR}}') | ||
| #2. check the status | ||
| stat=data.getData(datasetId,"GetSubset",asynchronous=True,id=str(image.pk)) | ||
| while stat.status != "completed": | ||
| time.sleep(1) | ||
| stat=data.getData(datasetId,"GetSubset",asynchronous=True,id=str(image.pk)) | ||
| #3. download the zip,unzip it and remove the zip (optional) | ||
| for res in stat.list: | ||
| if res["status"] == "failed": | ||
| print(res["exit_code"]) | ||
| else: | ||
| r=self.a.client(res["download"]["url"],{},"GET") | ||
| with open(str(res["download"]["url"].split("/")[4])+"_"+str(res["download"]["url"].split("/")[5]), 'wb' ) as f: | ||
| f.write( r.content ) | ||
| ``` | ||
| # Appendix 1 - Data format | ||
| ## date and date+time | ||
| Supported string date/date+time format are: | ||
| * '%Y-%m-%dT%H:%M:%S', | ||
| * '%Y-%m-%dT%H:%M:%SZ', | ||
| * '%Y-%m-%d' | ||
| ### GeoJson | ||
| Geometry have to follow the latest geojson standard [rfc7946](https://tools.ietf.org/html/rfc7946)<br> | ||
| In particular Polygons and MultiPolygons should follow the right-hand rule<br> | ||
| ### Geometry | ||
| ```python | ||
| #This geometry will return all the results it has intersected within it | ||
| geometry = { "type": "Polygon", "coordinates": [ [ [ 43.916666667, 15.716666667 ], [ 43.916666667, 15.416666667 ] , [ 44.216666667, 15.416666667 ], [ 44.216666667, 15.716666667 ], [ 43.916666667, 15.716666667 ] ] ] } | ||
| ``` | ||
| ```python | ||
| #This geometry will return all the results it has intersected on its outside | ||
| geometry = { "type": "Polygon", "coordinates": [ [ [ 43.84986877441406,15.925676536359038 ], [ 44.6539306640625,15.950766025306109 ],[ 44.681396484375,15.194084972583916 ], [ 43.8189697265625,15.20998780073036 ], [ 43.84986877441406,15.925676536359038 ] ] ] } | ||
| ``` | ||
| ### Output Formats | ||
| | request | output format | | ||
| | ---- | ---- | | ||
| | GetFile | - | | ||
| | GetSubset | tiff,png | | ||
| | GetTimeseries | json,csv | | ||
| | GetProcessing | tiff,png | | ||
| ### Processing Function | ||
| | type | description | | ||
| | ---- | ---- | | ||
| | average | execute the average from the products | | ||
| | overlap | execute the products overlap without any specific strategy | | ||
| | mosterecent | puts on the top the most recent product | | ||
| | leastrecent | puts on top the least recent product | | ||
| | minvalue | for each pixel, puts on top the minimum value of the pixel | | ||
| | maxvalue | for each pixel, puts on top the maximum value of the pixel | |
| requests>=2.22.0 | ||
| imageio==2.9.0 | ||
| tqdm==4.62.3 | ||
| numpy==1.21.2 | ||
| python-dateutil==2.8.2 | ||
| matplotlib==3.4.3 | ||
| certifi==2021.5.30 | ||
| charset-normalizer==2.0.6 | ||
| cycler==0.10.0 | ||
| idna==3.2 | ||
| kiwisolver==1.3.2 | ||
| Pillow==8.3.2 | ||
| pyparsing==2.4.7 | ||
| six==1.16.0 | ||
| urllib3==1.26.7 |
| LICENSE | ||
| README.md | ||
| pyproject.toml | ||
| setup.py | ||
| src/adamapi/__init__.py | ||
| src/adamapi/authorization.py | ||
| src/adamapi/datasets.py | ||
| src/adamapi/get_data.py | ||
| src/adamapi/search.py | ||
| src/adamapi.egg-info/PKG-INFO | ||
| src/adamapi.egg-info/SOURCES.txt | ||
| src/adamapi.egg-info/dependency_links.txt | ||
| src/adamapi.egg-info/requires.txt | ||
| src/adamapi.egg-info/top_level.txt |
| adamapi |
| """ | ||
| Copyright (c) 2020 MEEO s.r.l. | ||
| Permission is hereby granted, free of charge, to any person obtaining a copy | ||
| of this software and associated documentation files (the "Software"), to deal | ||
| in the Software without restriction, including without limitation the rights | ||
| to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | ||
| copies of the Software, and to permit persons to whom the Software is | ||
| furnished to do so, subject to the following conditions: | ||
| The above copyright notice and this permission notice shall be included in all | ||
| copies or substantial portions of the Software. | ||
| THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | ||
| IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | ||
| FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | ||
| AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER | ||
| LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, | ||
| OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE | ||
| SOFTWARE. | ||
| """ | ||
| class AdamApiError( Exception ): | ||
| """Base class for exceptions on the adamapi module.""" | ||
| pass | ||
| class AdamApiMessage( Exception ): | ||
| """Base class for adamapi mesage""" | ||
| def __init__(self,json): | ||
| self.pk=json["id"] if "id" in json else "" | ||
| self.status=json["status"] if "status" in json else "" | ||
| self.location = json["location"] if "location" in json else "" | ||
| self.error = json["error"] if "error" in json else "" | ||
| self.list = json["tasks"] if "tasks" in json else "" | ||
| from . import authorization | ||
| from . import datasets | ||
| from . import get_data | ||
| from . import search | ||
| Auth = authorization.Auth | ||
| Datasets = datasets.Datasets | ||
| GetData = get_data.GetData | ||
| Search = search.Search |
| """ | ||
| Copyright (c) 2022 MEEO s.r.l. | ||
| Permission is hereby granted, free of charge, to any person obtaining a copy | ||
| of this software and associated documentation files (the "Software"), to deal | ||
| in the Software without restriction, including without limitation the rights | ||
| to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | ||
| copies of the Software, and to permit persons to whom the Software is | ||
| furnished to do so, subject to the following conditions: | ||
| The above copyright notice and this permission notice shall be included in all | ||
| copies or substantial portions of the Software. | ||
| THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | ||
| IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | ||
| FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | ||
| AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER | ||
| LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, | ||
| OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE | ||
| SOFTWARE. | ||
| """ | ||
| import os | ||
| import requests | ||
| import re | ||
| import logging | ||
| from datetime import datetime,timedelta | ||
| logger = logging.getLogger( 'adamapi' ) | ||
| from . import AdamApiError | ||
| class Auth(): | ||
| def __init__( self, key=None, url=None ): | ||
| """ | ||
| Authorization class used to manage users credentials | ||
| """ | ||
| self.key = key | ||
| self.url = url | ||
| self.authorized = False | ||
| self.access_obj=None | ||
| self.lock=False | ||
| def authorize( self ): | ||
| """ | ||
| try to autorize the adamapi to the target adam instance | ||
| """ | ||
| #verify if key are valid | ||
| self._check() | ||
| url = os.path.join( self.url,'apis','v1','authorization', 'adamapi' ) | ||
| headers={ 'Authorization': self.key } | ||
| r=requests.post( url,data={},headers=headers ) | ||
| res = r.json() | ||
| if r.status_code !=200: | ||
| raise AdamApiError( res['title'] ) | ||
| else: | ||
| logger.info( res ) | ||
| self.authorized = True | ||
| self.access_obj= res | ||
| return res | ||
| def getAuthToken(self): | ||
| return { | ||
| "authorization_token" : self.access_obj["access_token"], | ||
| "expires_at" : self.access_obj["expires_at"] | ||
| } | ||
| def _check( self ): | ||
| """ | ||
| verify if the user key and the adam url are valid | ||
| """ | ||
| self.getKey() | ||
| self.getAdamCore() | ||
| if self.key is None: | ||
| raise AdamApiError( "key can not be None" ) | ||
| if self.url is None: | ||
| raise AdamApiError( "url (adam endpoint) can not be None" ) | ||
| def setKey( self, key ): | ||
| """ | ||
| set the user key | ||
| """ | ||
| self.key = key | ||
| def setAdamCore( self, url ): | ||
| """ | ||
| set the adam endpoint url | ||
| """ | ||
| self.url = url | ||
| def getKey( self ): | ||
| """ | ||
| key priority: | ||
| 1 - use the key declared on this module | ||
| 2 - use a key exported as envvars | ||
| 3 - try to retrieve the key from $HOME/.adamapirc | ||
| """ | ||
| if self.key is not None: | ||
| pass | ||
| elif os.environ.get( 'ADAMAPI_KEY') is not None: | ||
| self.key = os.environ.get( 'ADAMAPI_KEY' ) | ||
| logger.debug( 'Get key from envvar' ) | ||
| else: | ||
| try: | ||
| self.key = self._getRc()[ 'key' ] | ||
| logger.debug( 'Get key from adamapirc' ) | ||
| except: | ||
| None | ||
| if self.key is None: #again | ||
| logger.warning( "API key not specified" ) | ||
| return self.key | ||
| def getAdamCore( self ): | ||
| """ | ||
| adam endpoint priority: | ||
| 1 - use the url declared on this module | ||
| 2 - use a url exported as envvars | ||
| 3 - try to retrieve the url from $HOME/.adamapirc | ||
| """ | ||
| if self.url is not None: | ||
| pass | ||
| elif os.environ.get( 'ADAMAPI_URL') is not None: | ||
| self.url = os.environ.get( 'ADAMAPI_URL' ) | ||
| logger.debug( 'Get url from envvar' ) | ||
| else: | ||
| try: | ||
| self.url = self._getRc()[ 'url' ] | ||
| logger.debug( 'Get url from adamapirc' ) | ||
| except: | ||
| None | ||
| if self.url is None: #again | ||
| logger.warning( "ADAM API endpoint not specified" ) | ||
| return self.url | ||
| def _getRc( self ): | ||
| """ | ||
| check if ~/.adamapirc file exist ant try to extract url and key from it | ||
| """ | ||
| api_rc = os.environ.get( 'ADAMAPI_RC', os.path.expanduser( '~/.adamapirc' ) ) | ||
| config_dict = {} | ||
| if os.path.isfile( api_rc ): | ||
| with open( api_rc ) as f: | ||
| for line in f.readlines(): | ||
| line = line.strip() | ||
| if line.lstrip( '#' ) == line: | ||
| try: | ||
| key, val = line.split( '=', 1 ) | ||
| config_dict[ key.strip() ] = val.strip() | ||
| except: | ||
| logger.warning( 'Invalid settings in "%s" - %s' %( api_rc, line ) ) | ||
| return config_dict | ||
| def _checktokenExpire(self): | ||
| """ | ||
| check if token is expired | ||
| """ | ||
| if (datetime.strptime(self.access_obj['expires_at'],"%Y-%m-%dT%H:%M:%S.%fZ") - datetime.utcnow().replace(microsecond=0)) <= timedelta(minutes=15) : | ||
| return True | ||
| else: | ||
| return False | ||
| def _refreshToken(self): | ||
| """ | ||
| refresh tocken obj | ||
| """ | ||
| self.lock=True | ||
| params = {} | ||
| params["refresh_token"]=self.access_obj["refresh_token"] | ||
| self.access_obj = self.client(os.path.join( 'apis','v1', 'authorization', 'refresh' ), params, 'PUT').json() | ||
| self.lock=False | ||
| def client( self, query, params, request_type, authorization_required = True , force_raise = False ,enable_stream=False): | ||
| """ | ||
| The client method that perform the request appending the Authorization header | ||
| """ | ||
| #verify if the user already call the authorize method of the Auth instance | ||
| #if authorization_required and not self.authorized: | ||
| # self.authorize() | ||
| if not self.authorized and self.access_obj is not None : | ||
| raise AdamApiError("Unauthorized") | ||
| if not self.lock and self._checktokenExpire(): | ||
| self._refreshToken() | ||
| #verify if the passed url already include the host or if the adamcore url have to be prepended | ||
| if re.match( r'^http[s]?://.+', query ): | ||
| url = query | ||
| else: | ||
| url = os.path.join( self.url, query ) | ||
| if request_type not in [ 'GET', 'POST', 'PUT' ]: | ||
| raise AdamApiError( "Request type not supported" ) | ||
| logger.debug( "%s request started", request_type ) | ||
| if "\\" in url: | ||
| url = url.replace("\\","/") | ||
| if request_type == 'GET': | ||
| par = params.copy() | ||
| par["token"] = self.access_obj["access_token"] | ||
| r=requests.get( url, params=par ,stream=enable_stream) | ||
| elif request_type == 'POST': | ||
| par = params.copy() | ||
| par["token"] = self.access_obj["access_token"] | ||
| r=requests.post( url,data=par ) | ||
| elif request_type == 'PUT': | ||
| r=requests.put( url,data=params ) | ||
| logger.info(r.status_code) | ||
| if r.status_code in range(400,599): | ||
| if force_raise: | ||
| return r.raise_for_status() | ||
| else: | ||
| return r | ||
| else: | ||
| return r |
| """ | ||
| Copyright (c) 2022 MEEO s.r.l. | ||
| Permission is hereby granted, free of charge, to any person obtaining a copy | ||
| of this software and associated documentation files (the "Software"), to deal | ||
| in the Software without restriction, including without limitation the rights | ||
| to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | ||
| copies of the Software, and to permit persons to whom the Software is | ||
| furnished to do so, subject to the following conditions: | ||
| The above copyright notice and this permission notice shall be included in all | ||
| copies or substantial portions of the Software. | ||
| THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | ||
| IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | ||
| FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | ||
| AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER | ||
| LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, | ||
| OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE | ||
| SOFTWARE. | ||
| """ | ||
| import os | ||
| import logging | ||
| logger=logging.getLogger('adamapi') | ||
| from . import AdamApiError | ||
| class Datasets(): | ||
| def __init__(self, client): | ||
| self.client = client | ||
| self.LOG = logger | ||
| def getDatasets(self,datasetId=None,**kwargs): | ||
| params={} | ||
| params["client"]="adamapi" | ||
| if datasetId: | ||
| url=os.path.join("apis","v2","datasets",datasetId.split(":")[0]) | ||
| else: | ||
| url=os.path.join("apis","v2","datasets","list") | ||
| if 'page' in kwargs: | ||
| params['page']=kwargs['page'] | ||
| else: | ||
| params['page']=0 | ||
| if 'maxRecords' in kwargs: | ||
| params['maxRecords']=kwargs['maxRecords'] | ||
| else: | ||
| params['maxRecords']=10 | ||
| r = self.client.client( url, params, 'GET' ) | ||
| self.LOG.info( 'Datasets request executed' ) | ||
| response = r.json() | ||
| return response |
| """ | ||
| Copyright (c) 2022 MEEO s.r.l. | ||
| Permission is hereby granted, free of charge, to any person obtaining a copy | ||
| of this software and associated documentation files (the "Software"), to deal | ||
| in the Software without restriction, including without limitation the rights | ||
| to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | ||
| copies of the Software, and to permit persons to whom the Software is | ||
| furnished to do so, subject to the following conditions: | ||
| The above copyright notice and this permission notice shall be included in all | ||
| copies or substantial portions of the Software. | ||
| THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | ||
| IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | ||
| FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | ||
| AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER | ||
| LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, | ||
| OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE | ||
| SOFTWARE. | ||
| """ | ||
| import os | ||
| import time | ||
| from osgeo import ogr | ||
| import sys | ||
| import json | ||
| from tqdm import tqdm | ||
| import errno | ||
| import requests | ||
| import zipfile | ||
| from datetime import datetime | ||
| import re | ||
| import logging | ||
| logger=logging.getLogger('adamapi') | ||
| from . import AdamApiError,AdamApiMessage | ||
| class GetData(object): | ||
| def __init__(self,client): | ||
| self.LOG=logger | ||
| self.client=client | ||
| def getData(self,datasetId,request,asynchronous=False,compress=False,rest=True,filters={},options={},**kwargs): | ||
| """ | ||
| Method to retrive: | ||
| 1) data in native format and native temporal resolution | ||
| 2) a subset of data in native format and native temporal resolution | ||
| 3) a timeseries of data in native format and native temporal resolution | ||
| 4) a processing on a specific datasetId | ||
| @datasetId required | ||
| @asynchronous required (default False) | ||
| @startDate required | ||
| @endDate required | ||
| @schema optional ( required only for subset,timeseries and processing ) | ||
| @geometry optional ( if schema is set, geometry param skipped in favor of the geometry param inside the schema if set ) | ||
| @outputDir optional ( set a different download directory inside adamapiresult main directory | ||
| """ | ||
| #version 2.1-dev | ||
| #step 1 check params | ||
| #step 2 manage synchronously and asynchronously different temporal resolution datasets in the same way | ||
| #asynchronous management | ||
| if "outputDir" in kwargs: | ||
| output_dir = os.path.join("adamapiresult",kwargs["outputDir"]) | ||
| else: | ||
| output_dir = "adamapiresult/" | ||
| self._checkDir(output_dir) | ||
| params,fname=self.checkParams( request,asynchronous, compress,rest,filters,options,kwargs) | ||
| startIndex=0 | ||
| maxRecords=10 | ||
| if asynchronous == True and "id" in params: | ||
| #get the order status | ||
| url=os.path.join( "apis", "v2", "orders", datasetId.split(":",1)[0], str(params["id"]) ) | ||
| response_order_status=self.client.client(url,params,"GET",force_raise=True,enable_stream=False) | ||
| if response_order_status.status_code == 200: | ||
| return AdamApiMessage(response_order_status.json()) | ||
| else: | ||
| raise AdamApiError(response_order_status.json()) | ||
| #request management - the order submit url | ||
| url=os.path.join( "apis", "v2", "orders", datasetId.split(":",1)[0] ) | ||
| #POST the order requests | ||
| response_order_submit = self.client.client(url,params,"POST",force_raise=False,enable_stream=False) | ||
| if response_order_submit.status_code==200: | ||
| if asynchronous == False: | ||
| #get the order status | ||
| url=os.path.join( "apis", "v2", "orders", datasetId.split(":",1)[0], str(response_order_submit.json()["id"]) ) | ||
| response_order_status=self.client.client(url,{},"GET",force_raise=True,enable_stream=False) | ||
| if response_order_status.status_code == 200: | ||
| while response_order_status.json()["status"] != "completed" and "tasks" not in response_order_status.json(): | ||
| time.sleep(1) | ||
| response_order_status=self.client.client(url,params,"GET",force_raise=True,enable_stream=False) | ||
| for elem in tqdm(response_order_status.json()["tasks"]): | ||
| url=elem[ "download"]["url"] | ||
| response_download=self.client.client(url,{},"GET",force_raise=False,enable_stream=False) | ||
| if response_download.status_code != 200: | ||
| print(AdamApiMessage(response_download.json())) | ||
| else: | ||
| outname=re.findall('filename="(.+)"', response_download.headers['content-disposition'])[0] | ||
| with open(os.path.join(output_dir,outname), 'wb' ) as f: | ||
| f.write( response_download.content ) | ||
| return AdamApiMessage(response_order_status.json()) | ||
| else: | ||
| return AdamApiMessage(response_order_submit.json()) | ||
| else: | ||
| raise AdamApiError(response_order_submit.json()) | ||
| def checkParams(self, request,asynchronous, compress,rest,filters,options,kwargs): | ||
| params={} | ||
| fname=None | ||
| op_kwargs = kwargs.copy() | ||
| #manage outDir | ||
| if "outputDir" in op_kwargs: | ||
| del(op_kwargs["outputDir"]) | ||
| #manage filters | ||
| params["filters"] = {} | ||
| filt = {} | ||
| for(key,value)in filters.items(): | ||
| if key == "geometry": | ||
| if isinstance(value,str): | ||
| filt[key]=value | ||
| else: | ||
| filt[key]=json.dumps(value) | ||
| else: | ||
| filt[key]=value | ||
| #manage options | ||
| opt = {} | ||
| for(key,value)in options.items(): | ||
| opt[key]=value | ||
| for (key,value) in op_kwargs.items(): | ||
| params[key] = value | ||
| #check date | ||
| if "startDate" in filt and not self._checkDate(filt["startDate"]): | ||
| raise AdamApiError("Parameter Error. Invalid date format [%Y-%m-%d,%Y-%m-%dT%H:%M:%SZ,%Y-%m-%dT%H:%M:%S]") | ||
| if "endDate" in filt and not self._checkDate(filt["endDate"]): | ||
| raise AdamApiError("Parameter Error. Invalid date format [%Y-%m-%d,%Y-%m-%dT%H:%M:%SZ,%Y-%m-%dT%H:%M:%S]") | ||
| if "startDate" in filt and "endDate" in filt: | ||
| if filt["startDate"] > filt["endDate"]: | ||
| raise AdamApiError("Parameter Error. startDate > endDate") | ||
| #check Getfile | ||
| if "format" in opt and request == "GetFile": | ||
| raise AdamApiError("invalid option for GetFile request. remove format") | ||
| params["options"] = json.dumps(opt) | ||
| params["filters"] = json.dumps(filt) | ||
| params["client"] = "adamapi" | ||
| params["rest"] = rest | ||
| params["compress"] = compress | ||
| params["request"] = request | ||
| params["asynchronous"] = asynchronous | ||
| return params,fname | ||
| def _checkDate(self,date): | ||
| test=False | ||
| pattern=[ | ||
| "%Y-%m-%d", | ||
| "%Y-%m-%dT%H:%M:%SZ", | ||
| "%Y-%m-%dT%H:%M:%S" | ||
| ] | ||
| for p in pattern: | ||
| try: | ||
| datetime.strptime(date,p) | ||
| test=True | ||
| except: | ||
| pass | ||
| return test | ||
| def _checkDir(selfi,dirname): | ||
| try: | ||
| os.makedirs( dirname ) | ||
| except OSError as ose: | ||
| if ose.errno!=errno.EEXIST: | ||
| raise AdamApiError( ose ) |
| """ | ||
| Copyright (c) 2022 MEEO s.r.l. | ||
| Permission is hereby granted, free of charge, to any person obtaining a copy | ||
| of this software and associated documentation files (the "Software"), to deal | ||
| in the Software without restriction, including without limitation the rights | ||
| to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | ||
| copies of the Software, and to permit persons to whom the Software is | ||
| furnished to do so, subject to the following conditions: | ||
| The above copyright notice and this permission notice shall be included in all | ||
| copies or substantial portions of the Software. | ||
| THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | ||
| IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | ||
| FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | ||
| AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER | ||
| LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, | ||
| OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE | ||
| SOFTWARE. | ||
| """ | ||
| from datetime import datetime | ||
| import os | ||
| import logging | ||
| import json | ||
| logger=logging.getLogger('adamapi') | ||
| from . import AdamApiError | ||
| class Search(): | ||
| def __init__(self,client): | ||
| self.LOG=logger | ||
| self.client=client | ||
| def getProducts(self,datasetId,startIndex=0,maxRecords=10,**kwargs): | ||
| """ | ||
| Product Catalogue Search | ||
| @ datasetId, the dataseId ,required | ||
| @ startDate, search start date (%Y-%m-%d) | ||
| @ endDate, search end date (%Y-%m-%d) | ||
| @ maxRecords, number of result showed | ||
| @ startIndex, index of the first result to start showin | ||
| """ | ||
| params={} | ||
| url=os.path.join( 'apis','v2','opensearch',datasetId.split(":",1)[0],'records') | ||
| params['datasetId']=datasetId.split(":",1)[1] | ||
| params["client"]="adamapi" | ||
| for par in kwargs: | ||
| if par == 'geometry': | ||
| if type(kwargs[par]) is str: | ||
| params[par] = kwargs[par] | ||
| elif type(kwargs[par]) is dict: | ||
| params[par] = json.dumps(kwargs[par]) | ||
| elif par == 'startDate' and isinstance( kwargs[par], datetime ): | ||
| params[ 'startDate' ] = kwargs[par].strftime( '%Y-%m-%dT%H:%M:%SZ' ) | ||
| elif par == 'endDate' and isinstance( kwargs[par], datetime ): | ||
| params[ 'endDate' ] = kwargs[par].strftime( '%Y-%m-%dT%H:%M:%SZ' ) | ||
| else: | ||
| params[par]=kwargs[par] | ||
| params["page"]=startIndex | ||
| params["items"]=maxRecords | ||
| if 'geometry' in kwargs: | ||
| self.LOG.info(params) | ||
| response=self.client.client(url,params,"POST").json() | ||
| else: | ||
| response=self.client.client(url,params,"GET").json() | ||
| if 'errors' in response: | ||
| raise AdamApiError(response['errors']) | ||
| else: | ||
| return response |
+337
-301
| Metadata-Version: 2.1 | ||
| Name: adamapi | ||
| Version: 2.0.11 | ||
| Version: 2.1.0 | ||
| Summary: Python Adam API | ||
@@ -8,303 +8,3 @@ Home-page: https://git.services.meeo.it/das/adamapi | ||
| Author-email: info@meeo.it | ||
| License: UNKNOWN | ||
| Description: # Installation | ||
| ### Versioning | ||
| 1. adamapi==1.0.1 , This package works only with ADAMCORE 1. | ||
| 2. adamapi==2.0.11, This pachage works only with ADAMCORE 2. | ||
| ## Requirements | ||
| ```bash | ||
| sudo apt-get install python3-venv python3-gdal gdal-bin | ||
| ``` | ||
| ## Install with pip | ||
| ```bash | ||
| VENVNAME="adamapi" | ||
| python3 -m venv "${VENVNAME}" | ||
| PYTHONVERSION=$(ls ${VENVNAME}/lib/) | ||
| source "${VENVNAME}/bin/activate"; | ||
| python3 -m pip install --upgrade pip; | ||
| pip install adamapi==2.0.1r1 | ||
| ln -s "/usr/lib/python3/dist-packages/osgeo" "${VENVNAME}/lib/${PYTHONVERSION}/site-packages/osgeo" | ||
| ``` | ||
| # API DEFINITIONS | ||
| This document briefly describes the ADMAPI functionalities.<br> | ||
| The complete and interactive documentation is provided through a jupyter notebook *(LINK to notebook TBP)*.<br> | ||
| The ADAMAPI library is divided in 4 modules: | ||
| 1. Auth --> the authorization module | ||
| 2. Datasets --> to get the list of datasets | ||
| 3. Search --> to get the lists of products, including associated metadata (e.g. geometry, cloud cover, orbit, tile, ...) | ||
| 4. GetData --> to retrieve the product(s), including options to subset in space and time | ||
| ## 1 - Auth | ||
| This module takes care of user authentication and authorization.<br> | ||
| Without instancing an object of this module other components don't work.<br> | ||
| Auth module is based on the ADAMAPI_KEY, a key that uniquelly identifies the user. | ||
| **Class contructor and parameters** | ||
| ```python | ||
| from adamapi import Auth | ||
| a = Auth() | ||
| ``` | ||
| Parameters:<br> | ||
| | position/keyword | mandatory | type | default | description | | ||
| | ------ | ------ | ------ | ------ | ------ | | ||
| | | | | | | | ||
| **Public methods and parameters** | ||
| * **.setKey()** --> To setup the ADAMAPI_KEY<br> | ||
| Parameters: | ||
| | position/keyword | mandatory | type | default | description | | ||
| | ------ | ------ | ------ | ------ | ------ | | ||
| | 0 | True | str | | The ADAMAPI_KEY | | ||
| * **.setAdamCore()** --> To setup the url of the ADAM-CORE endpoint<br> | ||
| Parameters: | ||
| | position/keyword | mandatory | type | default | description | | ||
| | ------ | ------ | ------ | ------ | ------ | | ||
| | 0 | True | str | | The url like https://test.adamplatform.eu | | ||
| * **.authorize()** --> to instanciate an auth object<br> | ||
| Parameters: | ||
| | position/keyword | mandatory | type | default | description | | ||
| | ------ | ------ | ------ | ------ | ------ | | ||
| | | | | | | | ||
| * **.getAuthToken()** --> to get the authorization token<br> | ||
| Parameters: | ||
| | position/keyword | mandatory | type | default | description | | ||
| | ------ | ------ | ------ | ------ | ------ | | ||
| | | | | | | | ||
| ### 1.1 - ADAMAPI_KEY retrieval | ||
| To get the ADAMAPI_KEY, you need to access your ADAM portal and:<br> | ||
| 1. Select the "user icon" on the top right | ||
| 2. Expand / click the "USERNAME" | ||
| 3. Click on the "Api Key" to display your key | ||
| <br> | ||
| *Command-line ADAMAPI_KEY retrieval TBP* | ||
| ### 1.2 - ADAMAPI_KEY setup | ||
| There are three methods to setup the ADAMAPI_KEY and the ADAM-CORE instance: | ||
| 1. use the method setKey() and setAdamCore() | ||
| ```python | ||
| from adamapi import Auth | ||
| a = Auth() | ||
| a.setKey('<ADAMAPI_KEY>') | ||
| a.setAdamCore('https://test.adamplatform.eu') | ||
| ``` | ||
| 2. Export two envars like | ||
| ```bash | ||
| #open a Terminal and type: | ||
| export ADAMAPI_KEY='<ADAMAPI_KEY>' | ||
| export ADAMAPI_URL='https://test.adamplatform.eu' | ||
| ``` | ||
| 3. create a file called **.adamapirc** in the user home directory with the following content | ||
| ```text | ||
| key=<ADAMAPI_KEY> | ||
| url=https://test.adamplatform.eu | ||
| ``` | ||
| ### 1.3 - Examples | ||
| After ADAMAPI_KEY has been set up, an auth instance can be created with: | ||
| ```python | ||
| from adamapi import Auth | ||
| a = Auth() | ||
| a.authorize() | ||
| ``` | ||
| After authorize method you can retrive your autho token: | ||
| ```python | ||
| from adamapi import Auth | ||
| a = Auth() | ||
| a.authorize() | ||
| a.getAuthToken() | ||
| ``` | ||
| ## 2 - Datasets | ||
| This module provides datasets discovery functionality. | ||
| **Class contructor and parameters** | ||
| ```python | ||
| from adamapi import Datasets | ||
| datasets = Datasets( a ) | ||
| ``` | ||
| Parameters:<br> | ||
| | position/keyword | mandatory | type | default | description | | ||
| | ------ | ------ | ------ | ------ | ------ | | ||
| | 0 | True | Auth instance | | The ADAMAPI authorized instance obtained in the previous section | | ||
| **Public methods and parameters** | ||
| * **.getDatasets()** --> To retrieve datasets list <br> | ||
| Parameters: | ||
| | position/keyword | mandatory | type | default | description | | ||
| | ------ | ------ | ------ | ------ | ------ | | ||
| | 0 | False | str | | The datasetId. | | ||
| | page | False | numeric | 0 | Indicats a specific page | | ||
| | maxRecords | False | numeric | 10 | Max number of results in output. | | ||
| This .getDatasets() function can be used to retrive additional filters which are described in the key **filtersEnabled** (if exists). | ||
| ### 2.1 Examples | ||
| This module can be used in 2 different ways. | ||
| 1. To list all available datasets: | ||
| ```python | ||
| datasets = Datasets(a) | ||
| print(datasets.getDatasets()) | ||
| ``` | ||
| 2. To get detailed metadata about a specific dataset | ||
| ```python | ||
| datasets = Datasets(a) | ||
| print( datasets.getDatasets( '{{ID:DATASET}}' , page=0 , maxRecords=10 ) ) | ||
| ``` | ||
| 3. To get filtersEnabled. To use this additional filters see first example in Search section. | ||
| ```python | ||
| datasets = Datasets(a) | ||
| out=datasets.getDatasets("{{ID:DATASET}}") | ||
| print(out["filtersEnabled"]) | ||
| ``` | ||
| ## 3 - Search | ||
| This module provides discovery functionality through the products available on the ADAM instance. | ||
| **Class contructor and parameters** | ||
| ```python | ||
| from adamapi import Search | ||
| search = Search( a ) | ||
| ``` | ||
| Parameters:<br> | ||
| | position/keyword | mandatory | type | default | description | | ||
| | ------ | ------ | ------ | ------ | ------ | | ||
| | 0 | True | Auth instance | | The ADAMAPI authorized instance obtained in section 1-Auth | | ||
| **Public methods and parameters** | ||
| * **.getProducts()** --> To retrieve datasets list and metadata<br> | ||
| Parameters: | ||
| | position/keyword | mandatory | type | default | description | | ||
| | ------ | ------ | ------ | ------ | ------ | | ||
| | 0 | True | str | | The datasetId. | | ||
| | maxRecords | False | int | 10 | number of records | | ||
| | startIndex | False | int | 0 | starting record index | | ||
| | startDate | False | str or [datetime](https://docs.python.org/3/library/datetime.html) | | the start date | | ||
| | endDate | False | str or [datetime](https://docs.python.org/3/library/datetime.html) | | the end date | | ||
| | geometry | True | str or geojson | | GeoJson geometry,[geojson format](https://tools.ietf.org/html/rfc7946) [appendix](#geometry)| | ||
| ### 3.1 Examples | ||
| 1. Example1: | ||
| ```python | ||
| search=Search(a) | ||
| mongo_search=search.getProducts('{{ID:DATASET}}',maxRecords=1,startIndex=0,platform="{{VALUE}}") | ||
| ``` | ||
| 2. Example2: | ||
| ```python | ||
| search=Search(a) | ||
| mongo_search=search.getProducts('{{ID:DATASET}}',maxRecords=1,startIndex=0) | ||
| ``` | ||
| ## 4 - getData | ||
| This module provides data access of raster, spatial subset and timeseries. | ||
| **Class contructor and parameters** | ||
| ```python | ||
| from adamapi import GetData | ||
| data=GetData(a) | ||
| ``` | ||
| Parameters:<br> | ||
| | position/keyword | mandatory | type | default | description | | ||
| | ------ | ------ | ------ | ------ | ------ | | ||
| | 0 | True | Auth Instance | | The ADAMAPI authorized instance obtained in the section 1-Auth | | ||
| **Public methods and parameters** | ||
| * **.getData()** --> To retrieve a product<br> | ||
| Parameters to retrive a product: | ||
| | position/keyword | mandatory | type | default | description | | ||
| | ------ | ------ | ------ | ------ | ------ | | ||
| | 0 | True | str | | The datasetId | | ||
| | 1 | True | str | | The productId | | ||
| | outputFname | False | str | "adamapiresults/datasetId" | absolute filename or relative filename without extension. If not set, the retrived product is saved in "adamapiresults/datasetId.tif" | | ||
| Parameters to retrive a subset or a timeseries: | ||
| | position/keyword | mandatory | type | default | description | | ||
| | ------ | ------ | ------ | ------ | ------ | | ||
| | 0 | True | str | | The datasetId | | ||
| | 1 | True | str | | startDate | | ||
| | 2 | True | str | | endDate | | ||
| | 3 | False | boolean | False | asyncronous management for subset and timeseries order | | ||
| | geometry | True | str or geojson | | GeoJson geometry, if the geometry is a Point the api return a timeseries,[geojson format](https://tools.ietf.org/html/rfc7946) [appendix](#geometry)| | ||
| | outputFname | False | str | "adamapiresults/datasetId" | absolute filename without extension. If not set, the retrived product is saved in "adamapiresults/datasetId" | | ||
| ### 4.1 Examples | ||
| ```python | ||
| data=GetData(a) | ||
| #to retrive a raster | ||
| image=data.getData('{{ID:DATASET}}',productId='{{productId}}') | ||
| #or image=data.getData('{{ID:DATASET}}',productId='{{productId}}',outputFname="adamapi/test_get_data") | ||
| #to retrive a subset | ||
| image=data.getData(datasetId, startDate, endDate, geometry = '{"type": "Polygon", "coordinates": [ [ [ 43.916666667, 15.716666667 ], [43.916666667, 15.416666667 ], [ 44.216666667, 15.416666667 ], [ 44.216666667, 15.716666667 ], [ 43.916666667, 15.716666667 ]]]}') | ||
| #to retrive a timeseries | ||
| px_series=data.getData(datasetId, startDate, endDate, geometry = '{"type": "Point","coordinates":[43.916666667, 15.716666667]}') | ||
| #asyncronous management | ||
| image=data.getData(datasetId, startDate, endDate, asyncron= True,geometry = '{"type": "Polygon", "coordinates": [ [ [ 43.916666667, 15.716666667 ], [43.916666667, 15.416666667 ], [ 44.216666667, 15.416666667 ], [ 44.216666667, 15.716666667 ], [ 43.916666667, 15.716666667 ]]]}') | ||
| #check order status | ||
| - get order info: | ||
| image.pk -> order key | ||
| image.status -> order status | ||
| image.location -> order location when status is completed | ||
| image=data.getData(datasetId, asyncron= True,pk=image.pk) | ||
| ``` | ||
| ## Appendix | ||
| ### Geometry | ||
| ```python | ||
| #This geometry will return all the results it has intersected within it | ||
| geometry = { "type": "Polygon", "coordinates": [ [ [ 43.916666667, 15.716666667 ], [ 43.916666667, 15.416666667 ] , [ 44.216666667, 15.416666667 ], [ 44.216666667, 15.716666667 ], [ 43.916666667, 15.716666667 ] ] ] } | ||
| ``` | ||
| ```python | ||
| #This geometry will return all the results it has intersected on its outside | ||
| geometry = { "type": "Polygon", "coordinates": [ [ [ 43.84986877441406,15.925676536359038 ], [ 44.6539306640625,15.950766025306109 ],[ 44.681396484375,15.194084972583916 ], [ 43.8189697265625,15.20998780073036 ], [ 43.84986877441406,15.925676536359038 ] ] ] } | ||
| ``` | ||
| Platform: UNKNOWN | ||
| Classifier: Programming Language :: Python :: 3 | ||
| Classifier: Programming Language :: Python :: 3.6 | ||
| Classifier: Programming Language :: Python :: 3.8 | ||
| Classifier: License :: OSI Approved :: MIT License | ||
@@ -314,1 +14,337 @@ Classifier: Operating System :: Unix | ||
| Description-Content-Type: text/markdown | ||
| License-File: LICENSE | ||
| # Installation | ||
| ### Versioning | ||
| 1. adamapi==2.0.11 , This package works only with ADAMCORE 1. | ||
| 2. adamapi==2.1.0, This pachage works only with ADAMCORE 2. | ||
| ## Requirements | ||
| ```bash | ||
| sudo apt-get install python3-venv python3-gdal gdal-bin | ||
| ``` | ||
| ## Install with pip | ||
| ```bash | ||
| VENVNAME="adamapi" | ||
| python3 -m venv "${VENVNAME}" | ||
| source "${VENVNAME}/bin/activate"; | ||
| python3 -m pip install --upgrade pip; | ||
| pip install adamapi | ||
| ln -s "/usr/lib/python3/dist-packages/osgeo" "${VENVNAME}/lib/python3.8/site-packages/osgeo" | ||
| ``` | ||
| # API DEFINITIONS | ||
| This document briefly describes the ADMAPI functionalities.<br> | ||
| The ADAMAPI library is divided in 4 modules: | ||
| 1. Auth --> the authorization module | ||
| 2. Datasets --> to get the list of datasets | ||
| 3. Search --> to get the lists of products, including associated metadata (e.g. geometry, cloud cover, orbit, tile, ...) | ||
| 4. GetData --> to retrieve the product(s), including options to subset in space and time in the native data granularity and reduced processing capacity | ||
| ## 1 - Auth | ||
| This module takes care of user authentication and authorization.<br> | ||
| Without instancing an object of this module other components don't work.<br> | ||
| Auth module is based on the ADAMAPI_KEY, a key that uniquelly identifies the user. | ||
| **Class contructor and parameters** | ||
| ```python | ||
| from adamapi import Auth | ||
| a = Auth() | ||
| ``` | ||
| Parameters:<br> | ||
| | position/keyword | mandatory | type | default | description | | ||
| | ------ | ------ | ------ | ------ | ------ | | ||
| | | | | | | | ||
| **Public methods and parameters** | ||
| * **.setKey()** --> To setup the ADAMAPI_KEY<br> | ||
| Parameters: | ||
| | position/keyword | mandatory | type | default | description | | ||
| | ------ | ------ | ------ | ------ | ------ | | ||
| | 0 | True | str | | The ADAMAPI_KEY | | ||
| * **.setAdamCore()** --> To setup the url of the ADAM-CORE endpoint<br> | ||
| Parameters: | ||
| | position/keyword | mandatory | type | default | description | | ||
| | ------ | ------ | ------ | ------ | ------ | | ||
| | 0 | True | str | | The url like https://test.adamplatform.eu | | ||
| * **.authorize()** --> to instanciate an auth object<br> | ||
| Parameters: | ||
| | position/keyword | mandatory | type | default | description | | ||
| | ------ | ------ | ------ | ------ | ------ | | ||
| | | | | | | | ||
| * **.getAuthToken()** --> to get the authorization token<br> | ||
| Parameters: | ||
| | position/keyword | mandatory | type | default | description | | ||
| | ------ | ------ | ------ | ------ | ------ | | ||
| | | | | | | | ||
| ### 1.1 - ADAMAPI_KEY retrieval | ||
| To get the ADAMAPI_KEY, you need to access your ADAM portal and:<br> | ||
| 1. Select the "user icon" on the top right | ||
| 2. Expand / click the "USERNAME" | ||
| 3. Click on the "Api Key" to display your key | ||
| <br> | ||
| *Command-line ADAMAPI_KEY retrieval TBP* | ||
| ### 1.2 - ADAMAPI_KEY setup | ||
| There are three methods to setup the ADAMAPI_KEY and the ADAM-CORE instance: | ||
| 1. use the method setKey() and setAdamCore() | ||
| ```python | ||
| from adamapi import Auth | ||
| a = Auth() | ||
| a.setKey('<ADAMAPI_KEY>') | ||
| a.setAdamCore('https://test.adamplatform.eu') | ||
| ``` | ||
| 2. Export two envars like | ||
| ```bash | ||
| #open a Terminal and type: | ||
| export ADAMAPI_KEY='<ADAMAPI_KEY>' | ||
| export ADAMAPI_URL='https://test.adamplatform.eu' | ||
| ``` | ||
| 3. create a file called **.adamapirc** in the user home directory with the following content | ||
| ```text | ||
| key=<ADAMAPI_KEY> | ||
| url=https://test.adamplatform.eu | ||
| ``` | ||
| ### 1.3 - Examples | ||
| After ADAMAPI_KEY has been set up, an auth instance can be created with: | ||
| ```python | ||
| from adamapi import Auth | ||
| a = Auth() | ||
| a.authorize() | ||
| ``` | ||
| After authorize method you can retrive your autho token: | ||
| ```python | ||
| from adamapi import Auth | ||
| a = Auth() | ||
| a.authorize() | ||
| a.getAuthToken() | ||
| ``` | ||
| ## 2 - Datasets | ||
| This module provides datasets discovery functionality. | ||
| **Class contructor and parameters** | ||
| ```python | ||
| from adamapi import Datasets | ||
| datasets = Datasets( a ) | ||
| ``` | ||
| Parameters:<br> | ||
| | position/keyword | mandatory | type | default | description | | ||
| | ------ | ------ | ------ | ------ | ------ | | ||
| | 0 | True | Auth instance | | The ADAMAPI authorized instance obtained in the previous section | | ||
| **Public methods and parameters** | ||
| * **.getDatasets()** --> To retrieve datasets list <br> | ||
| Parameters: | ||
| | position/keyword | mandatory | type | default | description | | ||
| | ------ | ------ | ------ | ------ | ------ | | ||
| | 0 | False | str | | The datasetId. | | ||
| | page | False | numeric | 0 | Indicats a specific page | | ||
| | maxRecords | False | numeric | 10 | Max number of results in output. | | ||
| This .getDatasets() function can be used to retrive additional filters which are described in the key **filtersEnabled** (if exists). | ||
| ### 2.1 Examples | ||
| This module can be used in 2 different ways. | ||
| 1. To list all available datasets: | ||
| ```python | ||
| datasets = Datasets(a) | ||
| print(datasets.getDatasets()) | ||
| ``` | ||
| 2. To get detailed metadata about a specific dataset | ||
| ```python | ||
| datasets = Datasets(a) | ||
| print( datasets.getDatasets( '{{ID:DATASET}}' , page=0 , maxRecords=10 ) ) | ||
| ``` | ||
| 3. To get filtersEnabled. To use this additional filters see first example in Search section. | ||
| ```python | ||
| datasets = Datasets(a) | ||
| out=datasets.getDatasets("{{ID:DATASET}}") | ||
| print(out["filtersEnabled"]) | ||
| ``` | ||
| ## 3 - Search | ||
| This module provides discovery functionality through the products available on the ADAM instance. | ||
| **Class contructor and parameters** | ||
| ```python | ||
| from adamapi import Search | ||
| search = Search( a ) | ||
| ``` | ||
| Parameters:<br> | ||
| | position/keyword | mandatory | type | default | description | | ||
| | ------ | ------ | ------ | ------ | ------ | | ||
| | 0 | True | Auth instance | | The ADAMAPI authorized instance obtained in section 1-Auth | | ||
| **Public methods and parameters** | ||
| * **.getProducts()** --> To retrieve datasets list and metadata<br> | ||
| Parameters: | ||
| | position/keyword | mandatory | type | default | description | | ||
| | ------ | ------ | ------ | ------ | ------ | | ||
| | 0 | True | str | | The datasetId. | | ||
| | maxRecords | False | int | 10 | number of records | | ||
| | startIndex | False | int | 0 | starting record index | | ||
| | startDate | False | str or [datetime](https://docs.python.org/3/library/datetime.html) | | the start date | | ||
| | endDate | False | str or [datetime](https://docs.python.org/3/library/datetime.html) | | the end date | | ||
| | geometry | True | str or geojson | | GeoJson geometry,[geojson format](https://tools.ietf.org/html/rfc7946) [appendix](#geometry)| | ||
| ### 3.1 Examples | ||
| 1. Example1: | ||
| ```python | ||
| search=Search(a) | ||
| mongo_search=search.getProducts('{{ID:DATASET}}',maxRecords=1,startIndex=0,platform="{{VALUE}}") | ||
| ``` | ||
| 2. Example2: | ||
| ```python | ||
| search=Search(a) | ||
| mongo_search=search.getProducts('{{ID:DATASET}}',maxRecords=1,startIndex=0) | ||
| ``` | ||
| ## 4 - GetData | ||
| This module provides data access of raster, spatial subset, timeseries in the native data granularity and reduced processing capacity. | ||
| **Class contructor and parameters** | ||
| ```python | ||
| from adamapi import GetData | ||
| data=GetData(a) | ||
| ``` | ||
| Parameters:<br> | ||
| | position/keyword | mandatory | type | default | description | | ||
| | ------ | ------ | ------ | ------ | ------ | | ||
| | 0 | True | Auth Instance | | The ADAMAPI authorized instance obtained in the section 1-Auth | | ||
| **Public methods and parameters** | ||
| * **.getData()** --> To retrieve a specific product or a dataset in its native granularity, to get a subset of it, to perform a timeseries or to exec simple processing <br> | ||
| | position/keyword | mandatory | type | default | description | | ||
| | ------ | ------ | ------ | ------ | ------ | | ||
| | 0 | True | str | | The datasetId | | ||
| | 1 | True | str | GetFile | request type. available values: GetFile,GetSubset, GetTimeseries and GetProcessing | | ||
| | asynchronous | False | boolean | False | rappesents how the request will be performed | | ||
| | compress | False | boolean | False | return a zip file | | ||
| | rest | False | boolean | True | perform RESTful order ignoring explorer state on the server and equalization configured using the explorer gui | | ||
| | filters | True | json | {} | json object with filters parameter. startDate and endDate are required inside it | | ||
| | options | False | json | {} | request option | | ||
| | outputDir | False | str | `adamapiresults/` | set a different download directory inside `adamapiresult/` main directory | | ||
| ### 4.1 Examples | ||
| ```python | ||
| data=GetData(a) | ||
| #to retrive a specific product | ||
| image = data.getData('{{ID:DATASET}}',"GetFile",asynchronous=False,compress=False,rest=False,filters={"startDate":'{{STARTDATE}}',"endDate":'{{ENDDATE}}',"productId":'{{PRODUCTID}}'},outputDir='{{OUTPUT_DIR}}') | ||
| #to retrieve a dataset in its native granularity | ||
| data=GetData(self.a) | ||
| image = data.getData('{{ID:DATASET}}',"GetFile",asynchronous=False,compress=False,rest=False,filters={"startDate":'{{STARTDATE}}',"endDate":'{{ENDDATE}}',"geometry":'{{GEOMETRY}}'},outputDir='{{OUTPUT_DIR}}') | ||
| ``` | ||
| For the GetSubset,GetTimeseries and GetProcessing requests you need to add the `options` parameter with these constraints : [output formats](#output-formats) and [functions](#processing-function)(only for processing request) | ||
| ```python | ||
| #subset example | ||
| image = data.getData('{{ID:DATASET}}',"GetSubset",asynchronous=False,compress=False,rest=False,filters={"startDate":'{{STARTDATE}}',"endDate":'{{ENDDATE}}',"geometry":'{{GEOMETRY}}'},options={"format":'{{FORMATS}}'},outputDir='{{OUTPUT_DIR}}') | ||
| #timeseries example | ||
| image = data.getData('{{ID:DATASET}}',"GetTimeseries",asynchronous=False,compress=False,rest=False,filters={"startDate":'{{STARTDATE}}',"endDate":'{{ENDDATE}}',"geometry":'{{GEOMETRY}}'},options={"format":'{{FORMATS}}'},outputDir='{{OUTPUT_DIR}}') | ||
| #processing example | ||
| image = data.getData('{{ID:DATASET}}',"GetProcessing",asynchronous=False,compress=False,rest=False,filters={"startDate":'{{STARTDATE}}',"endDate":'{{ENDDATE}}',"geometry":'{{GEOMETRY}}'},options={"format":'{{FORMAT}}',"function":'{{FUNCTION}}'},outputDir='{{OUTPUT_DIR}}') | ||
| ``` | ||
| ### 4.3 Asyncronous Example | ||
| ```python | ||
| #1. execute the request | ||
| image = data.getData('{{ID:DATASET}}',"GetSubset",asynchronous=False,compress=False,rest=False,filters={"startDate":'{{STARTDATE}}',"endDate":'{{ENDDATE}}',"geometry":'{{GEOMETRY}}'},options={"format":'{{FORMATS}}'},outputDir='{{OUTPUT_DIR}}') | ||
| #2. check the status | ||
| stat=data.getData(datasetId,"GetSubset",asynchronous=True,id=str(image.pk)) | ||
| while stat.status != "completed": | ||
| time.sleep(1) | ||
| stat=data.getData(datasetId,"GetSubset",asynchronous=True,id=str(image.pk)) | ||
| #3. download the zip,unzip it and remove the zip (optional) | ||
| for res in stat.list: | ||
| if res["status"] == "failed": | ||
| print(res["exit_code"]) | ||
| else: | ||
| r=self.a.client(res["download"]["url"],{},"GET") | ||
| with open(str(res["download"]["url"].split("/")[4])+"_"+str(res["download"]["url"].split("/")[5]), 'wb' ) as f: | ||
| f.write( r.content ) | ||
| ``` | ||
| # Appendix 1 - Data format | ||
| ## date and date+time | ||
| Supported string date/date+time format are: | ||
| * '%Y-%m-%dT%H:%M:%S', | ||
| * '%Y-%m-%dT%H:%M:%SZ', | ||
| * '%Y-%m-%d' | ||
| ### GeoJson | ||
| Geometry have to follow the latest geojson standard [rfc7946](https://tools.ietf.org/html/rfc7946)<br> | ||
| In particular Polygons and MultiPolygons should follow the right-hand rule<br> | ||
| ### Geometry | ||
| ```python | ||
| #This geometry will return all the results it has intersected within it | ||
| geometry = { "type": "Polygon", "coordinates": [ [ [ 43.916666667, 15.716666667 ], [ 43.916666667, 15.416666667 ] , [ 44.216666667, 15.416666667 ], [ 44.216666667, 15.716666667 ], [ 43.916666667, 15.716666667 ] ] ] } | ||
| ``` | ||
| ```python | ||
| #This geometry will return all the results it has intersected on its outside | ||
| geometry = { "type": "Polygon", "coordinates": [ [ [ 43.84986877441406,15.925676536359038 ], [ 44.6539306640625,15.950766025306109 ],[ 44.681396484375,15.194084972583916 ], [ 43.8189697265625,15.20998780073036 ], [ 43.84986877441406,15.925676536359038 ] ] ] } | ||
| ``` | ||
| ### Output Formats | ||
| | request | output format | | ||
| | ---- | ---- | | ||
| | GetFile | - | | ||
| | GetSubset | tiff,png | | ||
| | GetTimeseries | json,csv | | ||
| | GetProcessing | tiff,png | | ||
| ### Processing Function | ||
| | type | description | | ||
| | ---- | ---- | | ||
| | average | execute the average from the products | | ||
| | overlap | execute the products overlap without any specific strategy | | ||
| | mosterecent | puts on the top the most recent product | | ||
| | leastrecent | puts on top the least recent product | | ||
| | minvalue | for each pixel, puts on top the minimum value of the pixel | | ||
| | maxvalue | for each pixel, puts on top the maximum value of the pixel | |
+19
-9
| """ | ||
| Copyright (c) 2020 MEEO s.r.l. | ||
| Copyright (c) 2022 MEEO s.r.l. | ||
@@ -26,3 +26,3 @@ Permission is hereby granted, free of charge, to any person obtaining a copy | ||
| with open("../doc/api_definitions_no_image.md","r") as fh: | ||
| with open("README.md","r") as fh: | ||
| long_description=fh.read() | ||
@@ -32,3 +32,3 @@ | ||
| name="adamapi", | ||
| version="2.0.11", | ||
| version="2.1.0", | ||
| author="MEEO s.r.l.", | ||
@@ -40,11 +40,23 @@ author_email="info@meeo.it", | ||
| url="https://git.services.meeo.it/das/adamapi", | ||
| packages=['adamapi'], | ||
| packages=setuptools.find_packages(where="src"), | ||
| package_dir={"":"src"}, | ||
| install_requires=[ | ||
| 'requests >= 2.22.0', | ||
| 'imageio' | ||
| 'imageio == 2.9.0', | ||
| 'tqdm == 4.62.3', | ||
| 'numpy == 1.21.2', | ||
| 'python-dateutil == 2.8.2', | ||
| 'matplotlib == 3.4.3', | ||
| "certifi == 2021.5.30", | ||
| "charset-normalizer == 2.0.6", | ||
| "cycler == 0.10.0", | ||
| "idna == 3.2", | ||
| "kiwisolver == 1.3.2", | ||
| "Pillow == 8.3.2", | ||
| "pyparsing == 2.4.7", | ||
| "six == 1.16.0", | ||
| "urllib3 ==1.26.7" | ||
| ], | ||
| classifiers=[ | ||
| "Programming Language :: Python :: 3", | ||
| "Programming Language :: Python :: 3.6", | ||
| "Programming Language :: Python :: 3.8", | ||
| "License :: OSI Approved :: MIT License", | ||
@@ -55,3 +67,1 @@ "Operating System :: Unix", | ||
| ) | ||
| Metadata-Version: 2.1 | ||
| Name: adamapi | ||
| Version: 2.0.11 | ||
| Summary: Python Adam API | ||
| Home-page: https://git.services.meeo.it/das/adamapi | ||
| Author: MEEO s.r.l. | ||
| Author-email: info@meeo.it | ||
| License: UNKNOWN | ||
| Description: # Installation | ||
| ### Versioning | ||
| 1. adamapi==1.0.1 , This package works only with ADAMCORE 1. | ||
| 2. adamapi==2.0.11, This pachage works only with ADAMCORE 2. | ||
| ## Requirements | ||
| ```bash | ||
| sudo apt-get install python3-venv python3-gdal gdal-bin | ||
| ``` | ||
| ## Install with pip | ||
| ```bash | ||
| VENVNAME="adamapi" | ||
| python3 -m venv "${VENVNAME}" | ||
| PYTHONVERSION=$(ls ${VENVNAME}/lib/) | ||
| source "${VENVNAME}/bin/activate"; | ||
| python3 -m pip install --upgrade pip; | ||
| pip install adamapi==2.0.1r1 | ||
| ln -s "/usr/lib/python3/dist-packages/osgeo" "${VENVNAME}/lib/${PYTHONVERSION}/site-packages/osgeo" | ||
| ``` | ||
| # API DEFINITIONS | ||
| This document briefly describes the ADMAPI functionalities.<br> | ||
| The complete and interactive documentation is provided through a jupyter notebook *(LINK to notebook TBP)*.<br> | ||
| The ADAMAPI library is divided in 4 modules: | ||
| 1. Auth --> the authorization module | ||
| 2. Datasets --> to get the list of datasets | ||
| 3. Search --> to get the lists of products, including associated metadata (e.g. geometry, cloud cover, orbit, tile, ...) | ||
| 4. GetData --> to retrieve the product(s), including options to subset in space and time | ||
| ## 1 - Auth | ||
| This module takes care of user authentication and authorization.<br> | ||
| Without instancing an object of this module other components don't work.<br> | ||
| Auth module is based on the ADAMAPI_KEY, a key that uniquelly identifies the user. | ||
| **Class contructor and parameters** | ||
| ```python | ||
| from adamapi import Auth | ||
| a = Auth() | ||
| ``` | ||
| Parameters:<br> | ||
| | position/keyword | mandatory | type | default | description | | ||
| | ------ | ------ | ------ | ------ | ------ | | ||
| | | | | | | | ||
| **Public methods and parameters** | ||
| * **.setKey()** --> To setup the ADAMAPI_KEY<br> | ||
| Parameters: | ||
| | position/keyword | mandatory | type | default | description | | ||
| | ------ | ------ | ------ | ------ | ------ | | ||
| | 0 | True | str | | The ADAMAPI_KEY | | ||
| * **.setAdamCore()** --> To setup the url of the ADAM-CORE endpoint<br> | ||
| Parameters: | ||
| | position/keyword | mandatory | type | default | description | | ||
| | ------ | ------ | ------ | ------ | ------ | | ||
| | 0 | True | str | | The url like https://test.adamplatform.eu | | ||
| * **.authorize()** --> to instanciate an auth object<br> | ||
| Parameters: | ||
| | position/keyword | mandatory | type | default | description | | ||
| | ------ | ------ | ------ | ------ | ------ | | ||
| | | | | | | | ||
| * **.getAuthToken()** --> to get the authorization token<br> | ||
| Parameters: | ||
| | position/keyword | mandatory | type | default | description | | ||
| | ------ | ------ | ------ | ------ | ------ | | ||
| | | | | | | | ||
| ### 1.1 - ADAMAPI_KEY retrieval | ||
| To get the ADAMAPI_KEY, you need to access your ADAM portal and:<br> | ||
| 1. Select the "user icon" on the top right | ||
| 2. Expand / click the "USERNAME" | ||
| 3. Click on the "Api Key" to display your key | ||
| <br> | ||
| *Command-line ADAMAPI_KEY retrieval TBP* | ||
| ### 1.2 - ADAMAPI_KEY setup | ||
| There are three methods to setup the ADAMAPI_KEY and the ADAM-CORE instance: | ||
| 1. use the method setKey() and setAdamCore() | ||
| ```python | ||
| from adamapi import Auth | ||
| a = Auth() | ||
| a.setKey('<ADAMAPI_KEY>') | ||
| a.setAdamCore('https://test.adamplatform.eu') | ||
| ``` | ||
| 2. Export two envars like | ||
| ```bash | ||
| #open a Terminal and type: | ||
| export ADAMAPI_KEY='<ADAMAPI_KEY>' | ||
| export ADAMAPI_URL='https://test.adamplatform.eu' | ||
| ``` | ||
| 3. create a file called **.adamapirc** in the user home directory with the following content | ||
| ```text | ||
| key=<ADAMAPI_KEY> | ||
| url=https://test.adamplatform.eu | ||
| ``` | ||
| ### 1.3 - Examples | ||
| After ADAMAPI_KEY has been set up, an auth instance can be created with: | ||
| ```python | ||
| from adamapi import Auth | ||
| a = Auth() | ||
| a.authorize() | ||
| ``` | ||
| After authorize method you can retrive your autho token: | ||
| ```python | ||
| from adamapi import Auth | ||
| a = Auth() | ||
| a.authorize() | ||
| a.getAuthToken() | ||
| ``` | ||
| ## 2 - Datasets | ||
| This module provides datasets discovery functionality. | ||
| **Class contructor and parameters** | ||
| ```python | ||
| from adamapi import Datasets | ||
| datasets = Datasets( a ) | ||
| ``` | ||
| Parameters:<br> | ||
| | position/keyword | mandatory | type | default | description | | ||
| | ------ | ------ | ------ | ------ | ------ | | ||
| | 0 | True | Auth instance | | The ADAMAPI authorized instance obtained in the previous section | | ||
| **Public methods and parameters** | ||
| * **.getDatasets()** --> To retrieve datasets list <br> | ||
| Parameters: | ||
| | position/keyword | mandatory | type | default | description | | ||
| | ------ | ------ | ------ | ------ | ------ | | ||
| | 0 | False | str | | The datasetId. | | ||
| | page | False | numeric | 0 | Indicats a specific page | | ||
| | maxRecords | False | numeric | 10 | Max number of results in output. | | ||
| This .getDatasets() function can be used to retrive additional filters which are described in the key **filtersEnabled** (if exists). | ||
| ### 2.1 Examples | ||
| This module can be used in 2 different ways. | ||
| 1. To list all available datasets: | ||
| ```python | ||
| datasets = Datasets(a) | ||
| print(datasets.getDatasets()) | ||
| ``` | ||
| 2. To get detailed metadata about a specific dataset | ||
| ```python | ||
| datasets = Datasets(a) | ||
| print( datasets.getDatasets( '{{ID:DATASET}}' , page=0 , maxRecords=10 ) ) | ||
| ``` | ||
| 3. To get filtersEnabled. To use this additional filters see first example in Search section. | ||
| ```python | ||
| datasets = Datasets(a) | ||
| out=datasets.getDatasets("{{ID:DATASET}}") | ||
| print(out["filtersEnabled"]) | ||
| ``` | ||
| ## 3 - Search | ||
| This module provides discovery functionality through the products available on the ADAM instance. | ||
| **Class contructor and parameters** | ||
| ```python | ||
| from adamapi import Search | ||
| search = Search( a ) | ||
| ``` | ||
| Parameters:<br> | ||
| | position/keyword | mandatory | type | default | description | | ||
| | ------ | ------ | ------ | ------ | ------ | | ||
| | 0 | True | Auth instance | | The ADAMAPI authorized instance obtained in section 1-Auth | | ||
| **Public methods and parameters** | ||
| * **.getProducts()** --> To retrieve datasets list and metadata<br> | ||
| Parameters: | ||
| | position/keyword | mandatory | type | default | description | | ||
| | ------ | ------ | ------ | ------ | ------ | | ||
| | 0 | True | str | | The datasetId. | | ||
| | maxRecords | False | int | 10 | number of records | | ||
| | startIndex | False | int | 0 | starting record index | | ||
| | startDate | False | str or [datetime](https://docs.python.org/3/library/datetime.html) | | the start date | | ||
| | endDate | False | str or [datetime](https://docs.python.org/3/library/datetime.html) | | the end date | | ||
| | geometry | True | str or geojson | | GeoJson geometry,[geojson format](https://tools.ietf.org/html/rfc7946) [appendix](#geometry)| | ||
| ### 3.1 Examples | ||
| 1. Example1: | ||
| ```python | ||
| search=Search(a) | ||
| mongo_search=search.getProducts('{{ID:DATASET}}',maxRecords=1,startIndex=0,platform="{{VALUE}}") | ||
| ``` | ||
| 2. Example2: | ||
| ```python | ||
| search=Search(a) | ||
| mongo_search=search.getProducts('{{ID:DATASET}}',maxRecords=1,startIndex=0) | ||
| ``` | ||
| ## 4 - getData | ||
| This module provides data access of raster, spatial subset and timeseries. | ||
| **Class contructor and parameters** | ||
| ```python | ||
| from adamapi import GetData | ||
| data=GetData(a) | ||
| ``` | ||
| Parameters:<br> | ||
| | position/keyword | mandatory | type | default | description | | ||
| | ------ | ------ | ------ | ------ | ------ | | ||
| | 0 | True | Auth Instance | | The ADAMAPI authorized instance obtained in the section 1-Auth | | ||
| **Public methods and parameters** | ||
| * **.getData()** --> To retrieve a product<br> | ||
| Parameters to retrive a product: | ||
| | position/keyword | mandatory | type | default | description | | ||
| | ------ | ------ | ------ | ------ | ------ | | ||
| | 0 | True | str | | The datasetId | | ||
| | 1 | True | str | | The productId | | ||
| | outputFname | False | str | "adamapiresults/datasetId" | absolute filename or relative filename without extension. If not set, the retrived product is saved in "adamapiresults/datasetId.tif" | | ||
| Parameters to retrive a subset or a timeseries: | ||
| | position/keyword | mandatory | type | default | description | | ||
| | ------ | ------ | ------ | ------ | ------ | | ||
| | 0 | True | str | | The datasetId | | ||
| | 1 | True | str | | startDate | | ||
| | 2 | True | str | | endDate | | ||
| | 3 | False | boolean | False | asyncronous management for subset and timeseries order | | ||
| | geometry | True | str or geojson | | GeoJson geometry, if the geometry is a Point the api return a timeseries,[geojson format](https://tools.ietf.org/html/rfc7946) [appendix](#geometry)| | ||
| | outputFname | False | str | "adamapiresults/datasetId" | absolute filename without extension. If not set, the retrived product is saved in "adamapiresults/datasetId" | | ||
| ### 4.1 Examples | ||
| ```python | ||
| data=GetData(a) | ||
| #to retrive a raster | ||
| image=data.getData('{{ID:DATASET}}',productId='{{productId}}') | ||
| #or image=data.getData('{{ID:DATASET}}',productId='{{productId}}',outputFname="adamapi/test_get_data") | ||
| #to retrive a subset | ||
| image=data.getData(datasetId, startDate, endDate, geometry = '{"type": "Polygon", "coordinates": [ [ [ 43.916666667, 15.716666667 ], [43.916666667, 15.416666667 ], [ 44.216666667, 15.416666667 ], [ 44.216666667, 15.716666667 ], [ 43.916666667, 15.716666667 ]]]}') | ||
| #to retrive a timeseries | ||
| px_series=data.getData(datasetId, startDate, endDate, geometry = '{"type": "Point","coordinates":[43.916666667, 15.716666667]}') | ||
| #asyncronous management | ||
| image=data.getData(datasetId, startDate, endDate, asyncron= True,geometry = '{"type": "Polygon", "coordinates": [ [ [ 43.916666667, 15.716666667 ], [43.916666667, 15.416666667 ], [ 44.216666667, 15.416666667 ], [ 44.216666667, 15.716666667 ], [ 43.916666667, 15.716666667 ]]]}') | ||
| #check order status | ||
| - get order info: | ||
| image.pk -> order key | ||
| image.status -> order status | ||
| image.location -> order location when status is completed | ||
| image=data.getData(datasetId, asyncron= True,pk=image.pk) | ||
| ``` | ||
| ## Appendix | ||
| ### Geometry | ||
| ```python | ||
| #This geometry will return all the results it has intersected within it | ||
| geometry = { "type": "Polygon", "coordinates": [ [ [ 43.916666667, 15.716666667 ], [ 43.916666667, 15.416666667 ] , [ 44.216666667, 15.416666667 ], [ 44.216666667, 15.716666667 ], [ 43.916666667, 15.716666667 ] ] ] } | ||
| ``` | ||
| ```python | ||
| #This geometry will return all the results it has intersected on its outside | ||
| geometry = { "type": "Polygon", "coordinates": [ [ [ 43.84986877441406,15.925676536359038 ], [ 44.6539306640625,15.950766025306109 ],[ 44.681396484375,15.194084972583916 ], [ 43.8189697265625,15.20998780073036 ], [ 43.84986877441406,15.925676536359038 ] ] ] } | ||
| ``` | ||
| Platform: UNKNOWN | ||
| Classifier: Programming Language :: Python :: 3 | ||
| Classifier: Programming Language :: Python :: 3.6 | ||
| Classifier: Programming Language :: Python :: 3.8 | ||
| Classifier: License :: OSI Approved :: MIT License | ||
| Classifier: Operating System :: Unix | ||
| Requires-Python: >=3.6 | ||
| Description-Content-Type: text/markdown |
| imageio | ||
| requests>=2.22.0 |
| setup.py | ||
| adamapi/__init__.py | ||
| adamapi/authorization.py | ||
| adamapi/datasets.py | ||
| adamapi/get_data.py | ||
| adamapi/search.py | ||
| adamapi.egg-info/PKG-INFO | ||
| adamapi.egg-info/SOURCES.txt | ||
| adamapi.egg-info/dependency_links.txt | ||
| adamapi.egg-info/requires.txt | ||
| adamapi.egg-info/top_level.txt |
| adamapi |
| """ | ||
| Copyright (c) 2020 MEEO s.r.l. | ||
| Permission is hereby granted, free of charge, to any person obtaining a copy | ||
| of this software and associated documentation files (the "Software"), to deal | ||
| in the Software without restriction, including without limitation the rights | ||
| to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | ||
| copies of the Software, and to permit persons to whom the Software is | ||
| furnished to do so, subject to the following conditions: | ||
| The above copyright notice and this permission notice shall be included in all | ||
| copies or substantial portions of the Software. | ||
| THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | ||
| IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | ||
| FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | ||
| AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER | ||
| LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, | ||
| OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE | ||
| SOFTWARE. | ||
| """ | ||
| class AdamApiError( Exception ): | ||
| """Base class for exceptions on the adamapi module.""" | ||
| pass | ||
| class AdamApiMessage( Exception ): | ||
| """Base class for adamapi mesage""" | ||
| def __init__(self,json): | ||
| self.pk=json["pk"] if "pk" in json else "" | ||
| self.status=json["status"] if "status" in json else "" | ||
| self.location = json["location"] if "location" in json else "" | ||
| self.error = json["error"] if "error" in json else "" | ||
| from . import authorization | ||
| from . import datasets | ||
| from . import get_data | ||
| from . import search | ||
| Auth = authorization.Auth | ||
| Datasets = datasets.Datasets | ||
| GetData = get_data.GetData | ||
| Search = search.Search |
| """ | ||
| Copyright (c) 2020 MEEO s.r.l. | ||
| Permission is hereby granted, free of charge, to any person obtaining a copy | ||
| of this software and associated documentation files (the "Software"), to deal | ||
| in the Software without restriction, including without limitation the rights | ||
| to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | ||
| copies of the Software, and to permit persons to whom the Software is | ||
| furnished to do so, subject to the following conditions: | ||
| The above copyright notice and this permission notice shall be included in all | ||
| copies or substantial portions of the Software. | ||
| THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | ||
| IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | ||
| FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | ||
| AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER | ||
| LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, | ||
| OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE | ||
| SOFTWARE. | ||
| """ | ||
| import os | ||
| import requests | ||
| import re | ||
| import logging | ||
| from datetime import datetime,timedelta | ||
| logger = logging.getLogger( 'adamapi' ) | ||
| from . import AdamApiError | ||
| class Auth(): | ||
| def __init__( self, key=None, url=None ): | ||
| """ | ||
| Authorization class used to manage users credentials | ||
| """ | ||
| self.key = key | ||
| self.url = url | ||
| self.authorized = False | ||
| self.access_obj=None | ||
| self.lock=False | ||
| def authorize( self ): | ||
| """ | ||
| try to autorize the adamapi to the target adam instance | ||
| """ | ||
| #verify if key are valid | ||
| self._check() | ||
| url = os.path.join( self.url,'apis','v1','authorization', 'adamapi' ) | ||
| headers={ 'Authorization': self.key } | ||
| r=requests.post( url,data={},headers=headers ) | ||
| res = r.json() | ||
| if r.status_code !=200: | ||
| raise AdamApiError( res['title'] ) | ||
| else: | ||
| logger.info( res ) | ||
| self.authorized = True | ||
| self.access_obj= res | ||
| return res | ||
| def getAuthToken(self): | ||
| return { | ||
| "authorization_token" : self.access_obj["access_token"], | ||
| "expires_at" : self.access_obj["expires_at"] | ||
| } | ||
| def _check( self ): | ||
| """ | ||
| verify if the user key and the adam url are valid | ||
| """ | ||
| self.getKey() | ||
| self.getAdamCore() | ||
| if self.key is None: | ||
| raise AdamApiError( "key can not be None" ) | ||
| if self.url is None: | ||
| raise AdamApiError( "url (adam endpoint) can not be None" ) | ||
| def setKey( self, key ): | ||
| """ | ||
| set the user key | ||
| """ | ||
| self.key = key | ||
| def setAdamCore( self, url ): | ||
| """ | ||
| set the adam endpoint url | ||
| """ | ||
| self.url = url | ||
| def getKey( self ): | ||
| """ | ||
| key priority: | ||
| 1 - use the key declared on this module | ||
| 2 - use a key exported as envvars | ||
| 3 - try to retrieve the key from $HOME/.adamapirc | ||
| """ | ||
| if self.key is not None: | ||
| pass | ||
| elif os.environ.get( 'ADAMAPI_KEY') is not None: | ||
| self.key = os.environ.get( 'ADAMAPI_KEY' ) | ||
| logger.debug( 'Get key from envvar' ) | ||
| else: | ||
| try: | ||
| self.key = self._getRc()[ 'key' ] | ||
| logger.debug( 'Get key from adamapirc' ) | ||
| except: | ||
| None | ||
| if self.key is None: #again | ||
| logger.warning( "API key not specified" ) | ||
| return self.key | ||
| def getAdamCore( self ): | ||
| """ | ||
| adam endpoint priority: | ||
| 1 - use the url declared on this module | ||
| 2 - use a url exported as envvars | ||
| 3 - try to retrieve the url from $HOME/.adamapirc | ||
| """ | ||
| if self.url is not None: | ||
| pass | ||
| elif os.environ.get( 'ADAMAPI_URL') is not None: | ||
| self.url = os.environ.get( 'ADAMAPI_URL' ) | ||
| logger.debug( 'Get url from envvar' ) | ||
| else: | ||
| try: | ||
| self.url = self._getRc()[ 'url' ] | ||
| logger.debug( 'Get url from adamapirc' ) | ||
| except: | ||
| None | ||
| if self.url is None: #again | ||
| logger.warning( "ADAM API endpoint not specified" ) | ||
| return self.url | ||
| def _getRc( self ): | ||
| """ | ||
| check if ~/.adamapirc file exist ant try to extract url and key from it | ||
| """ | ||
| api_rc = os.environ.get( 'ADAMAPI_RC', os.path.expanduser( '~/.adamapirc' ) ) | ||
| config_dict = {} | ||
| if os.path.isfile( api_rc ): | ||
| with open( api_rc ) as f: | ||
| for line in f.readlines(): | ||
| line = line.strip() | ||
| if line.lstrip( '#' ) == line: | ||
| try: | ||
| key, val = line.split( '=', 1 ) | ||
| config_dict[ key.strip() ] = val.strip() | ||
| except: | ||
| logger.warning( 'Invalid settings in "%s" - %s' %( api_rc, line ) ) | ||
| return config_dict | ||
| def _checktokenExpire(self): | ||
| """ | ||
| check if token is expired | ||
| """ | ||
| if (datetime.strptime(self.access_obj['expires_at'],"%Y-%m-%dT%H:%M:%S.%fZ") - datetime.utcnow().replace(microsecond=0)) <= timedelta(minutes=15) : | ||
| return True | ||
| else: | ||
| return False | ||
| def _refreshToken(self): | ||
| """ | ||
| refresh tocken obj | ||
| """ | ||
| self.lock=True | ||
| params = {} | ||
| params["refresh_token"]=self.access_obj["refresh_token"] | ||
| self.access_obj = self.client(os.path.join( 'apis','v1', 'authorization', 'refresh' ), params, 'PUT').json() | ||
| self.lock=False | ||
| def client( self, query, params, request_type, authorization_required = True , force_raise = False ,enable_stream=False): | ||
| """ | ||
| The client method that perform the request appending the Authorization header | ||
| """ | ||
| #verify if the user already call the authorize method of the Auth instance | ||
| #if authorization_required and not self.authorized: | ||
| # self.authorize() | ||
| if not self.authorized and self.access_obj is not None : | ||
| raise AdamApiError("Unauthorized") | ||
| if not self.lock and self._checktokenExpire(): | ||
| self._refreshToken() | ||
| #verify if the passed url already include the host or if the adamcore url have to be prepended | ||
| if re.match( r'^http[s]?://.+', query ): | ||
| url = query | ||
| else: | ||
| url = os.path.join( self.url, query ) | ||
| if request_type not in [ 'GET', 'POST', 'PUT' ]: | ||
| raise AdamApiError( "Request type not supported" ) | ||
| logger.debug( "%s request started", request_type ) | ||
| if "\\" in url: | ||
| url = url.replace("\\","/") | ||
| if request_type == 'GET': | ||
| par = params.copy() | ||
| par["token"] = self.access_obj["access_token"] | ||
| r=requests.get( url, params=par ,stream=enable_stream) | ||
| elif request_type == 'POST': | ||
| par = params.copy() | ||
| par["token"] = self.access_obj["access_token"] | ||
| r=requests.post( url,data=par ) | ||
| elif request_type == 'PUT': | ||
| r=requests.put( url,data=params ) | ||
| logger.info(r.status_code) | ||
| if r.status_code in range(400,599): | ||
| if force_raise: | ||
| return r.raise_for_status() | ||
| else: | ||
| raise AdamApiError(r.json()['title']) | ||
| else: | ||
| return r |
| """ | ||
| Copyright (c) 2020 MEEO s.r.l. | ||
| Permission is hereby granted, free of charge, to any person obtaining a copy | ||
| of this software and associated documentation files (the "Software"), to deal | ||
| in the Software without restriction, including without limitation the rights | ||
| to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | ||
| copies of the Software, and to permit persons to whom the Software is | ||
| furnished to do so, subject to the following conditions: | ||
| The above copyright notice and this permission notice shall be included in all | ||
| copies or substantial portions of the Software. | ||
| THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | ||
| IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | ||
| FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | ||
| AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER | ||
| LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, | ||
| OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE | ||
| SOFTWARE. | ||
| """ | ||
| import os | ||
| import logging | ||
| logger=logging.getLogger('adamapi') | ||
| from . import AdamApiError | ||
| class Datasets(): | ||
| def __init__(self, client): | ||
| self.client = client | ||
| self.LOG = logger | ||
| def getDatasets(self,datasetId=None,**kwargs): | ||
| params={} | ||
| params["type"]="adamapi" | ||
| if datasetId: | ||
| url=os.path.join("apis","v2","datasets",datasetId.split(":")[0]) | ||
| else: | ||
| url=os.path.join("apis","v2","datasets","list") | ||
| if 'page' in kwargs: | ||
| params['page']=kwargs['page'] | ||
| else: | ||
| params['page']=0 | ||
| if 'maxRecords' in kwargs: | ||
| params['maxRecords']=kwargs['maxRecords'] | ||
| else: | ||
| params['maxRecords']=10 | ||
| r = self.client.client( url, params, 'GET' ) | ||
| self.LOG.info( 'Datasets request executed' ) | ||
| response = r.json() | ||
| return response | ||
| #def getDatasets( self, datasetId=None,**kwargs ): | ||
| # """ | ||
| # getDataset convention: | ||
| # 1. without parametes returns all datasets present over the adam endpoint; | ||
| # 2. with datasetId paramenter returns a specific description of this dataset | ||
| # @ {ID:datasetId}: the datasetId | ||
| # @ page: Indicats a specific page | ||
| # @ maxRecords: Max number of results in output | ||
| # """ | ||
| # params={ 'datasetId':datasetId } | ||
| # params["type"]="adamapi" | ||
| # if 'page' in kwargs: | ||
| # params['page']=kwargs['page'] | ||
| # else: | ||
| # params['page']=0 | ||
| # if 'maxRecords' in kwargs: | ||
| # params['maxRecords']=kwargs['maxRecords'] | ||
| # else: | ||
| # params['maxRecords']=10 | ||
| # r = self.client.client( os.path.join( 'datasets', 'api', 'datasets' ), params, 'GET' ) | ||
| # self.LOG.info( 'Datasets request executed' ) | ||
| # response = r.json() | ||
| # if 'errors' in response: | ||
| # raise AdamApiError( response[ 'errors' ] ) | ||
| # return response[ 'response' ] |
| """ | ||
| Copyright (c) 2020 MEEO s.r.l. | ||
| Permission is hereby granted, free of charge, to any person obtaining a copy | ||
| of this software and associated documentation files (the "Software"), to deal | ||
| in the Software without restriction, including without limitation the rights | ||
| to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | ||
| copies of the Software, and to permit persons to whom the Software is | ||
| furnished to do so, subject to the following conditions: | ||
| The above copyright notice and this permission notice shall be included in all | ||
| copies or substantial portions of the Software. | ||
| THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | ||
| IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | ||
| FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | ||
| AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER | ||
| LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, | ||
| OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE | ||
| SOFTWARE. | ||
| """ | ||
| from datetime import timedelta, datetime, timezone | ||
| import os | ||
| import time | ||
| from osgeo import ogr | ||
| import imageio | ||
| import json | ||
| import requests | ||
| import errno | ||
| import csv | ||
| import logging | ||
| logger=logging.getLogger('adamapi') | ||
| from . import AdamApiError,AdamApiMessage | ||
| class GetData(object): | ||
| def __init__(self,client): | ||
| self.LOG=logger | ||
| self.client=client | ||
| def getData(self,datasetId,startDate=None,endDate=None,asyncron=False,**kwargs): | ||
| """ | ||
| Method to retrive: | ||
| 1) the complete product | ||
| 2) a geographic subset of the product | ||
| 3) a timeseries on a point | ||
| @datasetId required | ||
| @startDate required for point 2 and 3, invalid for point 1 | ||
| @endDate required for point 2 and 3, invalid for point 1 | ||
| @outputFname optional | ||
| @geometry invalid in the case 1 but required in the case 2 and 3 | ||
| @productid invalid in the case 2 and 3 but required in the case 1 | ||
| """ | ||
| params,fname=self.checkParams(datasetId,startDate,endDate,kwargs) | ||
| if "pk" in params: | ||
| status = self._checkStatus(params["pk"],asyncron) | ||
| if status == "running": | ||
| return AdamApiMessage({"pk":params["pk"],"status":status}) | ||
| elif status == "completed": | ||
| try: | ||
| url_download = os.path.join("apis","v2","chart","order",str(params["pk"]),"download") | ||
| response_download = self.client.client(url_download,{},"GET",force_raise=True,enable_stream=True) | ||
| response_download.raise_for_status() | ||
| except requests.exceptions.HTTPError as er: | ||
| url_download = os.path.join("apis","v2","subset","order",str(params["pk"]),"download") | ||
| response_download = self.client.client(url_download,{},"GET",force_raise=True,enable_stream=True) | ||
| except Exception as e: | ||
| self.LOG.exception("Un error occurd : %s"%(e)) | ||
| raise AdamApiMessage( {"error":e} ) | ||
| with open(fname,"wb") as f: | ||
| for chunk in response_download.iter_content(chunk_size=128): | ||
| f.write(chunk) | ||
| return AdamApiMessage({"pk":params["pk"],"status":status,"location":fname}) | ||
| else: | ||
| return AdamApiMessage({"pk":params["pk"],"status":status}) | ||
| if fname is None: | ||
| self.LOG.error("Unprocessable requests with productId and geometry. Execute a request with only once of two params") | ||
| raise AdamApiMessage( {"error":"Unprocessable requests with productId and geometry. Execute a request with only once of two params"}) | ||
| self._checkDirFile(fname) | ||
| if "productId" in params: | ||
| url = os.path.join("apis","v2","opensearch",datasetId.split(":",1)[0],"records/",params["productId"]+"/","download") | ||
| method = "GET" | ||
| isorder=False | ||
| if "geometry" in params: | ||
| isorder=True | ||
| method = "POST" | ||
| try: | ||
| geom = ogr.CreateGeometryFromJson(params["geometry"]) | ||
| except: | ||
| geom = ogr.CreateGeometryFromJson(json.dumps(params["geometry"])) | ||
| if geom.GetGeometryName() == "POINT": | ||
| url = os.path.join("apis","v2","chart","order",datasetId.split(":",1)[0]+"/") | ||
| is_timeseries=True | ||
| else: | ||
| is_timeseries=False | ||
| url = os.path.join("apis","v2","subset","order",datasetId.split(":",1)[0]+"/") | ||
| try: | ||
| response_order = self.client.client(url,params,method,force_raise = True,enable_stream=False) | ||
| response_order.raise_for_status() | ||
| if isorder: | ||
| if response_order.json()["status"] == "started": | ||
| if not asyncron: | ||
| status = self._checkStatus(response_order.json()["pk"],asyncron) | ||
| if status == "completed": | ||
| if is_timeseries: | ||
| url_download = os.path.join("apis","v2","chart","order",str(response_order.json()["pk"]),"download") | ||
| else: | ||
| url_download = os.path.join("apis","v2","subset","order",str(response_order.json()["pk"]),"download") | ||
| try: | ||
| response_download = self.client.client(url_download,{},"GET",force_raise=True,enable_stream=True) | ||
| response_download.raise_for_status() | ||
| with open(fname,"wb") as f: | ||
| for chunk in response_download.iter_content(chunk_size=128): | ||
| f.write(chunk) | ||
| except requests.exceptions.HTTPError as er: | ||
| raise AdamApiMessage( {"error":response_download.json()}) | ||
| except Exception as e: | ||
| self.LOG.exception("Un error occurd : %s"%(e)) | ||
| raise AdamApiMessage( {"error":e} ) | ||
| else: | ||
| return AdamApiMessage(response_order.json()) | ||
| if status == "failed": | ||
| self.LOG.error("Failed request") | ||
| raise AdamApiMessage({"status":status,"error":"Failed Request"}) | ||
| else: | ||
| with open( fname, 'wb' ) as f: | ||
| f.write( response_order.content ) | ||
| except requests.exceptions.HTTPError as er: | ||
| raise AdamApiMessage({"error":response_order.json()}) | ||
| except Exception as e: | ||
| self.LOG.exception("Un error occurd : %s"%(e)) | ||
| raise AdamApiMessage( {"error":e} ) | ||
| return fname | ||
| def checkParams(self,datasetId,start_date,end_date,kwargs): | ||
| params={} | ||
| fname=None | ||
| op_kwargs = kwargs.copy() | ||
| if "productId" in kwargs and "geometry" in kwargs: | ||
| return params , None | ||
| if "outputFname" in kwargs and kwargs["outputFname"] is not None: | ||
| fname = kwargs["outputFname"] | ||
| del(op_kwargs["outputFname"]) | ||
| else: | ||
| fname="adamapiresults/%s"%(datasetId.split(":")[-1]) | ||
| for(key,value)in op_kwargs.items(): | ||
| if key == "geometry": | ||
| if isinstance(value,str): | ||
| params[key]=value | ||
| else: | ||
| params[key]=json.dumps(value) | ||
| else: | ||
| params[key]=value | ||
| if "pk" in kwargs: | ||
| return params,fname | ||
| if "productId" not in kwargs: | ||
| if start_date > end_date: | ||
| raise AdamApiError("Invalid Temporal Filter: startDate > endDate") | ||
| params["start_date"]=start_date | ||
| params["end_date"]=end_date | ||
| params["type"]="adamapi" | ||
| return params,fname | ||
| def _checkStatus(self,oid,asyncron): | ||
| """ | ||
| wrap /status/order | ||
| """ | ||
| animation = "|/-\\" | ||
| idx = 0 | ||
| params={} | ||
| url = os.path.join("apis","v2","subset","order",str(oid),"status") | ||
| if asyncron: | ||
| r = self.client.client(url,params,"GET",force_raise=True) | ||
| status = r.json()["status"] | ||
| else: | ||
| status = "running" | ||
| while status == "running": | ||
| time.sleep(2) | ||
| r = self.client.client(url,params,"GET",force_raise=True) | ||
| status = r.json()["status"] | ||
| if status == "running": | ||
| print(animation[idx % len(animation)], end="\r") | ||
| idx += 1 | ||
| else: | ||
| break | ||
| return status | ||
| def _checkDirFile(self,filename): | ||
| dirname = os.path.dirname( filename ) | ||
| if dirname.strip(): | ||
| try: | ||
| os.makedirs( dirname ) | ||
| except OSError as ose: | ||
| if ose.errno!=errno.EEXIST: | ||
| raise AdamApiError( ose ) |
| """ | ||
| Copyright (c) 2020 MEEO s.r.l. | ||
| Permission is hereby granted, free of charge, to any person obtaining a copy | ||
| of this software and associated documentation files (the "Software"), to deal | ||
| in the Software without restriction, including without limitation the rights | ||
| to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | ||
| copies of the Software, and to permit persons to whom the Software is | ||
| furnished to do so, subject to the following conditions: | ||
| The above copyright notice and this permission notice shall be included in all | ||
| copies or substantial portions of the Software. | ||
| THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | ||
| IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | ||
| FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | ||
| AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER | ||
| LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, | ||
| OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE | ||
| SOFTWARE. | ||
| """ | ||
| from datetime import datetime | ||
| import os | ||
| import logging | ||
| import json | ||
| logger=logging.getLogger('adamapi') | ||
| from . import AdamApiError | ||
| class Search(): | ||
| def __init__(self,client): | ||
| self.LOG=logger | ||
| self.client=client | ||
| def getProducts(self,datasetId,startIndex=0,maxRecords=10,**kwargs): | ||
| """ | ||
| Product Catalogue Search | ||
| @ datasetId, the dataseId ,required | ||
| @ startDate, search start date (%Y-%m-%d) | ||
| @ endDate, search end date (%Y-%m-%d) | ||
| @ maxRecords, number of result showed | ||
| @ startIndex, index of the first result to start showin | ||
| """ | ||
| params={} | ||
| url=os.path.join( 'apis','v2','opensearch',datasetId.split(":",1)[0],'records') | ||
| params['datasetId']=datasetId.split(":",1)[1] | ||
| params["type"]="adamapi" | ||
| for par in kwargs: | ||
| if par == 'geometry': | ||
| if type(kwargs[par]) is str: | ||
| params[par] = kwargs[par] | ||
| elif type(kwargs[par]) is dict: | ||
| params[par] = json.dumps(kwargs[par]) | ||
| elif par == 'startDate' and isinstance( kwargs[par], datetime ): | ||
| params[ 'startDate' ] = kwargs[par].strftime( '%Y-%m-%dT%H:%M:%SZ' ) | ||
| elif par == 'endDate' and isinstance( kwargs[par], datetime ): | ||
| params[ 'endDate' ] = kwargs[par].strftime( '%Y-%m-%dT%H:%M:%SZ' ) | ||
| else: | ||
| params[par]=kwargs[par] | ||
| params["page"]=startIndex | ||
| params["items"]=maxRecords | ||
| if 'geometry' in kwargs: | ||
| self.LOG.info(params) | ||
| response=self.client.client(url,params,"POST").json() | ||
| else: | ||
| response=self.client.client(url,params,"GET").json() | ||
| if 'errors' in response: | ||
| raise AdamApiError(response['errors']) | ||
| else: | ||
| return response |
Alert delta unavailable
Currently unable to show alert delta for PyPI packages.
Improved metrics
- Total package byte prevSize
63153
20.54%- Number of package files
16
23.08%Worsened metrics
- Lines of code
555
-6.25%