Big News: Socket raises $60M Series C at a $1B valuation to secure software supply chains for AI-driven development.Announcement →
github3api
Artifacts are distributable versions of a package. It typically includes the source code and metadata
Advanced tools
github3api - pypi Package Compare versions
| Metadata-Version: 2.1 | ||
| Name: github3api | ||
| Version: 0.1.1 | ||
| Version: 0.1.2 | ||
| Summary: An advanced REST client for the GitHub API | ||
@@ -27,2 +27,3 @@ Home-page: https://github.com/soda480/github3api | ||
| # github3api # | ||
| [](https://github.com/soda480/github3api/actions) | ||
@@ -33,6 +34,7 @@ [](https://codecov.io/gh/soda480/github3api) | ||
| # github3api # | ||
| An advanced REST client for the GitHub API. It is a subclass of [rest3client](https://pypi.org/project/rest3client/) tailored for the GitHub API with special optional directives for GET requests that can return all pages from an endpoint or return a generator that can be iterated over. By default all requests will be retried if ratelimit request limit is reached. | ||
| An advanced REST client for the GitHub API. It is a subclass of [rest3client](https://pypi.org/project/rest3client/) tailored for the GitHub API with special optional directives for GET requests that can return all pages from an endpoint or return a generator that can be iterated over (for paged requests). By default all requests will be retried if ratelimit request limit is reached. | ||
| Support for executing Graphql queries including paging; Graphql queries are also retried if Graphql rate limiting occurs. | ||
| ### Installation ### | ||
@@ -108,2 +110,54 @@ ```bash | ||
| `graphql` - execute graphql query | ||
| ```python | ||
| query = """ | ||
| query($query:String!, $page_size:Int!) { | ||
| search(query: $query, type: REPOSITORY, first: $page_size) { | ||
| repositoryCount | ||
| edges { | ||
| node { | ||
| ... on Repository { | ||
| nameWithOwner | ||
| } | ||
| } | ||
| } | ||
| } | ||
| } | ||
| """ | ||
| variables = {"query": "org:edgexfoundry", "page_size":100} | ||
| client.graphql(query, variables) | ||
| ``` | ||
| `graphql paging` - execute paged graphql query | ||
| ```python | ||
| query = """ | ||
| query ($query: String!, $page_size: Int!, $cursor: String!) { | ||
| search(query: $query, type: REPOSITORY, first: $page_size, after: $cursor) { | ||
| repositoryCount | ||
| pageInfo { | ||
| endCursor | ||
| hasNextPage | ||
| } | ||
| edges { | ||
| cursor | ||
| node { | ||
| ... on Repository { | ||
| nameWithOwner | ||
| } | ||
| } | ||
| } | ||
| } | ||
| } | ||
| """ | ||
| variables = {"query": "org:edgexfoundry", "page_size":100} | ||
| for page in client.graphql(query, variables, page=True, keys='data.search'): | ||
| for repo in page: | ||
| print(repo['node']['nameWithOwner']) | ||
| ``` | ||
| For Graphql paged queries: | ||
| - the query should include the necessary pageInfo and cursor attributes | ||
| - the keys method argument is a dot annotated string that is used to access the resulting dictionary response object | ||
| - the query is retried every 60 seconds (for up to an hour) if a ratelimit occur | ||
| ### Projects using `github3api` ### | ||
@@ -143,3 +197,3 @@ | ||
| -e https_proxy \ | ||
| -v $PWD:/github3api \ | ||
| -v $PWD:/code \ | ||
| github3api:latest \ | ||
@@ -146,0 +200,0 @@ /bin/sh |
@@ -21,2 +21,3 @@ | ||
| from retrying import retry | ||
| from rest3client import RESTclient | ||
@@ -33,4 +34,17 @@ from requests.exceptions import HTTPError | ||
| DEFAULT_PAGE_SIZE = 30 | ||
| DEFAULT_GRAPHQL_PAGE_SIZE = 100 | ||
| class GraphqlRateLimitError(Exception): | ||
| """ GraphQL Rate Limit Error | ||
| """ | ||
| pass | ||
| class GraphqlError(Exception): | ||
| """ GraphQL Error | ||
| """ | ||
| pass | ||
| class GitHubAPI(RESTclient): | ||
@@ -226,13 +240,80 @@ """ An advanced REST client for the GitHub API | ||
| @staticmethod | ||
| def _retry_chunkedencodingerror_error(exception): | ||
| """ return True if exception is ChunkedEncodingError, False otherwise | ||
| retry: | ||
| wait_fixed:10000 | ||
| stop_max_attempt_number:120 | ||
| def clear_cursor(query, cursor): | ||
| """ return query with all cursor references removed if no cursor | ||
| """ | ||
| logger.debug(f"checking if '{type(exception).__name__}' exception is a ChunkedEncodingError error") | ||
| if isinstance(exception, ChunkedEncodingError): | ||
| logger.info('ratelimit error encountered - retrying request in 10 seconds') | ||
| if not cursor: | ||
| query = query.replace('after: $cursor', '') | ||
| query = query.replace('$cursor: String!', '') | ||
| return query | ||
| @staticmethod | ||
| def sanitize_query(query): | ||
| """ sanitize query | ||
| """ | ||
| return query.replace('\n', ' ').replace(' ', '').strip() | ||
| @staticmethod | ||
| def raise_if_error(response): | ||
| """ raise GraphqlRateLimitError if error exists in errors | ||
| """ | ||
| if 'errors' in response: | ||
| logger.debug(f'errors detected in graphql response: {response}') | ||
| for error in response['errors']: | ||
| if error.get('type', '') == 'RATE_LIMITED': | ||
| raise GraphqlRateLimitError(error.get('message', '')) | ||
| raise GraphqlError(response['errors'][0]['message']) | ||
| @staticmethod | ||
| def get_value(data, keys): | ||
| """ return value represented by keys dot notated string from data dictionary | ||
| """ | ||
| if '.' in keys: | ||
| key, rest = keys.split('.', 1) | ||
| if key in data: | ||
| return GitHubAPI.get_value(data[key], rest) | ||
| raise KeyError(f'dictionary does not have key {key}') | ||
| else: | ||
| return data[keys] | ||
| def _get_graphql_page(self, query, variables, keys): | ||
| """ return generator that yields page from graphql response | ||
| """ | ||
| variables['page_size'] = DEFAULT_GRAPHQL_PAGE_SIZE | ||
| variables['cursor'] = '' | ||
| while True: | ||
| updated_query = GitHubAPI.clear_cursor(query, variables['cursor']) | ||
| response = self.post('/graphql', json={'query': updated_query, 'variables': variables}) | ||
| GitHubAPI.raise_if_error(response) | ||
| yield GitHubAPI.get_value(response, f'{keys}.edges') | ||
| page_info = GitHubAPI.get_value(response, f'{keys}.pageInfo') | ||
| has_next_page = page_info['hasNextPage'] | ||
| if not has_next_page: | ||
| logger.debug('no more pages') | ||
| break | ||
| variables['cursor'] = page_info['endCursor'] | ||
| def check_graphqlratelimiterror(exception): | ||
| """ return True if exception is GraphQL Rate Limit Error, False otherwise | ||
| """ | ||
| logger.debug(f"checking if '{type(exception).__name__}' exception is a GraphqlRateLimitError") | ||
| if isinstance(exception, (GraphqlRateLimitError, TypeError)): | ||
| logger.debug('exception is a GraphqlRateLimitError - retrying request in 60 seconds') | ||
| return True | ||
| logger.debug(f'exception is not a ratelimit error: {exception}') | ||
| logger.debug(f'exception is not a GraphqlRateLimitError: {exception}') | ||
| return False | ||
| @retry(retry_on_exception=check_graphqlratelimiterror, wait_fixed=60000, stop_max_attempt_number=60) | ||
| def graphql(self, query, variables, page=False, keys=None): | ||
| """ execute graphql query and return response or paged response if page is True | ||
| """ | ||
| query = GitHubAPI.sanitize_query(query) | ||
| if page: | ||
| response = self._get_graphql_page(query, variables, keys) | ||
| else: | ||
| updated_query = GitHubAPI.clear_cursor(query, variables.get('cursor')) | ||
| response = self.post('/graphql', json={'query': updated_query, 'variables': variables}) | ||
| GitHubAPI.raise_if_error(response) | ||
| return response | ||
| check_graphqlratelimiterror = staticmethod(check_graphqlratelimiterror) |
+58
-4
| Metadata-Version: 2.1 | ||
| Name: github3api | ||
| Version: 0.1.1 | ||
| Version: 0.1.2 | ||
| Summary: An advanced REST client for the GitHub API | ||
@@ -27,2 +27,3 @@ Home-page: https://github.com/soda480/github3api | ||
| # github3api # | ||
| [](https://github.com/soda480/github3api/actions) | ||
@@ -33,6 +34,7 @@ [](https://codecov.io/gh/soda480/github3api) | ||
| # github3api # | ||
| An advanced REST client for the GitHub API. It is a subclass of [rest3client](https://pypi.org/project/rest3client/) tailored for the GitHub API with special optional directives for GET requests that can return all pages from an endpoint or return a generator that can be iterated over. By default all requests will be retried if ratelimit request limit is reached. | ||
| An advanced REST client for the GitHub API. It is a subclass of [rest3client](https://pypi.org/project/rest3client/) tailored for the GitHub API with special optional directives for GET requests that can return all pages from an endpoint or return a generator that can be iterated over (for paged requests). By default all requests will be retried if ratelimit request limit is reached. | ||
| Support for executing Graphql queries including paging; Graphql queries are also retried if Graphql rate limiting occurs. | ||
| ### Installation ### | ||
@@ -108,2 +110,54 @@ ```bash | ||
| `graphql` - execute graphql query | ||
| ```python | ||
| query = """ | ||
| query($query:String!, $page_size:Int!) { | ||
| search(query: $query, type: REPOSITORY, first: $page_size) { | ||
| repositoryCount | ||
| edges { | ||
| node { | ||
| ... on Repository { | ||
| nameWithOwner | ||
| } | ||
| } | ||
| } | ||
| } | ||
| } | ||
| """ | ||
| variables = {"query": "org:edgexfoundry", "page_size":100} | ||
| client.graphql(query, variables) | ||
| ``` | ||
| `graphql paging` - execute paged graphql query | ||
| ```python | ||
| query = """ | ||
| query ($query: String!, $page_size: Int!, $cursor: String!) { | ||
| search(query: $query, type: REPOSITORY, first: $page_size, after: $cursor) { | ||
| repositoryCount | ||
| pageInfo { | ||
| endCursor | ||
| hasNextPage | ||
| } | ||
| edges { | ||
| cursor | ||
| node { | ||
| ... on Repository { | ||
| nameWithOwner | ||
| } | ||
| } | ||
| } | ||
| } | ||
| } | ||
| """ | ||
| variables = {"query": "org:edgexfoundry", "page_size":100} | ||
| for page in client.graphql(query, variables, page=True, keys='data.search'): | ||
| for repo in page: | ||
| print(repo['node']['nameWithOwner']) | ||
| ``` | ||
| For Graphql paged queries: | ||
| - the query should include the necessary pageInfo and cursor attributes | ||
| - the keys method argument is a dot annotated string that is used to access the resulting dictionary response object | ||
| - the query is retried every 60 seconds (for up to an hour) if a ratelimit occur | ||
| ### Projects using `github3api` ### | ||
@@ -143,3 +197,3 @@ | ||
| -e https_proxy \ | ||
| -v $PWD:/github3api \ | ||
| -v $PWD:/code \ | ||
| github3api:latest \ | ||
@@ -146,0 +200,0 @@ /bin/sh |
+2
-2
@@ -24,5 +24,5 @@ #!/usr/bin/env python | ||
| name = 'github3api', | ||
| version = '0.1.1', | ||
| version = '0.1.2', | ||
| description = 'An advanced REST client for the GitHub API', | ||
| long_description = "[](https://github.com/soda480/github3api/actions)\n[](https://codecov.io/gh/soda480/github3api)\n[](https://frontend.code-inspector.com/project/13337/dashboard)\n[](https://badge.fury.io/py/github3api)\n\n# github3api #\nAn advanced REST client for the GitHub API. It is a subclass of [rest3client](https://pypi.org/project/rest3client/) tailored for the GitHub API with special optional directives for GET requests that can return all pages from an endpoint or return a generator that can be iterated over. By default all requests will be retried if ratelimit request limit is reached.\n\n\n### Installation ###\n```bash\npip install github3api\n```\n\n### Example Usage ###\n\n```python\n>>> from github3api import GitHubAPI\n```\n\n`GitHubAPI` instantiation\n```python\n# instantiate using no-auth\n>>> client = GitHubAPI()\n\n# instantiate using a token\n>>> client = GitHubAPI(bearer_token='****************')\n```\n\n`GET` request\n```python\n# GET request - return JSON response\n>>> client.get('/rate_limit')['resources']['core']\n{'limit': 60, 'remaining': 37, 'reset': 1588898701}\n\n# GET request - return raw resonse\n>>> client.get('/rate_limit', raw_response=True)\n<Response [200]>\n```\n\n`POST` request\n```python\n>>> client.post('/user/repos', json={'name': 'test-repo1'})['full_name']\n'soda480/test-repo1'\n\n>>> client.post('/repos/soda480/test-repo1/labels', json={'name': 'label1', 'color': '#006b75'})['url']\n'https://api.github.com/repos/soda480/test-repo1/labels/label1'\n```\n\n`PATCH` request\n```python\n>>> client.patch('/repos/soda480/test-repo1/labels/label1', json={'description': 'my label'})['url']\n'https://api.github.com/repos/soda480/test-repo1/labels/label1'\n```\n\n`DELETE` request\n```python \n>>> client.delete('/repos/soda480/test-repo1')\n```\n\n`GET all` directive - Get all pages from an endpoint and return list containing only matching attributes\n```python\nfor repo in client.get('/user/repos', _get='all', _attributes=['full_name']):\n print(repo['full_name'])\n```\n\n`GET page` directive - Yield a page from endpoint\n```python\nfor page in client.get('/user/repos', _get='page'):\n for repo in page:\n print(repo['full_name'])\n```\n\n`total` - Get total number of resources at given endpoint\n```python\nprint(client.total('/user/repos'))\n6218\n```\n\n### Projects using `github3api` ###\n\n* [edgexfoundry/sync-github-labels](https://github.com/edgexfoundry/cd-management/tree/git-label-sync) A script that synchronizes GitHub labels and milestones\n\n* [edgexfoundry/prune-github-tags](https://github.com/edgexfoundry/cd-management/tree/prune-github-tags) A script that prunes GitHub pre-release tags\n\n* [edgexfoundry/create-github-release](https://github.com/edgexfoundry/cd-management/tree/create-github-release) A script to facilitate creation of GitHub releases\n\n* [soda480/prepbadge](https://github.com/soda480/prepbadge) A script that creates multiple pull request workflows to update a target organization repos with badges\n\n* [soda480/github-contributions](https://github.com/soda480/github-contributions) A script to get contribution metrics for all members of a GitHub organization using the GitHub GraphQL API\n\n\n### Development ###\n\nEnsure the latest version of Docker is installed on your development server. Fork and clone the repository.\n\nBuild the Docker image:\n```sh\ndocker image build \\\n--target build-image \\\n--build-arg http_proxy \\\n--build-arg https_proxy \\\n-t \\\ngithub3api:latest .\n```\n\nRun the Docker container:\n```sh\ndocker container run \\\n--rm \\\n-it \\\n-e http_proxy \\\n-e https_proxy \\\n-v $PWD:/github3api \\\ngithub3api:latest \\\n/bin/sh\n```\n\nExecute the build:\n```sh\npyb -X\n```\n\nNOTE: commands above assume working behind a proxy, if not then the proxy arguments to both the docker build and run commands can be removed.\n", | ||
| long_description = '# github3api #\n[](https://github.com/soda480/github3api/actions)\n[](https://codecov.io/gh/soda480/github3api)\n[](https://frontend.code-inspector.com/project/13337/dashboard)\n[](https://badge.fury.io/py/github3api)\n\nAn advanced REST client for the GitHub API. It is a subclass of [rest3client](https://pypi.org/project/rest3client/) tailored for the GitHub API with special optional directives for GET requests that can return all pages from an endpoint or return a generator that can be iterated over (for paged requests). By default all requests will be retried if ratelimit request limit is reached.\n\nSupport for executing Graphql queries including paging; Graphql queries are also retried if Graphql rate limiting occurs.\n\n\n### Installation ###\n```bash\npip install github3api\n```\n\n### Example Usage ###\n\n```python\n>>> from github3api import GitHubAPI\n```\n\n`GitHubAPI` instantiation\n```python\n# instantiate using no-auth\n>>> client = GitHubAPI()\n\n# instantiate using a token\n>>> client = GitHubAPI(bearer_token=\'****************\')\n```\n\n`GET` request\n```python\n# GET request - return JSON response\n>>> client.get(\'/rate_limit\')[\'resources\'][\'core\']\n{\'limit\': 60, \'remaining\': 37, \'reset\': 1588898701}\n\n# GET request - return raw resonse\n>>> client.get(\'/rate_limit\', raw_response=True)\n<Response [200]>\n```\n\n`POST` request\n```python\n>>> client.post(\'/user/repos\', json={\'name\': \'test-repo1\'})[\'full_name\']\n\'soda480/test-repo1\'\n\n>>> client.post(\'/repos/soda480/test-repo1/labels\', json={\'name\': \'label1\', \'color\': \'#006b75\'})[\'url\']\n\'https://api.github.com/repos/soda480/test-repo1/labels/label1\'\n```\n\n`PATCH` request\n```python\n>>> client.patch(\'/repos/soda480/test-repo1/labels/label1\', json={\'description\': \'my label\'})[\'url\']\n\'https://api.github.com/repos/soda480/test-repo1/labels/label1\'\n```\n\n`DELETE` request\n```python \n>>> client.delete(\'/repos/soda480/test-repo1\')\n```\n\n`GET all` directive - Get all pages from an endpoint and return list containing only matching attributes\n```python\nfor repo in client.get(\'/user/repos\', _get=\'all\', _attributes=[\'full_name\']):\n print(repo[\'full_name\'])\n```\n\n`GET page` directive - Yield a page from endpoint\n```python\nfor page in client.get(\'/user/repos\', _get=\'page\'):\n for repo in page:\n print(repo[\'full_name\'])\n```\n\n`total` - Get total number of resources at given endpoint\n```python\nprint(client.total(\'/user/repos\'))\n6218\n```\n\n`graphql` - execute graphql query\n```python\nquery = """\n query($query:String!, $page_size:Int!) {\n search(query: $query, type: REPOSITORY, first: $page_size) {\n repositoryCount\n edges {\n node {\n ... on Repository {\n nameWithOwner\n }\n }\n }\n }\n }\n"""\nvariables = {"query": "org:edgexfoundry", "page_size":100}\nclient.graphql(query, variables)\n```\n\n`graphql paging` - execute paged graphql query\n```python\nquery = """\n query ($query: String!, $page_size: Int!, $cursor: String!) {\n search(query: $query, type: REPOSITORY, first: $page_size, after: $cursor) {\n repositoryCount\n pageInfo {\n endCursor\n hasNextPage\n }\n edges {\n cursor\n node {\n ... on Repository {\n nameWithOwner\n }\n }\n }\n }\n }\n"""\nvariables = {"query": "org:edgexfoundry", "page_size":100}\nfor page in client.graphql(query, variables, page=True, keys=\'data.search\'):\n for repo in page:\n print(repo[\'node\'][\'nameWithOwner\'])\n```\n\nFor Graphql paged queries:\n- the query should include the necessary pageInfo and cursor attributes\n- the keys method argument is a dot annotated string that is used to access the resulting dictionary response object\n- the query is retried every 60 seconds (for up to an hour) if a ratelimit occur\n\n### Projects using `github3api` ###\n\n* [edgexfoundry/sync-github-labels](https://github.com/edgexfoundry/cd-management/tree/git-label-sync) A script that synchronizes GitHub labels and milestones\n\n* [edgexfoundry/prune-github-tags](https://github.com/edgexfoundry/cd-management/tree/prune-github-tags) A script that prunes GitHub pre-release tags\n\n* [edgexfoundry/create-github-release](https://github.com/edgexfoundry/cd-management/tree/create-github-release) A script to facilitate creation of GitHub releases\n\n* [soda480/prepbadge](https://github.com/soda480/prepbadge) A script that creates multiple pull request workflows to update a target organization repos with badges\n\n* [soda480/github-contributions](https://github.com/soda480/github-contributions) A script to get contribution metrics for all members of a GitHub organization using the GitHub GraphQL API\n\n\n### Development ###\n\nEnsure the latest version of Docker is installed on your development server. Fork and clone the repository.\n\nBuild the Docker image:\n```sh\ndocker image build \\\n--target build-image \\\n--build-arg http_proxy \\\n--build-arg https_proxy \\\n-t \\\ngithub3api:latest .\n```\n\nRun the Docker container:\n```sh\ndocker container run \\\n--rm \\\n-it \\\n-e http_proxy \\\n-e https_proxy \\\n-v $PWD:/code \\\ngithub3api:latest \\\n/bin/sh\n```\n\nExecute the build:\n```sh\npyb -X\n```\n\nNOTE: commands above assume working behind a proxy, if not then the proxy arguments to both the docker build and run commands can be removed.\n', | ||
| long_description_content_type = 'text/markdown', | ||
@@ -29,0 +29,0 @@ classifiers = [ |
Alert delta unavailable
Currently unable to show alert delta for PyPI packages.
Improved metrics
- Total package byte prevSize
33231
30.09%- Lines of code
362
25.69%