Big News: Socket raises $60M Series C at a $1B valuation to secure software supply chains for AI-driven development.Announcement →
githubpullrequests
Artifacts are distributable versions of a package. It typically includes the source code and metadata
Advanced tools
githubpullrequests - pypi Package Compare versions
+21
-4
| Metadata-Version: 2.1 | ||
| Name: githubpullrequests | ||
| Version: 0.1.0 | ||
| Version: 0.2.0 | ||
| Summary: Create Pull Requests, using GitHub API and a list of repositories | ||
@@ -12,3 +12,12 @@ Home-page: https://github.com/evandrocoan/githubpullrequests | ||
| Created as a local alternative to: | ||
| 1. https://github.com/backstrokeapp/server/issues/102#issuecomment-451306979 Stopped working a few days ago with Recived an error: undefined | ||
| 1. https://github.com/wei/pull/issues/76#issue-397123888 Do not reset hard my default branch | ||
| ### Installation | ||
| Either clone this repository and run `python setup.py develop` or just use `pip install githubpullrequests` | ||
| ### Usage | ||
@@ -19,5 +28,5 @@ | ||
| usage: githubpullrequests [-h] [-f FILE] [-t TOKEN] [-mr MAXIMUM_REPOSITORIES] | ||
| [-c] [-s] | ||
| [-c] [-d] [-s] [-ei ENABLE_ISSUES] [-as ADD_STARS] | ||
| Create Pull Requests, using GitHub API and a list of repositories | ||
| Create Pull Requests, using GitHub API and a list of repositories. | ||
@@ -31,3 +40,3 @@ optional arguments: | ||
| only contents the file can have is the token, | ||
| optionally with a tralling new line. | ||
| optionally with a trailing new line. | ||
| -mr MAXIMUM_REPOSITORIES, --maximum-repositories MAXIMUM_REPOSITORIES | ||
@@ -39,2 +48,5 @@ The maximum count of repositories/requests to process | ||
| soons as possible. | ||
| -d, --dry-run Do a rehearsal of a performance or procedure instead | ||
| of the real one i.e., do not create any pull requests, | ||
| but simulates/pretends to do so. | ||
| -s, --synced-repositories | ||
@@ -46,2 +58,7 @@ Reports which repositories not Synchronized with Pull | ||
| required to know all available repositories. | ||
| -ei ENABLE_ISSUES, --enable-issues ENABLE_ISSUES | ||
| Enable the issue tracker on all repositories for the | ||
| given user. | ||
| -as ADD_STARS, --add-stars ADD_STARS | ||
| Add a star on all repositories for the given user. | ||
| ``` | ||
@@ -48,0 +65,0 @@ |
+20
-3
@@ -5,3 +5,12 @@ # Create Pull Requests | ||
| Created as a local alternative to: | ||
| 1. https://github.com/backstrokeapp/server/issues/102#issuecomment-451306979 Stopped working a few days ago with Recived an error: undefined | ||
| 1. https://github.com/wei/pull/issues/76#issue-397123888 Do not reset hard my default branch | ||
| ### Installation | ||
| Either clone this repository and run `python setup.py develop` or just use `pip install githubpullrequests` | ||
| ### Usage | ||
@@ -12,5 +21,5 @@ | ||
| usage: githubpullrequests [-h] [-f FILE] [-t TOKEN] [-mr MAXIMUM_REPOSITORIES] | ||
| [-c] [-s] | ||
| [-c] [-d] [-s] [-ei ENABLE_ISSUES] [-as ADD_STARS] | ||
| Create Pull Requests, using GitHub API and a list of repositories | ||
| Create Pull Requests, using GitHub API and a list of repositories. | ||
@@ -24,3 +33,3 @@ optional arguments: | ||
| only contents the file can have is the token, | ||
| optionally with a tralling new line. | ||
| optionally with a trailing new line. | ||
| -mr MAXIMUM_REPOSITORIES, --maximum-repositories MAXIMUM_REPOSITORIES | ||
@@ -32,2 +41,5 @@ The maximum count of repositories/requests to process | ||
| soons as possible. | ||
| -d, --dry-run Do a rehearsal of a performance or procedure instead | ||
| of the real one i.e., do not create any pull requests, | ||
| but simulates/pretends to do so. | ||
| -s, --synced-repositories | ||
@@ -39,2 +51,7 @@ Reports which repositories not Synchronized with Pull | ||
| required to know all available repositories. | ||
| -ei ENABLE_ISSUES, --enable-issues ENABLE_ISSUES | ||
| Enable the issue tracker on all repositories for the | ||
| given user. | ||
| -as ADD_STARS, --add-stars ADD_STARS | ||
| Add a star on all repositories for the given user. | ||
| ``` | ||
@@ -41,0 +58,0 @@ |
+1
-1
@@ -62,3 +62,3 @@ #!/usr/bin/env python3 | ||
| # | ||
| version = '0.1.0' | ||
| version = '0.2.0' | ||
@@ -65,0 +65,0 @@ install_requires=[ |
| Metadata-Version: 2.1 | ||
| Name: githubpullrequests | ||
| Version: 0.1.0 | ||
| Version: 0.2.0 | ||
| Summary: Create Pull Requests, using GitHub API and a list of repositories | ||
@@ -12,3 +12,12 @@ Home-page: https://github.com/evandrocoan/githubpullrequests | ||
| Created as a local alternative to: | ||
| 1. https://github.com/backstrokeapp/server/issues/102#issuecomment-451306979 Stopped working a few days ago with Recived an error: undefined | ||
| 1. https://github.com/wei/pull/issues/76#issue-397123888 Do not reset hard my default branch | ||
| ### Installation | ||
| Either clone this repository and run `python setup.py develop` or just use `pip install githubpullrequests` | ||
| ### Usage | ||
@@ -19,5 +28,5 @@ | ||
| usage: githubpullrequests [-h] [-f FILE] [-t TOKEN] [-mr MAXIMUM_REPOSITORIES] | ||
| [-c] [-s] | ||
| [-c] [-d] [-s] [-ei ENABLE_ISSUES] [-as ADD_STARS] | ||
| Create Pull Requests, using GitHub API and a list of repositories | ||
| Create Pull Requests, using GitHub API and a list of repositories. | ||
@@ -31,3 +40,3 @@ optional arguments: | ||
| only contents the file can have is the token, | ||
| optionally with a tralling new line. | ||
| optionally with a trailing new line. | ||
| -mr MAXIMUM_REPOSITORIES, --maximum-repositories MAXIMUM_REPOSITORIES | ||
@@ -39,2 +48,5 @@ The maximum count of repositories/requests to process | ||
| soons as possible. | ||
| -d, --dry-run Do a rehearsal of a performance or procedure instead | ||
| of the real one i.e., do not create any pull requests, | ||
| but simulates/pretends to do so. | ||
| -s, --synced-repositories | ||
@@ -46,2 +58,7 @@ Reports which repositories not Synchronized with Pull | ||
| required to know all available repositories. | ||
| -ei ENABLE_ISSUES, --enable-issues ENABLE_ISSUES | ||
| Enable the issue tracker on all repositories for the | ||
| given user. | ||
| -as ADD_STARS, --add-stars ADD_STARS | ||
| Add a star on all repositories for the given user. | ||
| ``` | ||
@@ -48,0 +65,0 @@ |
@@ -43,4 +43,6 @@ #!/usr/bin/env python3 | ||
| import json | ||
| import time | ||
| import github | ||
| import requests | ||
| import argparse | ||
@@ -61,2 +63,3 @@ import contextlib | ||
| from debug_tools.utilities import move_to_dict_beginning | ||
| from debug_tools.third_part import get_section_option | ||
| from debug_tools.estimated_time_left import sequence_timer | ||
@@ -68,32 +71,34 @@ from debug_tools.estimated_time_left import progress_info | ||
| headers = {} | ||
| MAXIMUM_WORSPACES_ENTRIES = 100 | ||
| g_is_already_running = False | ||
| log = getLogger( 127, __name__ ) | ||
| log = getLogger( 127, "" ) | ||
| def main(): | ||
| github_token = os.environ.get( 'GITHUBPULLREQUESTS_TOKEN', "" ) | ||
| gitmodules_files = [] | ||
| synced_repositories = False | ||
| maximum_repositories = 0 | ||
| github_token = os.environ.get( 'GITHUBPULLREQUESTS_TOKEN', "" ).strip() | ||
| # https://stackoverflow.com/questions/6382804/how-to-use-getopt-optarg-in-python-how-to-shift | ||
| argumentParser = argparse.ArgumentParser( description='Create Pull Requests, using GitHub API and a list of repositories' ) | ||
| argumentParser = argparse.ArgumentParser( description='Create Pull Requests, using GitHub API and a list of repositories.' ) | ||
| argumentParser.add_argument( "-f", "--file", action="append", | ||
| argumentParser.add_argument( "-f", "--file", action="append", default=[], | ||
| help="The file with the repositories informations" ) | ||
| argumentParser.add_argument( "-t", "--token", action="store", | ||
| argumentParser.add_argument( "-t", "--token", action="store", default="", | ||
| help="GitHub token with `public_repos` access, or the path " | ||
| "to a file with the Github token in plain text. The only contents " | ||
| "the file can have is the token, optionally with a tralling new line." ) | ||
| "the file can have is the token, optionally with a trailing new line." ) | ||
| argumentParser.add_argument( "-mr", "--maximum-repositories", action="store", type=int, | ||
| argumentParser.add_argument( "-mr", "--maximum-repositories", action="store", type=int, default=0, | ||
| help="The maximum count of repositories/requests to process per file." ) | ||
| argumentParser.add_argument( "-c", "--cancel-operation", action="store_true", | ||
| argumentParser.add_argument( "-c", "--cancel-operation", action="store_true", default=False, | ||
| help="If there is some batch operation running, cancel it as soons as possible." ) | ||
| argumentParser.add_argument( "-s", "--synced-repositories", action="store_true", | ||
| argumentParser.add_argument( "-d", "--dry-run", action="store_true", default=False, | ||
| help="Do a rehearsal of a performance or procedure instead of the real one " | ||
| "i.e., do not create any pull requests, but simulates/pretends to do so." ) | ||
| argumentParser.add_argument( "-s", "--synced-repositories", action="store_true", default=False, | ||
| help="Reports which repositories not Synchronized with Pull Requests. " | ||
@@ -104,2 +109,8 @@ "This also resets/skips any last session saved due old throw/raised exceptions, " | ||
| argumentParser.add_argument( "-ei", "--enable-issues", action="store", default="", | ||
| help="Enable the issue tracker on all repositories for the given user." ) | ||
| argumentParser.add_argument( "-as", "--add-stars", action="store", default="", | ||
| help="Add a star on all repositories for the given user." ) | ||
| argumentsNamespace = argumentParser.parse_args() | ||
@@ -115,30 +126,43 @@ # log( 1, argumentsNamespace ) | ||
| if argumentsNamespace.synced_repositories: | ||
| synced_repositories = argumentsNamespace.synced_repositories | ||
| if github_token: | ||
| global headers | ||
| if os.path.exists( github_token ): | ||
| with open( github_token, 'r', ) as input_file: | ||
| github_token = input_file.read() | ||
| if argumentsNamespace.maximum_repositories: | ||
| maximum_repositories = argumentsNamespace.maximum_repositories | ||
| github_token = github_token.strip() | ||
| headers = { "Authorization": f"Bearer {github_token}" } | ||
| log_ratelimit(headers) | ||
| if argumentsNamespace.file: | ||
| gitmodules_files = argumentsNamespace.file | ||
| else: | ||
| log.clean( "Error: Missing required command line argument `-f/--file`" ) | ||
| log.clean( "Error: Missing required command line argument `-t/--token`" ) | ||
| argumentParser.print_help() | ||
| return | ||
| pull_requester = PullRequester( github_token, maximum_repositories, synced_repositories ) | ||
| pull_requester.parse_gitmodules( gitmodules_files ) | ||
| pull_requester.publish_report() | ||
| if argumentsNamespace.enable_issues: | ||
| enable_github_issue_tracker( argumentsNamespace.enable_issues ) | ||
| if argumentsNamespace.add_stars: | ||
| add_stars_on_github_repositories( argumentsNamespace.add_stars ) | ||
| if argumentsNamespace.file: | ||
| pull_requester = PullRequester( | ||
| github_token, | ||
| argumentsNamespace.maximum_repositories, | ||
| argumentsNamespace.synced_repositories, | ||
| argumentsNamespace.dry_run | ||
| ) | ||
| pull_requester.parse_gitmodules( argumentsNamespace.file ) | ||
| pull_requester.publish_report() | ||
| log_ratelimit(headers) | ||
| class PullRequester(object): | ||
| def __init__(self, github_token, maximum_repositories=0, synced_repositories=False): | ||
| def __init__(self, github_token, maximum_repositories=0, synced_repositories=False, is_dry_run=False): | ||
| super(PullRequester, self).__init__() | ||
| self.is_dry_run = is_dry_run | ||
| self.github_token = github_token | ||
| if os.path.exists( github_token ): | ||
| with open( github_token, 'r', ) as input_file: | ||
| github_token = input_file.read() | ||
| if synced_repositories: | ||
@@ -155,3 +179,2 @@ self.lastSection = OrderedDict() | ||
| self.github_token = github_token.strip() | ||
| self.maximum_repositories = maximum_repositories | ||
@@ -285,18 +308,12 @@ self.synced_repositories = synced_repositories | ||
| if not upstream_user or not upstream_repository: | ||
| log( 1, "Skipping %s because the upstream is not defined...", section ) | ||
| self.skipped_repositories.append( "%s -> %s" % ( downstream_name, section ) ) | ||
| continue | ||
| branches = get_section_option( section, "branches", config_parser ) | ||
| local_branch, upstream_branch = parser_branches( branches ) | ||
| full_upstream_name = "{}/{}@{}".format( upstream_user, upstream_repository, upstream_branch ) | ||
| full_downstream_name = "{} -> {}".format( downstream_name, section ) | ||
| log( 1, branches ) | ||
| log( 1, 'upstream', upstream ) | ||
| log( 1, 'downstream', downstream ) | ||
| log( 1, 'upstream', full_upstream_name ) | ||
| log( 1, 'downstream', full_downstream_name ) | ||
| if not local_branch or not upstream_branch: | ||
| log.newline( count=3 ) | ||
| log( 1, "ERROR! Invalid branches `%s`", branches ) | ||
| if not downstream_user or not downstream_repository: | ||
@@ -306,7 +323,10 @@ log.newline( count=3 ) | ||
| fork_user = self.github_api.get_user( downstream_user ) | ||
| fork_repo = fork_user.get_repo( downstream_repository ) | ||
| full_upstream_name = "{}/{}@{}".format( upstream_user, upstream_repository, upstream_branch ) | ||
| full_downstream_name = "{} -> {}".format( downstream_name, section ) | ||
| try: | ||
| fork_user = self.github_api.get_user( downstream_user ) | ||
| fork_repo = fork_user.get_repo( downstream_repository ) | ||
| except github.GithubException as error: | ||
| self._register_error_reason( full_downstream_name, error ) | ||
| continue | ||
| self.downstream_users.add( downstream_user ) | ||
@@ -316,17 +336,30 @@ self.parsed_repositories.add( downstream_name ) | ||
| if not upstream_user or not upstream_repository: | ||
| log( 1, "Skipping %s because the upstream is not defined...", section ) | ||
| self.skipped_repositories.append( "%s -> %s" % ( downstream_name, section ) ) | ||
| continue | ||
| if not local_branch or not upstream_branch: | ||
| log.newline( count=3 ) | ||
| log( 1, "ERROR! Invalid branches `%s`", branches ) | ||
| try: | ||
| fork_pullrequest = fork_repo.create_pull( | ||
| "Update from {}".format( full_upstream_name ), | ||
| wrap_text( r""" | ||
| The upstream repository `{}` has some new changes that aren't in this fork. | ||
| So, here they are, ready to be merged! | ||
| if self.is_dry_run: | ||
| fork_pullrequest = fork_repo.url | ||
| This Pull Request was created programmatically by the | ||
| [githubpullrequests](https://github.com/evandrocoan/githubpullrequests). | ||
| """.format( full_upstream_name ), single_lines=True, ), | ||
| local_branch, | ||
| '{}:{}'.format( upstream_user, upstream_branch ), | ||
| False | ||
| ) | ||
| else: | ||
| fork_pullrequest = fork_repo.create_pull( | ||
| "Update from {}".format( full_upstream_name ), | ||
| wrap_text( r""" | ||
| The upstream repository `{}` has some new changes that aren't in this fork. | ||
| So, here they are, ready to be merged! | ||
| This Pull Request was created programmatically by the | ||
| [githubpullrequests](https://github.com/evandrocoan/githubpullrequests). | ||
| """.format( full_upstream_name ), single_lines=True, ), | ||
| local_branch, | ||
| '{}:{}'.format( upstream_user, upstream_branch ), | ||
| False | ||
| ) | ||
| # Then play with your Github objects | ||
@@ -336,17 +369,21 @@ successful_resquests += 1 | ||
| self.repositories_results['Successfully Created'].append(full_downstream_name) | ||
| fork_pullrequest.add_to_labels( "backstroke" ) | ||
| self.repositories_results['Successfully Created'].append( full_downstream_name ) | ||
| if not self.is_dry_run: fork_pullrequest.add_to_labels( "backstroke" ) | ||
| except github.GithubException as error: | ||
| error = "%s, %s" % (full_downstream_name, str( error ) ) | ||
| log( 1, 'Skipping... %s', error ) | ||
| self._register_error_reason( full_downstream_name, error ) | ||
| continue | ||
| for reason in self.skip_reasons: | ||
| if reason in error: | ||
| self.repositories_results[reason].append(full_downstream_name) | ||
| break | ||
| def _register_error_reason(self, full_downstream_name, error): | ||
| error = "%s, %s" % (full_downstream_name, str( error ) ) | ||
| log( 1, 'Skipping... %s', error ) | ||
| else: | ||
| self.repositories_results['Unknown Reason'].append(error) | ||
| for reason in self.skip_reasons: | ||
| if reason in error: | ||
| self.repositories_results[reason].append(full_downstream_name) | ||
| break | ||
| else: | ||
| self.repositories_results['Unknown Reason'].append(error) | ||
| def publish_report(self): | ||
@@ -427,3 +464,3 @@ log.newline() | ||
| log.newline() | ||
| log.clean(' Renamed Repositories:') | ||
| log.clean(' Possible Renamed Repositories:') | ||
@@ -459,15 +496,11 @@ index = 0 | ||
| if matches: | ||
| return matches.group(1), matches.group(2) | ||
| user = matches.group(1) | ||
| repository = matches.group(2) | ||
| if repository.endswith('.git'): repository = repository[:-4] | ||
| return user, repository | ||
| return "", "" | ||
| def get_section_option(section, option, configSettings): | ||
| if configSettings.has_option( section, option ): | ||
| return configSettings.get( section, option ) | ||
| return "" | ||
| @contextlib.contextmanager | ||
@@ -504,3 +537,163 @@ def lock_context_manager(): | ||
| def enable_github_issue_tracker(username): | ||
| def add_star(index, repository_id): | ||
| return wrap_text( """ | ||
| update%05d: updateRepository(input:{repositoryId:"%s", hasIssuesEnabled:true}) { | ||
| repository { | ||
| nameWithOwner | ||
| } | ||
| } | ||
| """ % ( index, repository_id ) ) | ||
| run_action_on_all_repositories(username, add_star) | ||
| def add_stars_on_github_repositories(username): | ||
| def add_star(index, repository_id): | ||
| return wrap_text( """ | ||
| update%05d: addStar(input:{starrableId:"%s"}) { | ||
| clientMutationId | ||
| starrable { | ||
| viewerHasStarred | ||
| } | ||
| } | ||
| """ % ( index, repository_id ) ) | ||
| run_action_on_all_repositories(username, add_star) | ||
| def run_action_on_all_repositories(username, action): | ||
| """ We can only update up to 100 repositories at a time | ||
| otherwise we get 502 bad gateway error from GitHub """ | ||
| queryvariables = { | ||
| "user": username, | ||
| "lastItem": None, | ||
| "items": 100, | ||
| } | ||
| while True: | ||
| repositories = get_all_user_repositories(queryvariables) | ||
| # log('repositories', repositories) | ||
| _enable_github_issue_tracker(repositories, action) | ||
| if not queryvariables['hasNextPage']: break | ||
| time.sleep(3) | ||
| def _enable_github_issue_tracker(repositories, action): | ||
| graphqlquery = "" | ||
| for index, repository in enumerate(repositories, start=1): | ||
| repository_id = repository[1] | ||
| graphqlquery += action(index, repository_id) + "\n" | ||
| graphqlresults = run_graphql_query( headers, wrap_text( """ | ||
| mutation UpdateUserRepositories { | ||
| %s | ||
| } | ||
| """ % graphqlquery ) | ||
| ) | ||
| log('graphqlresults', graphqlresults) | ||
| def get_all_user_repositories(queryvariables): | ||
| repositories_found = [] | ||
| graphqlquery = wrap_text( """ | ||
| query ListUserRepositories($user: String!, $items: Int!, $lastItem: String) { | ||
| repositoryOwner(login: $user) { | ||
| repositories(first: $items, after: $lastItem, orderBy: {field: STARGAZERS, direction: DESC}, ownerAffiliations: [OWNER]) { | ||
| pageInfo { | ||
| hasNextPage | ||
| endCursor | ||
| } | ||
| nodes { | ||
| name | ||
| id | ||
| isArchived | ||
| } | ||
| } | ||
| } | ||
| } | ||
| """ ) | ||
| graphqlresults = run_graphql_query( headers, graphqlquery, queryvariables ) | ||
| pageInfo = graphqlresults["data"]["repositoryOwner"]["repositories"]["pageInfo"] | ||
| nodes = graphqlresults["data"]["repositoryOwner"]["repositories"]["nodes"] | ||
| queryvariables['lastItem'] = pageInfo["endCursor"] | ||
| queryvariables['hasNextPage'] = pageInfo["hasNextPage"] | ||
| repositories_found.extend( (item['name'], item['id']) for item in nodes if not item['isArchived'] ) | ||
| # log(f"items {nodes} pageInfo {pageInfo}") | ||
| log(f"items {len(repositories_found)} pageInfo {pageInfo}") | ||
| return repositories_found | ||
| github_ratelimit_graphql = wrap_text( """ | ||
| rateLimit { | ||
| limit | ||
| cost | ||
| remaining | ||
| resetAt | ||
| } | ||
| viewer { | ||
| login | ||
| } | ||
| """ ) | ||
| def log_ratelimit(headers): | ||
| graphqlresults = run_graphql_query( headers, f"{{{github_ratelimit_graphql}}}" ) | ||
| resultdata = graphqlresults["data"] | ||
| log( | ||
| f"{resultdata['viewer']['login']}, " | ||
| f"limit {resultdata['rateLimit']['remaining']}, " | ||
| f"cost {resultdata['rateLimit']['cost']}, " | ||
| f"{resultdata['rateLimit']['remaining']}, " | ||
| f"{resultdata['rateLimit']['resetAt']}, " | ||
| ) | ||
| # A simple function to use requests.post to make the API call. Note the json= section. | ||
| # https://developer.github.com/v4/explorer/ | ||
| def run_graphql_query(headers, graphqlquery, queryvariables={}, graphql_url="https://api.github.com/graphql"): | ||
| """ headers { "Authorization": f"Bearer {github_token}" } """ | ||
| # https://github.com/evandrocoan/GithubRepositoryResearcher | ||
| # https://gist.github.com/gbaman/b3137e18c739e0cf98539bf4ec4366ad | ||
| request = requests.post( graphql_url, json={'query': graphqlquery, 'variables': queryvariables}, headers=headers ) | ||
| fix_line = lambda line: str(line).replace('\\n', '\n') | ||
| if request.status_code == 200: | ||
| result = request.json() | ||
| if "data" not in result or "errors" in result: | ||
| raise Exception( wrap_text( f""" | ||
| There were errors while processing the query! | ||
| graphqlquery: | ||
| {fix_line(graphqlquery)} | ||
| queryvariables: | ||
| {fix_line(queryvariables)} | ||
| errors: | ||
| {json.dumps( result, indent=2, sort_keys=True )} | ||
| """ ) ) | ||
| else: | ||
| raise Exception( wrap_text( f""" | ||
| Query failed to run by returning code of {request.status_code}. | ||
| graphqlquery: | ||
| {fix_line(graphqlquery)} | ||
| queryvariables: | ||
| {fix_line(queryvariables)} | ||
| """ ) ) | ||
| return result | ||
| if __name__ == "__main__": | ||
| main() |
Alert delta unavailable
Currently unable to show alert delta for PyPI packages.
Improved metrics
- Total package byte prevSize
77356
14.41%- Lines of code
616
33.05%