modulegraph

modulegraph

modulegraph determines a dependency graph between Python modules primarily by bytecode analysis for import statements.

modulegraph uses similar methods to modulefinder from the standard library, but uses a more flexible internal representation, has more extensive knowledge of special cases, and is extensible.

CI Status

.. image:: https://github.com/ronaldoussoren/modulegraph/workflows/Lint/badge.svg .. image:: https://github.com/ronaldoussoren/modulegraph/workflows/Test/badge.svg

Project links

• Documentation <https://modulegraph.readthedocs.io/en/latest/>_

• Issue Tracker <https://github.com/ronaldoussoren/modulegraph/issues>_

• Repository <https://github.com/ronaldoussoren/modulegraph/>_

Release history

Version 0.19.6

• Actually introduce Python 3.12 support

  Previous support was tested on an installation that previously had an early alpha installed without cleaning up, resulting in failing to remove all use of the imp module.

0.19.5

• Sigh.. Actually fix 2.7 support

  (code cleanup broke the PR in 0.19.4)

0.19.4

• Fix broken python 2.7 support

  PR by Josua Root

• Initial support for Python 3.12

  The changes to support Python 3.12 are a bit rough, tests pass but I'm not 100% convinced the changes are correct.

0.19.3

• Fix incompatibility with Python 3.11

0.19.2

• Fix project links in PyPI sidebar

0.19.1

• Explictly mention python 3.10 support in package metadata

0.19

• Fix incorrect path when package __init__ is an extension.

0.18

• Avoid exception when one of the items on "packages" is not a package (modulegraph.find_modules.find_needed_modules)

• #45: Modulegraph.foldReferences() called the wrong method

  Reported by Anthony Foglia.

0.17

• The .pyc format changed a little in Python 3.7

0.16

Features:

• Add LICENSE file to distribution

• Don't rely on pkg_resources to calculate package version

• Replace use of optparse by argparse as the former is deprecated

  Patch by htgoebel

• Attempt to reduce the maximum recursion needed to create the ModuleGraph

• Don't include the file type in the result from zipio.getmode

• Fix mismatched indents/dedents in ModuleGraph debug output

  Patch by codewarrior0

Bug fixes:

• Testsuite now passes on Windows (testd through appveyor)

  This only required changes to remove platform dependencies from the test suite.

0.15

Features:

• Issue #39: Traceback with for a syntax error when compiling async function

  On Python 3.5 some (invalid) async function definitions caused a modulegraph traceback, instead of adding "InvalidSourceModule" nodes to the graph.

• Issue #40: The graph now contains nodes of type "InvalidRelativeImport" for attempts to use relative imports that walk outside of a toplevel package.

• Module modulegraph.find_modules can no longer be used a script, use python -m modulegraph instead.

Bugfixes:

• Issue 38: Dot output broken in Python 3

  Patch by user elnuno on bitbucket.

• Issue 36: Make sure test suite works on systems other than macOS

  Patch by Hartmut Goebel

• Add support for "async def" to the AST scanner, needed to properly recognize imports in async function definitions.

0.14

Bugfixes:

• #33: Error scanning bytecode on python 3.4 or later

  The code using dis.get_instructions to scan the bytecode on Python 3.4 or later didn't work properly causing problems when trying to scan bytecode.

0.13

• Various documentation fixes by Thomas Kluyver.

• Fix incompatibility with recent versions of setuptools

  See also issue #206 in py2apps tracker <https://github.com/ronaldoussoren/py2app/issues/206/py2app-doesnt-work-with-release-433-of>_ for more information.

• Python 3: Ignore BOM at start of input files when compiling them.

  This matches the behavior of CPython, and avoids hard to diagnose problems. See also issue #178 in the py2app tracker <https://github.com/ronaldoussoren/py2app/issues/178/python-3-syntaxerror-invalid-character-in>_

• Python 3.6 introduced a new bytecode format (wordcode), adjust the bytecode scanner for that.

0.12.1

• Issue #25: Complex python files could cause an "maximum recursion depth exceeded" exception due to using stack-based recursion to walk the module AST.

0.12

• Added 'modulegraph.modulegraph.InvalidSourceModule'. This graph node is used for Python source modules that cannot be compiled (for example because they contain syntax errors).

  This is primarily useful for being able to create a graph for packages that have python 2.x or python 3.x compatibility in separate modules that contain code that isn't valid in the "other" python version.

• Added 'modulegraph.modulegraph.InvalidCompiledModule'. This graph node is used for Python bytecode modules that cannot be loaded.

• Added 'modulegraph.modulegraph.NamespacePackage'.

  Patch by bitbucket user htgoebel.

• No longer add a MissingModule node to the graph for 'collections.defaultdict' when using 'from collections import defaultdict' ('collections.defaultdict' is an attribute of 'collections', not a submodule).

• Fixed typo in ModuleGraph.getReferences()

• Added ModuleGraph.getReferers(tonode). This methods yields the nodes that are referencing tonode (the reverse of getReferences)

• The graph will no longer contain MissingModule nodes when using 'from ... import name' to import a global variable in a python module.

  There will still be MissingModule nodes for global variables in C extentions, and for 'from missing import name' when 'missing' is itself a MissingModule.

• Issue #18: Don't assume that a PEP 302 loader object has a path attribute. That attribute is not documented and is not always present.

0.11.2

0.11.1

• Issue #145: Don't exclude the platform specific 'path' modules (like ntpath)

0.11

This is a feature release

Features ........

• Hardcode knowlegde about the compatibility aliases in the email module (for python 2.5 upto 3.0).

  This makes it possible to remove a heavy-handed recipe from py2app.

• Added modegraph.zipio.getmode to fetch the Unix file mode for a file.

• Added some handy methods to modulegraph.modulegraph.ModuleGraph.

0.10.5

This is a bugfix release

• Don't look at the file extension to determine the file type in modulegraph.find_modules.parse_mf_results, but use the class of the item.

• Issue #13: Improved handing of bad relative imports ("from .foo import bar"), these tended to raise confusing errors and are now handled like any other failed import.

0.10.4

This is a bugfix release

• There were no 'classifiers' in the package metadata due to a bug in setup.py.

0.10.3

This is a bugfix release

Bugfixes ........

• modulegraph.find.modules.parse_mf_results failed when the main script of a py2app module didn't have a file name ending in '.py'.

0.10.2

This is a bugfix release

Bugfixes ........

• Issue #12: modulegraph would sometimes find the wrong package init module due to using the wrong search method. One easy way to reproduce the problem was to have a toplevel module named init.

  Reported by Kentzo.

0.10.1

This is a bugfix release

Bugfixes ........

• Issue #11: creating xrefs and dotty graphs from modulegraphs (the --xref and --graph options of py2app) didn't work with python 3 due to use of APIs that aren't available in that version of python.

  Reported by Andrew Barnert.

0.10

This is a minor feature release

Features ........

• modulegraph.find_modules.find_needed_modules claimed to automaticly include subpackages for the "packages" argument as well, but that code didn't work at all.

• Issue #9: The modulegraph script is deprecated, use "python -mmodulegraph" instead.

• Issue #10: Ensure that the result of "zipio.open" can be used in a with statement (that is, with zipio.open(...) as fp.

• No longer use "2to3" to support Python 3.

  Because of this modulegraph now supports Python 2.6 and later.

• Slightly improved HTML output, which makes it easier to manipulate the generated HTML using JavaScript.

  Patch by anatoly techtonik.

• Ensure modulegraph works with changes introduced after Python 3.3b1.

• Implement support for PEP 420 ("Implicit namespace packages") in Python 3.3.

• modulegraph.util.imp_walk is deprecated and will be removed in the next release of this package.

Bugfixes ........

• The module graph was incomplete, and generated incorrect warnings along the way, when a subpackage contained import statements for submodules.

  An example of this is sqlalchemy.util, the __init__.py file for this package contains imports of modules in that modules using the classic relative import syntax (that is import compat to import sqlalchemy.util.compat). Until this release modulegraph searched the wrong path to locate these modules (and hence failed to find them).

0.9.2

This is a bugfix release

Bugfixes ........

• The 'packages' option to modulegraph.find_modules.find_modules ignored the search path argument but always used the default search path.

• The 'imp_find_modules' function in modulegraph.util has an argument 'path', this was a string in previous release and can now also be a sequence.

• Don't crash when a module on the 'includes' list doesn't exist, but warn just like for missing 'packages' (modulegraph.find_modules.find_modules)

0.9.1

This is a bugfix release

Bug fixes .........

• Fixed the name of nodes imports in packages where the first element of a dotted name can be found but the rest cannot. This used to create a MissingModule node for the dotted name in the global namespace instead of relative to the package.

  That is, given a package "pkg" with submodule "sub" if the "init.py" of "pkg" contains "import sub.nomod" we now create a MissingModule node for "pkg.sub.nomod" instead of "sub.nomod".

  This fixes an issue with including the crcmod package in application bundles, first reported on the pythonmac-sig mailinglist by Brendan Simon.

0.9

This is a minor feature release

Features:

• Documentation is now generated using sphinx <http://pypi.python.org/pypi/sphinx>_ and can be viewed at http://packages.python.org/modulegraph.

  The documention is very rough at this moment and in need of reorganisation and language cleanup. I've basiclly writting the current version by reading the code and documenting what it does, the order in which classes and methods are document is therefore not necessarily the most useful.

• The repository has moved to bitbucket

• Renamed modulegraph.modulegraph.AddPackagePath to addPackagePath, likewise ReplacePackage is now replacePackage. The old name is still available, but is deprecated and will be removed before the 1.0 release.

• modulegraph.modulegraph contains two node types that are unused and have unclear semantics: FlatPackage and ArchiveModule. These node types are deprecated and will be removed before 1.0 is released.

• Added a simple commandline tool (modulegraph) that will print information about the dependency graph of a script.

• Added a module (zipio) for dealing with paths that may refer to entries inside zipfiles (such as source paths referring to modules in zipped eggfiles).

  With this addition modulegraph.modulegraph.os_listdir is deprecated and it will be removed before the 1.0 release.

Bug fixes:

• The __cmp__ method of a Node no longer causes an exception when the compared-to object is not a Node. Patch by Ivan Kozik.

• Issue #1: The initialiser for modulegraph.ModuleGraph caused an exception when an entry on the path (sys.path) doesn't actually exist.

  Fix by "skurylo", testcase by Ronald.

• The code no longer worked with python 2.5, this release fixes that.

• Due to the switch to mercurial setuptools will no longer include all required files. Fixed by adding a MANIFEST.in file

• The method for printing a .dot representation of a ModuleGraph works again.

0.8.1

This is a minor feature release

Features:

• from __future__ import absolute_import is now supported

• Relative imports (from . import module) are now supported

• Add support for namespace packages when those are installed using option --single-version-externally-managed (part of setuptools/distribute)

0.8

This is a minor feature release

Features:

• Initial support for Python 3.x

• It is now possible to run the test suite using python setup.py test.

  (The actual test suite is still fairly minimal though)