You're Invited:Meet the Socket Team at RSAC and BSidesSF 2026, March 23–26.RSVP →
Socket
Book a DemoSign in
Socket
Book a DemoSign in

crossenv

Package Overview
Dependencies
Maintainers
1
Versions
19
Alerts
File Explorer

Advanced tools

License
Socket logo

Install Socket

Detect and block malicious and high-risk dependencies

Install

crossenv - pypi Package Compare versions

Comparing version
1.1.3
 to
1.1.4
+186
-183
crossenv.egg-info/PKG-INFO

@@ -1,4 +0,4 @@

Metadata-Version: 1.2
Metadata-Version: 2.1
Name: crossenv
Version: 1.1.3
Version: 1.1.4
Summary: A cross-compiling tool for Python extension modules

@@ -9,183 +9,2 @@ Home-page: https://github.com/benfogle/crossenv

License: UNKNOWN
Description: Virtual Environments for Cross-Compiling Python Extension Modules
=============================================================================
|build status| |test status| |coverage status|
Documentation is available online at https://crossenv.readthedocs.io and in the
``docs`` directory.
Porting a Python app to an embedded device can be complicated. Once you have
Python built for your system, you may find yourself needing to include many
third-party libraries. Pure-Python libraries usually just work, but many
popular libraries rely on compiled C code, which can be challenging to build.
This package is a tool for cross-compiling extension modules. It creates a
special virtual environment such that ``pip`` or ``setup.py`` will cross
compile packages for you, usually with no further work on your part.
It can be used to:
* Build binary wheels, for installation on target.
* Install packages to a directory for upload or inclusion in a firmware image.
**Note**: While this tool can cross-compile *most* Python packages, it can't
solve all the problems of cross-compiling. In some cases manual intervention
may still be necessary.
This tool requires Python 3.5 or higher (host and build). Significant work has
gone into cross-compiling Python in newer versions, and many of the techniques
needed to do the cross compilation properly are not available on older
releases.
This tool currently only supports Linux build machines.
Vocabulary
-----------------------------------------------------------------------------
+---------------+------------------------------------------------------------+
| Host | The machine you are building **for**. (Android, iOS, other |
| | embedded systems.) |
+---------------+------------------------------------------------------------+
| Build | The machine you are building **on**. (Probably your |
| | desktop.) |
+---------------+------------------------------------------------------------+
| Host-python | The compiled Python binary and libraries that run on Host |
+---------------+------------------------------------------------------------+
| Build-python | The compiled Python binary and libraries that run on |
| | Build. |
+---------------+------------------------------------------------------------+
| Cross-python | Build-python, configured specially to build packages that |
| | can be run with Host-python. This tool creates |
| | Cross-python. |
+---------------+------------------------------------------------------------+
How it works
-----------------------------------------------------------------------------
Cross-python is set up carefully so that it reports all system information
exactly as Host-python would. When done correctly, a side effect of this is
that ``distutils`` and ``setuptools`` will cross-compile when building
packages. All of the normal packaging machinery still works correctly, so
dependencies, ABI tags, and so forth all work as expected.
Requirements
-----------------------------------------------------------------------------
You will need:
1. A version of Python (3.5 or later) that runs on Build. (Build-python.)
2. A version of Python that will run on Host. (Host-python.) This must be the
*same version* as Build-python.
3. The cross-compiling toolchain used to make Host-python. Make sure you set
PATH correctly to use it.
4. Any libraries your modules depend on, cross-compiled and installed
somewhere Cross-python can get to them. For example, the ``cryptography``
package depends on OpenSSL and libffi.
Installation
-----------------------------------------------------------------------------
Crossenv can be installed using pip::
$ pip install crossenv
Usage
-----------------------------------------------------------------------------
To create the virtual environment::
$ /path/to/build/python3 -m crossenv /path/to/host/python3 venv
This creates a folder named ``venv`` that contains two subordinate virtual
environments: one for Build-python, and one for Cross-python. When activated,
``python`` (or its alias ``cross-python``) can be used for cross compiling. If
needed, packages can be installed on Build (e.g., a package requires Cython to
install) with ``build-python``. There are equivalent ``pip``, ``cross-pip``,
and ``build-pip`` commands.
The cross-compiler to use, along with any extra flags needed, are taken from
information recorded when Host-python was compiled. To activate the
environment::
$ . venv/bin/activate
You can now see that ``python`` seems to think it's running on Host::
(cross) $ python -m sysconfig
...
Now you can cross compile! To install a package to
``venv/cross/lib/python3.6/site-packages``, you can use pip directly::
(cross) $ pip -v install numpy
...
You can use ``setup.py`` to build wheels::
(cross) $ pip install wheel
(cross) $ pip download numpy
Collecting numpy
Using cached numpy-1.14.1.zip
Saved ./numpy-1.14.1.zip
Successfully downloaded numpy
(cross) $ unzip -q ./numpy-1.14.1.zip
(cross) $ cd numpy-1.14.1
(cross) $ python setup.py bdist_wheel
...
When you need packages like Cython or cffi installed to build another module,
sometimes satisfying dependencies can get tricky. If you simply ``pip install``
the module, you may find it builds Cython as a prerequisite *for the host* and
then tries to run it on the build machine. This will fail, of course, but if we
install the necessary package for ``build-python``, then ``pip`` will pick up
the correct version during install.
For example, to build bcrypt and python-cryptography::
(cross) $ build-pip install cffi
(cross) $ pip install bcrypt
(cross) $ pip install cryptography
Some packages do explicit checks for existence of a package. For instance, a
package may do a check for Cython (other than simply trying to import it)
before proceeding with installation. If a package is installed with
``build-pip``, etc., then setuptools in ``cross-python`` does not recognize it
as installed. (Note that you can still import it even if setuptools can't see
it, so the naive check of ``import Cython`` will work fine so long as you did
``build-pip install Cython`` earlier.) This is by design. To selectively expose
build-python packages so that setuptools will count them as installed, you can
use the ``cross-expose`` script installed in the virtual environment.
Known Limitations
-----------------------------------------------------------------------------
* Upgrading ``cross-pip`` and ``build-pip`` must be done carefully, and it's
best not to do so unless you need to. If you need to: upgrade ``cross-pip``
first, then ``build-pip``.
* When installing scripts, the shebang (``#!``) line is wrong. This will
need to be fixed up before using on Host.
* Any dependant libraries used during the build, such as OpenSSL, are *not*
packaged in the wheel or install directory. You will need to ensure that
these libraries are installed on Host and can be used. This is the normal
Python behavior.
* Any setup-time requirement listed in ``setup.py`` under ``setup_requires``
will be installed in Cross-python's virtual environment, not Build-python.
This will mostly work anyway if they are pure-Python, but for packages
with extension modules (Cython, etc.), you will need to install them into
Build-python's environment first. It's often a good idea to do a
``build-pip install <whatever>`` prior to ``pip install <whatever>``.
.. |build status| image:: https://dev.azure.com/benfogle/crossenv/_apis/build/status/benfogle.crossenv?branchName=master
.. |test status| image:: https://img.shields.io/azure-devops/tests/benfogle/crossenv/1/master
.. |coverage status| image:: https://img.shields.io/azure-devops/coverage/benfogle/crossenv/1/master
Platform: UNKNOWN

@@ -198,1 +17,185 @@ Classifier: Development Status :: 3 - Alpha

Requires-Python: >=3.4
License-File: LICENSE.txt
Virtual Environments for Cross-Compiling Python Extension Modules
=============================================================================
|build status| |test status| |coverage status|
Documentation is available online at https://crossenv.readthedocs.io and in the
``docs`` directory.
Porting a Python app to an embedded device can be complicated. Once you have
Python built for your system, you may find yourself needing to include many
third-party libraries. Pure-Python libraries usually just work, but many
popular libraries rely on compiled C code, which can be challenging to build.
This package is a tool for cross-compiling extension modules. It creates a
special virtual environment such that ``pip`` or ``setup.py`` will cross
compile packages for you, usually with no further work on your part.
It can be used to:
* Build binary wheels, for installation on target.
* Install packages to a directory for upload or inclusion in a firmware image.
**Note**: While this tool can cross-compile *most* Python packages, it can't
solve all the problems of cross-compiling. In some cases manual intervention
may still be necessary.
This tool requires Python 3.5 or higher (host and build). Significant work has
gone into cross-compiling Python in newer versions, and many of the techniques
needed to do the cross compilation properly are not available on older
releases.
This tool currently only supports Linux build machines.
Vocabulary
-----------------------------------------------------------------------------
+---------------+------------------------------------------------------------+
| Host | The machine you are building **for**. (Android, iOS, other |
| | embedded systems.) |
+---------------+------------------------------------------------------------+
| Build | The machine you are building **on**. (Probably your |
| | desktop.) |
+---------------+------------------------------------------------------------+
| Host-python | The compiled Python binary and libraries that run on Host |
+---------------+------------------------------------------------------------+
| Build-python | The compiled Python binary and libraries that run on |
| | Build. |
+---------------+------------------------------------------------------------+
| Cross-python | Build-python, configured specially to build packages that |
| | can be run with Host-python. This tool creates |
| | Cross-python. |
+---------------+------------------------------------------------------------+
How it works
-----------------------------------------------------------------------------
Cross-python is set up carefully so that it reports all system information
exactly as Host-python would. When done correctly, a side effect of this is
that ``distutils`` and ``setuptools`` will cross-compile when building
packages. All of the normal packaging machinery still works correctly, so
dependencies, ABI tags, and so forth all work as expected.
Requirements
-----------------------------------------------------------------------------
You will need:
1. A version of Python (3.5 or later) that runs on Build. (Build-python.)
2. A version of Python that will run on Host. (Host-python.) This must be the
*same version* as Build-python.
3. The cross-compiling toolchain used to make Host-python. Make sure you set
PATH correctly to use it.
4. Any libraries your modules depend on, cross-compiled and installed
somewhere Cross-python can get to them. For example, the ``cryptography``
package depends on OpenSSL and libffi.
Installation
-----------------------------------------------------------------------------
Crossenv can be installed using pip::
$ pip install crossenv
Usage
-----------------------------------------------------------------------------
To create the virtual environment::
$ /path/to/build/python3 -m crossenv /path/to/host/python3 venv
This creates a folder named ``venv`` that contains two subordinate virtual
environments: one for Build-python, and one for Cross-python. When activated,
``python`` (or its alias ``cross-python``) can be used for cross compiling. If
needed, packages can be installed on Build (e.g., a package requires Cython to
install) with ``build-python``. There are equivalent ``pip``, ``cross-pip``,
and ``build-pip`` commands.
The cross-compiler to use, along with any extra flags needed, are taken from
information recorded when Host-python was compiled. To activate the
environment::
$ . venv/bin/activate
You can now see that ``python`` seems to think it's running on Host::
(cross) $ python -m sysconfig
...
Now you can cross compile! To install a package to
``venv/cross/lib/python3.6/site-packages``, you can use pip directly::
(cross) $ pip -v install numpy
...
You can use ``setup.py`` to build wheels::
(cross) $ pip install wheel
(cross) $ pip download numpy
Collecting numpy
Using cached numpy-1.14.1.zip
Saved ./numpy-1.14.1.zip
Successfully downloaded numpy
(cross) $ unzip -q ./numpy-1.14.1.zip
(cross) $ cd numpy-1.14.1
(cross) $ python setup.py bdist_wheel
...
When you need packages like Cython or cffi installed to build another module,
sometimes satisfying dependencies can get tricky. If you simply ``pip install``
the module, you may find it builds Cython as a prerequisite *for the host* and
then tries to run it on the build machine. This will fail, of course, but if we
install the necessary package for ``build-python``, then ``pip`` will pick up
the correct version during install.
For example, to build bcrypt and python-cryptography::
(cross) $ build-pip install cffi
(cross) $ pip install bcrypt
(cross) $ pip install cryptography
Some packages do explicit checks for existence of a package. For instance, a
package may do a check for Cython (other than simply trying to import it)
before proceeding with installation. If a package is installed with
``build-pip``, etc., then setuptools in ``cross-python`` does not recognize it
as installed. (Note that you can still import it even if setuptools can't see
it, so the naive check of ``import Cython`` will work fine so long as you did
``build-pip install Cython`` earlier.) This is by design. To selectively expose
build-python packages so that setuptools will count them as installed, you can
use the ``cross-expose`` script installed in the virtual environment.
Known Limitations
-----------------------------------------------------------------------------
* Upgrading ``cross-pip`` and ``build-pip`` must be done carefully, and it's
best not to do so unless you need to. If you need to: upgrade ``cross-pip``
first, then ``build-pip``.
* When installing scripts, the shebang (``#!``) line is wrong. This will
need to be fixed up before using on Host.
* Any dependant libraries used during the build, such as OpenSSL, are *not*
packaged in the wheel or install directory. You will need to ensure that
these libraries are installed on Host and can be used. This is the normal
Python behavior.
* Any setup-time requirement listed in ``setup.py`` under ``setup_requires``
will be installed in Cross-python's virtual environment, not Build-python.
This will mostly work anyway if they are pure-Python, but for packages
with extension modules (Cython, etc.), you will need to install them into
Build-python's environment first. It's often a good idea to do a
``build-pip install <whatever>`` prior to ``pip install <whatever>``.
.. |build status| image:: https://dev.azure.com/benfogle/crossenv/_apis/build/status/benfogle.crossenv?branchName=master
.. |test status| image:: https://img.shields.io/azure-devops/tests/benfogle/crossenv/1/master
.. |coverage status| image:: https://img.shields.io/azure-devops/coverage/benfogle/crossenv/1/master
+17
-8
crossenv/__init__.py

@@ -22,3 +22,3 @@ import venv

__version__ = '1.1.3'
__version__ = '1.1.4'

@@ -107,2 +107,4 @@ logger = logging.getLogger(__name__)

from PyPI.
:param host_machine: Host machine override seen by cross-python at
runtime. Default is guessed from host-python.
"""

@@ -124,3 +126,4 @@ def __init__(self, *,

host_sysconfigdata_file=None,
manylinux_tags=()):
manylinux_tags=(),
host_machine=None):
self.host_sysroot = host_sysroot

@@ -153,2 +156,3 @@ self.host_cc = None

self.manylinux_tags = manylinux_tags
self.host_machine = host_machine

@@ -480,8 +484,9 @@ self.find_host_python(host_python)

if len(host_info) > 1 and host_info[-1] == "powerpc64le":
# Test that this is still a special case when we can.
# On uname.machine=ppc64le, _PYTHON_HOST_PLATFORM is linux-powerpc64le
self.host_machine = "ppc64le"
else:
self.host_machine = self.host_gnu_type.split('-')[0]
if self.host_machine is None:
if len(host_info) > 1 and host_info[-1] == "powerpc64le":
# Test that this is still a special case when we can.
# On uname.machine=ppc64le, _PYTHON_HOST_PLATFORM is linux-powerpc64le
self.host_machine = "ppc64le"
else:
self.host_machine = self.host_gnu_type.split('-')[0]

@@ -995,2 +1000,5 @@ self.host_release = ''

times.""")
parser.add_argument('--machine', action='store',
help="""Override the value of os.uname().machine if cross-python is
unable to guess correctly.""")
parser.add_argument('-v', '--verbose', action='count', default=0,

@@ -1039,2 +1047,3 @@ help="""Verbose mode. May be specified multiple times to increase

manylinux_tags=args.manylinux,
host_machine=args.machine,
)

@@ -1041,0 +1050,0 @@ for env_dir in args.ENV_DIR:

+21
-1
crossenv/scripts/site.py.tmpl

@@ -58,2 +58,8 @@ # Import only what we absolutely need before the path fixup

def make_loader(original, patch):
if hasattr(original, 'exec_module'):
return CrossenvPatchLoader(original, patch)
else:
return CrossenvPatchLegacyLoader(original, patch)
class CrossenvPatchLoader(importlib.abc.Loader):

@@ -77,2 +83,16 @@ def __init__(self, original, patch):

class CrossenvPatchLegacyLoader(importlib.abc.Loader):
def __init__(self, original, patch):
self.original = original
self.patch = patch
def load_module(self, fullname):
module = self.original.load_module(fullname)
if not hasattr(module, '__patched__'):
_patch_module(module, self.patch)
return module
def __getattr__(self, name):
return getattr(self.original, name)
class CrossenvFinder(importlib.abc.MetaPathFinder):

@@ -135,3 +155,3 @@ """Mucks with import machinery in two ways:

patch = self.PATCHES[fullname]
spec.loader = CrossenvPatchLoader(spec.loader, patch)
spec.loader = make_loader(spec.loader, patch)
return spec

@@ -138,0 +158,0 @@ 

+186
-183
PKG-INFO

@@ -1,4 +0,4 @@

Metadata-Version: 1.2
Metadata-Version: 2.1
Name: crossenv
Version: 1.1.3
Version: 1.1.4
Summary: A cross-compiling tool for Python extension modules

@@ -9,183 +9,2 @@ Home-page: https://github.com/benfogle/crossenv

License: UNKNOWN
Description: Virtual Environments for Cross-Compiling Python Extension Modules
=============================================================================
|build status| |test status| |coverage status|
Documentation is available online at https://crossenv.readthedocs.io and in the
``docs`` directory.
Porting a Python app to an embedded device can be complicated. Once you have
Python built for your system, you may find yourself needing to include many
third-party libraries. Pure-Python libraries usually just work, but many
popular libraries rely on compiled C code, which can be challenging to build.
This package is a tool for cross-compiling extension modules. It creates a
special virtual environment such that ``pip`` or ``setup.py`` will cross
compile packages for you, usually with no further work on your part.
It can be used to:
* Build binary wheels, for installation on target.
* Install packages to a directory for upload or inclusion in a firmware image.
**Note**: While this tool can cross-compile *most* Python packages, it can't
solve all the problems of cross-compiling. In some cases manual intervention
may still be necessary.
This tool requires Python 3.5 or higher (host and build). Significant work has
gone into cross-compiling Python in newer versions, and many of the techniques
needed to do the cross compilation properly are not available on older
releases.
This tool currently only supports Linux build machines.
Vocabulary
-----------------------------------------------------------------------------
+---------------+------------------------------------------------------------+
| Host | The machine you are building **for**. (Android, iOS, other |
| | embedded systems.) |
+---------------+------------------------------------------------------------+
| Build | The machine you are building **on**. (Probably your |
| | desktop.) |
+---------------+------------------------------------------------------------+
| Host-python | The compiled Python binary and libraries that run on Host |
+---------------+------------------------------------------------------------+
| Build-python | The compiled Python binary and libraries that run on |
| | Build. |
+---------------+------------------------------------------------------------+
| Cross-python | Build-python, configured specially to build packages that |
| | can be run with Host-python. This tool creates |
| | Cross-python. |
+---------------+------------------------------------------------------------+
How it works
-----------------------------------------------------------------------------
Cross-python is set up carefully so that it reports all system information
exactly as Host-python would. When done correctly, a side effect of this is
that ``distutils`` and ``setuptools`` will cross-compile when building
packages. All of the normal packaging machinery still works correctly, so
dependencies, ABI tags, and so forth all work as expected.
Requirements
-----------------------------------------------------------------------------
You will need:
1. A version of Python (3.5 or later) that runs on Build. (Build-python.)
2. A version of Python that will run on Host. (Host-python.) This must be the
*same version* as Build-python.
3. The cross-compiling toolchain used to make Host-python. Make sure you set
PATH correctly to use it.
4. Any libraries your modules depend on, cross-compiled and installed
somewhere Cross-python can get to them. For example, the ``cryptography``
package depends on OpenSSL and libffi.
Installation
-----------------------------------------------------------------------------
Crossenv can be installed using pip::
$ pip install crossenv
Usage
-----------------------------------------------------------------------------
To create the virtual environment::
$ /path/to/build/python3 -m crossenv /path/to/host/python3 venv
This creates a folder named ``venv`` that contains two subordinate virtual
environments: one for Build-python, and one for Cross-python. When activated,
``python`` (or its alias ``cross-python``) can be used for cross compiling. If
needed, packages can be installed on Build (e.g., a package requires Cython to
install) with ``build-python``. There are equivalent ``pip``, ``cross-pip``,
and ``build-pip`` commands.
The cross-compiler to use, along with any extra flags needed, are taken from
information recorded when Host-python was compiled. To activate the
environment::
$ . venv/bin/activate
You can now see that ``python`` seems to think it's running on Host::
(cross) $ python -m sysconfig
...
Now you can cross compile! To install a package to
``venv/cross/lib/python3.6/site-packages``, you can use pip directly::
(cross) $ pip -v install numpy
...
You can use ``setup.py`` to build wheels::
(cross) $ pip install wheel
(cross) $ pip download numpy
Collecting numpy
Using cached numpy-1.14.1.zip
Saved ./numpy-1.14.1.zip
Successfully downloaded numpy
(cross) $ unzip -q ./numpy-1.14.1.zip
(cross) $ cd numpy-1.14.1
(cross) $ python setup.py bdist_wheel
...
When you need packages like Cython or cffi installed to build another module,
sometimes satisfying dependencies can get tricky. If you simply ``pip install``
the module, you may find it builds Cython as a prerequisite *for the host* and
then tries to run it on the build machine. This will fail, of course, but if we
install the necessary package for ``build-python``, then ``pip`` will pick up
the correct version during install.
For example, to build bcrypt and python-cryptography::
(cross) $ build-pip install cffi
(cross) $ pip install bcrypt
(cross) $ pip install cryptography
Some packages do explicit checks for existence of a package. For instance, a
package may do a check for Cython (other than simply trying to import it)
before proceeding with installation. If a package is installed with
``build-pip``, etc., then setuptools in ``cross-python`` does not recognize it
as installed. (Note that you can still import it even if setuptools can't see
it, so the naive check of ``import Cython`` will work fine so long as you did
``build-pip install Cython`` earlier.) This is by design. To selectively expose
build-python packages so that setuptools will count them as installed, you can
use the ``cross-expose`` script installed in the virtual environment.
Known Limitations
-----------------------------------------------------------------------------
* Upgrading ``cross-pip`` and ``build-pip`` must be done carefully, and it's
best not to do so unless you need to. If you need to: upgrade ``cross-pip``
first, then ``build-pip``.
* When installing scripts, the shebang (``#!``) line is wrong. This will
need to be fixed up before using on Host.
* Any dependant libraries used during the build, such as OpenSSL, are *not*
packaged in the wheel or install directory. You will need to ensure that
these libraries are installed on Host and can be used. This is the normal
Python behavior.
* Any setup-time requirement listed in ``setup.py`` under ``setup_requires``
will be installed in Cross-python's virtual environment, not Build-python.
This will mostly work anyway if they are pure-Python, but for packages
with extension modules (Cython, etc.), you will need to install them into
Build-python's environment first. It's often a good idea to do a
``build-pip install <whatever>`` prior to ``pip install <whatever>``.
.. |build status| image:: https://dev.azure.com/benfogle/crossenv/_apis/build/status/benfogle.crossenv?branchName=master
.. |test status| image:: https://img.shields.io/azure-devops/tests/benfogle/crossenv/1/master
.. |coverage status| image:: https://img.shields.io/azure-devops/coverage/benfogle/crossenv/1/master
Platform: UNKNOWN

@@ -198,1 +17,185 @@ Classifier: Development Status :: 3 - Alpha

Requires-Python: >=3.4
License-File: LICENSE.txt
Virtual Environments for Cross-Compiling Python Extension Modules
=============================================================================
|build status| |test status| |coverage status|
Documentation is available online at https://crossenv.readthedocs.io and in the
``docs`` directory.
Porting a Python app to an embedded device can be complicated. Once you have
Python built for your system, you may find yourself needing to include many
third-party libraries. Pure-Python libraries usually just work, but many
popular libraries rely on compiled C code, which can be challenging to build.
This package is a tool for cross-compiling extension modules. It creates a
special virtual environment such that ``pip`` or ``setup.py`` will cross
compile packages for you, usually with no further work on your part.
It can be used to:
* Build binary wheels, for installation on target.
* Install packages to a directory for upload or inclusion in a firmware image.
**Note**: While this tool can cross-compile *most* Python packages, it can't
solve all the problems of cross-compiling. In some cases manual intervention
may still be necessary.
This tool requires Python 3.5 or higher (host and build). Significant work has
gone into cross-compiling Python in newer versions, and many of the techniques
needed to do the cross compilation properly are not available on older
releases.
This tool currently only supports Linux build machines.
Vocabulary
-----------------------------------------------------------------------------
+---------------+------------------------------------------------------------+
| Host | The machine you are building **for**. (Android, iOS, other |
| | embedded systems.) |
+---------------+------------------------------------------------------------+
| Build | The machine you are building **on**. (Probably your |
| | desktop.) |
+---------------+------------------------------------------------------------+
| Host-python | The compiled Python binary and libraries that run on Host |
+---------------+------------------------------------------------------------+
| Build-python | The compiled Python binary and libraries that run on |
| | Build. |
+---------------+------------------------------------------------------------+
| Cross-python | Build-python, configured specially to build packages that |
| | can be run with Host-python. This tool creates |
| | Cross-python. |
+---------------+------------------------------------------------------------+
How it works
-----------------------------------------------------------------------------
Cross-python is set up carefully so that it reports all system information
exactly as Host-python would. When done correctly, a side effect of this is
that ``distutils`` and ``setuptools`` will cross-compile when building
packages. All of the normal packaging machinery still works correctly, so
dependencies, ABI tags, and so forth all work as expected.
Requirements
-----------------------------------------------------------------------------
You will need:
1. A version of Python (3.5 or later) that runs on Build. (Build-python.)
2. A version of Python that will run on Host. (Host-python.) This must be the
*same version* as Build-python.
3. The cross-compiling toolchain used to make Host-python. Make sure you set
PATH correctly to use it.
4. Any libraries your modules depend on, cross-compiled and installed
somewhere Cross-python can get to them. For example, the ``cryptography``
package depends on OpenSSL and libffi.
Installation
-----------------------------------------------------------------------------
Crossenv can be installed using pip::
$ pip install crossenv
Usage
-----------------------------------------------------------------------------
To create the virtual environment::
$ /path/to/build/python3 -m crossenv /path/to/host/python3 venv
This creates a folder named ``venv`` that contains two subordinate virtual
environments: one for Build-python, and one for Cross-python. When activated,
``python`` (or its alias ``cross-python``) can be used for cross compiling. If
needed, packages can be installed on Build (e.g., a package requires Cython to
install) with ``build-python``. There are equivalent ``pip``, ``cross-pip``,
and ``build-pip`` commands.
The cross-compiler to use, along with any extra flags needed, are taken from
information recorded when Host-python was compiled. To activate the
environment::
$ . venv/bin/activate
You can now see that ``python`` seems to think it's running on Host::
(cross) $ python -m sysconfig
...
Now you can cross compile! To install a package to
``venv/cross/lib/python3.6/site-packages``, you can use pip directly::
(cross) $ pip -v install numpy
...
You can use ``setup.py`` to build wheels::
(cross) $ pip install wheel
(cross) $ pip download numpy
Collecting numpy
Using cached numpy-1.14.1.zip
Saved ./numpy-1.14.1.zip
Successfully downloaded numpy
(cross) $ unzip -q ./numpy-1.14.1.zip
(cross) $ cd numpy-1.14.1
(cross) $ python setup.py bdist_wheel
...
When you need packages like Cython or cffi installed to build another module,
sometimes satisfying dependencies can get tricky. If you simply ``pip install``
the module, you may find it builds Cython as a prerequisite *for the host* and
then tries to run it on the build machine. This will fail, of course, but if we
install the necessary package for ``build-python``, then ``pip`` will pick up
the correct version during install.
For example, to build bcrypt and python-cryptography::
(cross) $ build-pip install cffi
(cross) $ pip install bcrypt
(cross) $ pip install cryptography
Some packages do explicit checks for existence of a package. For instance, a
package may do a check for Cython (other than simply trying to import it)
before proceeding with installation. If a package is installed with
``build-pip``, etc., then setuptools in ``cross-python`` does not recognize it
as installed. (Note that you can still import it even if setuptools can't see
it, so the naive check of ``import Cython`` will work fine so long as you did
``build-pip install Cython`` earlier.) This is by design. To selectively expose
build-python packages so that setuptools will count them as installed, you can
use the ``cross-expose`` script installed in the virtual environment.
Known Limitations
-----------------------------------------------------------------------------
* Upgrading ``cross-pip`` and ``build-pip`` must be done carefully, and it's
best not to do so unless you need to. If you need to: upgrade ``cross-pip``
first, then ``build-pip``.
* When installing scripts, the shebang (``#!``) line is wrong. This will
need to be fixed up before using on Host.
* Any dependant libraries used during the build, such as OpenSSL, are *not*
packaged in the wheel or install directory. You will need to ensure that
these libraries are installed on Host and can be used. This is the normal
Python behavior.
* Any setup-time requirement listed in ``setup.py`` under ``setup_requires``
will be installed in Cross-python's virtual environment, not Build-python.
This will mostly work anyway if they are pure-Python, but for packages
with extension modules (Cython, etc.), you will need to install them into
Build-python's environment first. It's often a good idea to do a
``build-pip install <whatever>`` prior to ``pip install <whatever>``.
.. |build status| image:: https://dev.azure.com/benfogle/crossenv/_apis/build/status/benfogle.crossenv?branchName=master
.. |test status| image:: https://img.shields.io/azure-devops/tests/benfogle/crossenv/1/master
.. |coverage status| image:: https://img.shields.io/azure-devops/coverage/benfogle/crossenv/1/master