sshpyk

Remote Jupyter Kernels via SSH tunnels ######################################

The design of this pakage is based upon SSH Kernel <https://github.com/bernhard-42/ssh_ipykernel>_ which in turn is based upon remote_ikernel <https://bitbucket.org/tdaff/remote_ikernel>. This implementation shares a common set of command line parameters with SSH Kernel <https://github.com/bernhard-42/ssh_ipykernel>, but it was implemented from scratch to adapt to recent changes to :code:jupyter_client (which broke :code:ssh_ipykernel) and to support Python 3.10. This package adds an :code:ls implementation which allows listing info about the available kernel specifications.

While there are modest additions to SSH Kernel <https://github.com/bernhard-42/ssh_ipykernel>, there are also modest subtractions. There are fewer configuration options for things like the internal name used by Jupyter Client <https://jupyter-client.readthedocs.io/en/stable/#> to refer to the created kernel.

Listing the Jupyter Kernels that are available

---

It can be difficult to know which Jupyter Kernels are available because there are multiple locations where the Kernel Spec files <https://jupyter-client.readthedocs.io/en/latest/kernels.html#kernel-specs>_ can be found. :code:sshpyk has an :code:ls option which lists the kernels that are available (even those which are not SSH Kernel <https://github.com/bernhard-42/ssh_ipykernel>_ or :code:sshpyk kernels::

bash$ bash$ python -m sshpyk.kernel.ls --no-check -a python3 /Users/drs/develop/python/conda/envs/py310/share/jupyter/kernels/python3 python3.8 /usr/local/share/jupyter/kernels/python3.8 python3dbg /Users/drs/Library/Jupyter/kernels/python3dbg ssh__sshhost06test001 /Users/drs/Library/Jupyter/kernels/ssh__sshhost06test001 ssh__sshhost06test002 /Users/drs/Library/Jupyter/kernels/ssh__sshhost06test002 ssh_host06_host06homehost06condaenvspy310 /Users/drs/Library/Jupyter/kernels/ssh_host06_host06homehost06condaenvspy310 bash$

The :code:--no-check (or alternatively :code:-nc) flag indicates that the validity of the kernel spec files should not be checked. The :code:-a (or :code:--all) flag indicates that it should show all kernel specifications rather than just the ones for SSH Kernel <https://github.com/bernhard-42/ssh_ipykernel>_ or :code:sshpyk kernel specification files.

If :code:--no-check is not supplied, part of listing the kernel information will include verify that the Python executable specified in the kernel specification exist on the local and remote systems. This check allows the ouput to be colorized so red text indicates a problem. :code:--local will limit the check to just the local Python executable and :code:--remote will limit the check to only the remote Python executable. These options also list the local or remote Python path instead of the path to the kernel specification directory.

Command line "ls" options

The following options are available for listing the Jupyter kernel specifications:

--help, -h ^^^^^^^^^^

          Show help information and exit.

--all, -a ^^^^^^^^^

          List all kernels that are available rather than just
          `SSH Kernel <https://github.com/bernhard-42/ssh_ipykernel>`_ and sshpyk kernels.

--local, -l ^^^^^^^^^^^

          Only list the information for the local Python executable.

--remote, -r ^^^^^^^^^^^^

          Only list the information for the remote Python executable.
          

--no-check, -nc ^^^^^^^^^^^^^^^

          Do not check for the existence local or remote Python executables. This option
          can be used alone or with other options, e.g. with :code:`-l`. :code:`--no-check`
          avoids colorization to indicate problems so it can be useful for scripting.

Adding a new Kernel for a Remote System

---

:code:sshpyk can also be used to add a specification file for a remote Python Kernel. For a remote kernel to work

• :code:ssypyk must be installed on the local system
• :code:ipykernel must be installed on the remote system

Once these requirements are satisfied, the new kernel can be added like::

bash$ python3 -m sshpyk.kernel.add --host host06 --python /home/host06/conda/envs/py310 --display-name 'host06 kernel'

This will add a Python kernel which will run on :code:host06, and it will be called :code:host06 kernel when it is listed as an option for the user. We can check to see if it is now included by using the :code:ls functionality::

bash$ python3 -m sshpyk.kernel.ls -r -a -nc python3 localhost:python python3.8 localhost:/opt/local/bin/python3.8 python3dbg localhost:/Users/drs/develop/casagui-ic-debugging/iclean_demo_venv/bin/python ssh__sshhost06test001 host06:/home/host06/conda/envs/py310/bin/python ssh__sshhost06test002 host06:/home/host06/conda/envs/py310/bin/python ssh_host06_host06homehost06condaenvspy310 host06:/home/host06/conda/envs/py310/bin/python ssh_host06_host06kernel host06:/home/host06/conda/envs/py310/bin/python bash$

Unlike the example above, here we have asked that the remote Python path be displayed instead of showing the kernel specificaton directory. Since we again asked that all kernels be displayed instead of only the SSH kernels, a Python path is displayed for the non-SSH kernels, but it is the local Python path as indicated by :code:localhost:. Because these three kernels are non-SSH kernels this is the only Python path that is available. However for the SSH kernels, we can see the remote Python path listed. These paths are prefixed with the hostname, here :code:host06:. We can also see the newly added kernel listed as :code:ssh_host06_host06kernel. This name is an internal name created from the :code:--display-name string which the end user will typically never see.

Command line "add" options

--help, -h ^^^^^^^^^^

         Show help information and exit.

--timeout TIMEOUT, -t TIMEOUT ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

         Specify timeout to wait for kernel startup text.
         This option is **not used** by :code:`sshpyk`. It is only used by :code:`ssh_ipykernel`.

--env [ENV ...], -e [ENV ...] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

         Specify environment variables for access by code executed within the remote Python
         kernel the form: :code:`"NAME=VALUE"`

--display-name DISPLAY_NAME, -d DISPLAY_NAME ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

         Specify the string to be used to describe this kernel to the end user.

--session ^^^^^^^^^

         Signal that session information should be stored in :code:`~/.sshpyk/sessions` for this kernel". This
         option is **not used** by :code:`ssh_ipykernel`. Note that when :code:`--session` is used the terminal
         output generated from the SSH connection is also directed to :code:`~/.sshpyk/sessions`.

--sudo, -s ^^^^^^^^^^

         Use :code:`sudo` to start kernel on the remote machine.
         This option is **not currently used** by :code:`sshpyk`. It is only used by :code:`ssh_ipykernel`.

--host HOST, -H HOST ^^^^^^^^^^^^^^^^^^^^

         The name of remote host as used to connect with SSH.

--python PATH, -p PATH ^^^^^^^^^^^^^^^^^^^^^^

         Path to remote python installation. This is the path to the root of the Python
         installation so the Python executable would be found in :code:`<PATH>/bin/python`.

SSH configuration notes

The host name used above is different from the Internet Protocol name for hosts which have a well defined address. It is also a name apart from the physical network where the host can be found. SSH allows for rationalizing the naming of the hosts to which you have access. This is done through the :code:$HOME/.ssh/config file.

The most useful configuration option with respect to :code:sshpyk is the ability to set up access to a host behind a bastion host <https://en.wikipedia.org/wiki/Bastion_host>_. Assuming, the host named :code:host06 from above is behind a bastion host, a configuration entry in :code:$HOME/.ssh/config like::

Host host06 User HOST06-USERNAME ForwardX11Trusted yes ProxyCommand ssh [e-mail] -W %h:%p

will allow the local account to use its SSH credentials for the user name :code:BASTION-USERNAME on :code:ssh.example.com for access to :code:host06 which is on some protected network behind :code:ssh.example.com. When the local account runs :code:ssh host06, SSH will first connect as :code:BASTION-USERNAME on :code:ssh.example.com and then it connect to host :code:host06 as username :code:HOST06-USERNAME.

This sort of configuration will allow :code:host06 to be allowed as a hostname for remote kernels.