.. _NDEx: https://ndexbio.org .. _NDEx CX: https://www.home.ndexbio.org/data-model/
Warning: This repository is for development and features may change. Please use this at your own risk.
.. image:: https://img.shields.io/pypi/v/ndexutil.svg :target: https://pypi.python.org/pypi/ndexutil
.. image:: https://img.shields.io/travis/ndexbio/ndexutil.svg :target: https://travis-ci.org/ndexbio/ndexutil
.. image:: https://coveralls.io/repos/github/ndexbio/ndexutils/badge.svg?branch=master :target: https://coveralls.io/github/ndexbio/ndexutils?branch=master
.. image:: https://readthedocs.org/projects/ndexutils/badge/?version=latest :target: https://ndexutil.readthedocs.io/en/latest/?badge=latest :alt: Documentation Status
ndex2 <https://pypi.org/project/ndex2>_networkx <https://pypi.org/project/networkx>_biothings_client <https://pypi.org/project/biothings-client>_requests <https://pypi.org/project/requests>_requests-toolbelt <https://pypi.org/project/requests_toolbelt>_pandas <https://pypi.org/project/pandas>_mygene <https://pypi.org/project/mygene>_jsonschema <https://pypi.org/project/jsonschema>_urllib3 <https://pypi.org/project/urllib3>_ijson <https://pypi.org/project/ijson>_Installed if pip install ndexutil[cytoscape] is run
py4cytoscape <https://pypi.org/project/py4cytoscape>_.. code-block::
git clone https://github.com/ndexbio/ndexutils cd ndexutils make dist pip install dist/ndexutil*whl
OR via PyPI <https://pypi.org/ndexutils>_
.. code-block::
pip install ndexutil
WARNING: Please consider this tool alpha and probably contains errors/bugs and could cause data corruption or loss. You have been warned
ndexmisctools.py lets caller perform some operations on NDEx_ via the
command line.
This tool follows format of ndexmisctools.py <COMMAND> <args>
For more information run ndexmisctools.py --help and ndexmisctools.py <COMMAND> --help
COMMANDS:
addnodeattrib - adds an attribute to all nodes in a network
Credentials to NDEx_ server must be stored in the configuration (default ~/.ndexutils.conf)
and be formatted as follows:
.. code-block::
[addprofile] user=bob password=6ea8f0ab0b2e server=public.ndexbio.org
The following command adds foo node attribute with value
my new value to all nodes in the network 9025480b-6fbc-4efe-9cd8-b575ce49dfda
.. code-block::
ndexmisctools.py --profile addprofile addnodeattrib --uuid 9025480b-6fbc-4efe-9cd8-b575ce49dfda --name foo --value 'my new value'
.. warning::
This command changes the network in place, so it is a destructive call with NO undo
copynetwork - copies NDEx_ network between accounts and even servers
For copying the source and destination credentials must be stored in the configuration (default ~/.ndexutils.conf)
and be formatted as follows:
.. code-block::
[mycopyprofile] source_user=bob source_password=6ea8f0ab0b2e source_server=public.ndexbio.org dest_user=smith dest_password=4efe9cd8 dest_server=public.ndexbio.org
The following command copies the network 9025480b-6fbc-4efe-9cd8-b575ce49dfda from source credentials defined in configuration to dest
.. code-block::
ndexmisctools.py --profile mycopyprofile copynetwork --uuid 9025480b-6fbc-4efe-9cd8-b575ce49dfda
cytoscapelayout - updates layout for network in NDEx_ using Cytoscape
NOTE: This tool requires installation of py4cytoscape <https://pypi.org/project/py4cytoscape>_
and Cytoscape to be activately running (tested with Cytsocape 3.8+)
This command requires four positional parameters. The first is the layout algorithm
to use followed by (username, password, and server) which are credentials for
NDEx_ server to update the layout on the network.
Any of these credential fields set to '-' will
force this tool to obtain the information from (default ~/.ndexutils.conf) configuration file
under the profile specified by the --profile field in this format:
.. code-block::
[<value of --profile>]
user = <NDEx username>
password = <NDEx password>
server = <NDEx server ie public.ndexbio.org>
Using credentials from myattrib profile, the following command adds grid
layout to the network 9025480b-6fbc-4efe-9cd8-b575ce49dfda
.. code-block::
ndexmisctools.py --profile myattrib networkxlayout grid - - - --uuid 9025480b-6fbc-4efe-9cd8-b575ce49dfda
To get a list of available layouts printed to standard out, replace the layout with listlayout. Just put
fake values in for credentials and any text for the --uuid flag:
.. code-block::
ndexmisctools.py cytoscapelayout listlayout x y z --uuid ignored
deletenetwork - deletes a network or all networks under a given networkset from NDEx_
Credentials to NDEx_ server must be stored in the configuration (default ~/.ndexutils.conf)
and be formatted as follows:
.. code-block::
[deleteprofile] user=bob password=6ea8f0ab0b2e server=public.ndexbio.org
The following command deletes the network with UUID 9025480b-6fbc-4efe-9cd8-b575ce49dfda from NDEx_
.. code-block::
ndexmisctools.py --profile deleteprofile deletenetwork --uuid 9025480b-6fbc-4efe-9cd8-b575ce49dfda
uploadnetwork - uploads network to NDEx_
This command uploads a network to NDEx_
Credentials must be stored in the configuration (default ~/.ndexutils.conf)
and be formatted as follows:
.. code-block::
[myattrib] user=bob password=6ea8f0ab0b2e server=public.ndexbio.org
The following command uploads foo.cx network to NDEx_
.. code-block::
ndexmisctools.py --profile myattrib uploadnetwork foo.cx
featurednetworkreport - generates report about featurend networks on NDEx_
This command requires one positional parameter server.
For production use public.ndexbio.org
If the positional parameter is set to '-', it will
force this tool to obtain the information from (default ~/.ndexutils.conf) configuration file
under the profile specified by the --profile field in this format:
.. code-block::
[<value of --profile>]
server = <NDEx server ie public.ndexbio.org>
The command below queries the production server for the featured networks json file
and then queries NDEx_ for all networks that are part of the featured networks.
.. code-block::
ndexmisctools.py featurednetworkreport public.ndexbio.org --output foo.csv
NOTE: If a networkset contains more then 500 networks, only the first 500 are examined
networkattribupdate - updates network attributes on network in NDEx_
WARNING: Currently name, version, and description CANNOT be updated with this command.
Credentials must be stored in the configuration (default ~/.ndexutils.conf)
and be formatted as follows:
.. code-block::
[myattrib] user=bob password=6ea8f0ab0b2e server=public.ndexbio.org
The following command updates foo network attribute on the network 9025480b-6fbc-4efe-9cd8-b575ce49dfda
.. code-block::
ndexmisctools.py --profile myattrib networkattribupdate --uuid 9025480b-6fbc-4efe-9cd8-b575ce49dfda --name foo --type string --value 'my new value'
networkxlayout - updates layout for network in NDEx_ using networkx library
This command requires four positional parameters. The first is the layout algorithm
to use followed by (username, password, and server) which are credentials for
NDEx_ server to update the layout on the network.
Any of these credential fields set to '-' will
force this tool to obtain the information from (default ~/.ndexutils.conf) configuration file
under the profile specified by the --profile field in this format:
.. code-block::
[<value of --profile>]
user = <NDEx username>
password = <NDEx password>
server = <NDEx server ie public.ndexbio.org>
By default this tool only updates the cartesianLayout aspect. To update the full network
use add the --updatefullnetwork flag.
Using credentials from myattrib profile, the following command adds spring
layout to the network 9025480b-6fbc-4efe-9cd8-b575ce49dfda
.. code-block::
ndexmisctools.py --profile myattrib networkxlayout spring - - - --uuid 9025480b-6fbc-4efe-9cd8-b575ce49dfda
removenodeattrib - removes an attribute from all nodes in a network
Credentials to NDEx_ server must be stored in the configuration (default ~/.ndexutils.conf)
and be formatted as follows:
.. code-block::
[removeprofile] user=bob password=6ea8f0ab0b2e server=public.ndexbio.org
The following command removes foo node attribute from all nodes
in the network 9025480b-6fbc-4efe-9cd8-b575ce49dfda
.. code-block::
ndexmisctools.py --profile removeprofile removenodeattrib --uuid 9025480b-6fbc-4efe-9cd8-b575ce49dfda --name foo
.. warning::
This command changes the network in place, so it is a destructive call with NO undo
systemproperty - updates showcase, visibility, and indexing for single network or all networks in networkset in NDEx_
NOTE: --showcase has no effect if network visibility is private
Credentials must be stored in the configuration (default ~/.ndexutils.conf)
and be formatted as follows:
.. code-block::
[myattrib] user=bob password=6ea8f0ab0b2e server=public.ndexbio.org
The following command enables showcase and sets indexing to meta for network with id 9025480b-6fbc-4efe-9cd8-b575ce49dfda
.. code-block::
ndexmisctools.py --profile myattrib systemproperty --uuid 9025480b-6fbc-4efe-9cd8-b575ce49dfda --showcase --indexlevel meta
The following command sets visibility to public for all networks in networkset with id e9580d43-ec14-4be8-9977-9de88e1d410a
.. code-block::
ndexmisctools.py --profile myattrib systemproperty --networksetid e9580d43-ec14-4be8-9977-9de88e1d410a --visibility public
styleupdate - update style of network in NDEx_
Credentials must be stored in the configuration (default ~/.ndexutils.conf)
and be formatted as follows:
.. code-block::
[styleprofile] user=bob password=6ea8f0ab0b2e server=public.ndexbio.org
The following command updates style of network with UUID 9025480b-6fbc-4efe-9cd8-b575ce49dfda with style from network specified by this UUID e9580d43-ec14-4be8-9977-9de88e1d410a
.. code-block::
ndexmisctools.py --profile styleprofile styleupdate --uuid 9025480b-6fbc-4efe-9cd8-b575ce49dfda --styleuuid e9580d43-ec14-4be8-9977-9de88e1d410a
tsvloader - Loads TSV files as networks into NDEx_
The tsvloader command loads an edge list file in tab separated format (hence TSV) and using a load plan, loads that data as a network into NDEx <https://ndexbio.org>_.
This tool attempts to mimic behavior of the older tsv_uploader.py script located here: https://github.com/ndexbio/load-content
This new version uses the more memory efficient StreamTSVLoader.
This command requires five positional parameters.
The first three (username, password, and server) are credentials for
NDEx_ server to upload the network.
Any of these first three credential fields set to '-' will
force this tool to obtain the information from (default ~/.ndexutils.conf) configuration file
under the profile specified by the --profile field in this format:
.. code-block::
[<value of --profile>]
user = <NDEx username>
password = <NDEx password>
server = <NDEx server ie public.ndexbio.org>
The forth positional parameter tsv_file (see _TSV Loader section below) should be
set to edge list file in tab separated format and the
fifth or last positional parameter load_plan should be
set to the load plan. The load plan is a JSON formatted text
file that maps the columns to nodes, edges, and attributes
in the network.
By default this tool does not generate much output to
standard out/error. For more verbosity add three to five -v parameters
to left of command name tsvloader as seen in examples below.
NOTE: If using --layout to run a Cytoscape layout, py4cytoscape <https://pypi.org/project/py4cytoscape>_
must be installed and Cytoscape to be activately running (tested with Cytsocape 3.8+)
In example below the - - - tells the program to read the credentials
from the configuration file
.. code-block::
ndexmisctools.py -vvvv tsvloader - - - datafile.tsv load.plan
Here is an example where the name and description of the network is set
and the -t specifies a template network used to get style and in this case
since --copyattribs is set the network attributes (minus @context) are
also copied to the new network
.. code-block::
ndexmisctools.py -vvv tsvloader bob xx public.ndexbio.org \
datafile.tsv loadplan.json --uppercaseheader \
-t dafe07ca-0676-11ea-93e0-525400c25d22 \
--name mynetwork --description 'some text' --copyattribs
In this example an alternate header is prepended via the --header flag.
NOTE: The --header flag does NOT remove an existing header
.. code-block::
ndexmisctools.py -vv --profile foo tsvloader - - public.ndexbio.org \
datafile.tsv loadplan.json \
--header 'col1 col2 col3' \
-t some_cx_file.cx \
-u 48a26aa0-0677-11ea-93e0-525400c25d22
If successful 0 is returned otherwise there was an error.
.. _TSV_Loader: .
This module contains the Tab Separated Variable Loader (TSV Loader, see ndexutil/streamtsvloader.py module) which generates
an NDEx CX_ file from a tab separated
text file of edge data and attributes.
To load data a load plan must be created. This plan tells the loader how to map the
columns in the file to nodes, and edges. This load plan needs to validate against
this load plan JSON schema <https://github.com/ndexbio/ndexutils/blob/master/ndexutil/tsv/loading_plan_schema.json>_
Example TSV file
.. code-block::
SOURCE TARGET WEIGHT
ABCD AAA1 0.555
GGGG BBBB 0.305
SOURCE is the source node, TARGET is target node
A schema that could be:
.. code-block::
{
"source_plan":
{
"node_name_column": "SOURCE"
},
"target_plan":
{
"node_name_column": "TARGET"
},
"edge_plan":
{
"default_predicate": "unknown",
"property_columns": [
{
"column_name": "WEIGHT",
"attribute_name": "weight",
"data_type": "double"
}
]
}
}
Example below assumes the following:
NDEx CX_ with a style... code-block::
import ndex2
from ndexutil.tsv.streamtsvloader import StreamTSVLoader
# using ndex2 client library read CX file as NiceCXNetwork object
style_network = ndex2.create_nice_cx_from_file('./style.cx')
loader = StreamTSVLoader('./loadplan.json', style_network)
with open('./input.tsv', 'r') as tsvfile:
with open('./output.cx', 'w') as outfile:
loader.write_cx_network(tsvfile, outfile)
tqdm <https://pypi.org/project/tqdm>__ and ijson <https://pypi.org/project/ijson>__ are now required packages
Added --indexlevel, --showcase, --visibility, and --destnetworkset to
ndexmisctools.py copynetwork command to allow caller to set copied
networks accessibility and to add copied network to a networkset. This command
also stores the original NDEx network UUID in __copynewtork_orig_ndex_uuid network
attribute
Added new command to ndexmisctools.py named uploadnetwork that lets caller upload networks to NDEx
Added following methods to ndexutil/ndex.py:
get_network_summary(), get_nice_cx_from_server(), set_index_and_showcase(), save_network_to_ndex()
--styleuuid was failing to update network in
ndexmisctools.py styleupdate commandAdded Cytoscape layouts to ndexmisctools.py tsvloader via --layout flag
Passing - for layout to ndexmisctools.py cytoscapelayout will run force-directed-cl
layout in Cytoscape
ndexmisctools.py cytoscapelayout by default updates only the layout aspect
on network in NDEx. This can be changed to perform a full network update with
new --updatefullnetwork flag
--layout flag to ndexmisctools.py tsvloader command to enable layouts
to be applied to network generated by loaderAdded new command to ndexmisctools.py named networkxlayout that lets caller apply networkx layouts to network on NDEx
Added new command to ndexmisctools.py named cytoscapelayout that lets caller apply Cytoscape layouts to network on NDEx
Added new command to ndexmisctools.py named featurednetworkreport that generates a report about featured networks on NDEx
Added new command to ndexmisctools.py named addnodeattrib that lets caller add an attribute to all nodes in a network
Added new command to ndexmisctools.py named removenodeattrib that lets caller remove an attribute from all nodes in a network
Added --typeonly flag to networkattribupdate command in ndexmisctools.py that lets caller just update the data type for a given network attribute
Added new command to ndexmisctools.py named deletenetwork that lets caller delete a network or delete all networks in a given networkset from NDEx
Added new command to ndexmisctools.py named styleupdate that lets caller update the style of a network or all networks in a given set on NDEx
Fixed bug in StreamTSVLoader where 2 @context network attributes were added to network if @context was set in load plan and @context was passed in the network attributes parameter for write_cx_network()
Added new command to ndexmisctools.py named tsvloader that lets caller load TSV files as networks into NDEx
Added GeneSymbolSearcher class to loaderutils.py module
Minor bug fix in NetworkIssueReport get_fullreport_as_string() where issue text is wrapped in str() in case its not a string
FAQs
Unsupported NDEx utilities
We found that ndexutil demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 4 open source maintainers collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
Research
The Socket Research Team breaks down a malicious wrapper package that uses obfuscation to harvest credentials and exfiltrate sensitive data.
Research
Security News
Attackers used a malicious npm package typosquatting a popular ESLint plugin to steal sensitive data, execute commands, and exploit developer systems.
Security News
The Ultralytics' PyPI Package was compromised four times in one weekend through GitHub Actions cache poisoning and failure to rotate previously compromised API tokens.