sparrow-tool

sparrow_tool

---

Install

pip install sparrow-tool
# Or dev version
pip install sparrow-tool[dev]
# Or
pip install -e .
# Or
pip install -e .[dev]

Usage

Safe logger in multiprocessing

from sparrow.log import Logger
import numpy as np
logger = Logger(name='train-log', log_dir='./logs', )
logger.info("hello","numpy:",np.arange(10))

logger2 = Logger.get_logger('train-log')
print(id(logger2) == id(logger))
>>> True

Multiprocessing SyncManager

Open server first:

$ spr start-server

The defualt port 50001.

(Process1) productor:

from sparrow.multiprocess.client import Client
client = Client(port=50001)
client.update_dict({'a': 1, 'b': 2})

(Process2) consumer:

from sparrow.multiprocess.client import Client
client = Client(port=50001)
print(client.get_dict_data())

>>> {'a': 1, 'b': 2}

Common tools

• Kill process by port

  $ spr kill {port}
  

• pack & unpack
  support archive format: "zip", "tar", "gztar", "bztar", or "xztar".

  $ spr pack pack_dir
  

  $ spr unpack filename extract_dir
  

Vector Database && Search

Milvus

• start

  $ spr milvus start
  

• stop

  $ spr milvus stop
  

• remove database

  $ spr milvus rm