Research
Recent Trends in Malicious Packages Targeting Discord
The Socket research team breaks down a sampling of malicious packages that download and execute files, among other suspicious behaviors, targeting the popular Discord platform.
Readme
This is the implementation of the approach described in the paper:
Sriram Krishna and Nishant Sinha. Gestop: Customizable Gesture Control of Computer Systems 8th ACM IKDD CODS and 26th COMAD. 2021. 405-409.
Built on top of mediapipe, this project aims to be a tool to interact with a computer through hand gestures. Out of the box, using this tool, it is possible to:
In addition, it is possible to extend and customize the functionality of the application in numerous ways:
Installation using pip
inside a virtual environment is highly recommended. To do so:
python -m venv env
source env/bin/activate
pip install gestop
In addition to the Python dependencies, OpenCV and xdotool are also required by Gestop.
To start the Gestop server:
python -m gestop.receiver
The client, or the keypoint generator, can be setup either through MediaPipe's C++ API, or through its Python API. The Python API is simpler to setup and is recommended.
python -m gestop.keypoint_gen.hand_tracking
Note: Run build instructions in the mediapipe/
directory, not inside this directory.
bazel build -c opt --verbose_failures --copt -DMESA_EGL_NO_X11_HEADERS --copt -DEGL_NO_X11 gestop:hand_tracking_gpu
GLOG_logtostderr=1 bazel-bin/gestop/hand_tracking_gpu --calculator_graph_config_file=gestop/gestop/keypoint_gen/hand_tracking_desktop_live.pbtxt
bazel build -c opt --define MEDIAPIPE_DISABLE_GPU=1 gestop:hand_tracking_cpu
GLOG_logtostderr=1 bazel-bin/gestop/hand_tracking_cpu --calculator_graph_config_file=gestop/keypoint_gen/hand_tracking_desktop_live.pbtxt
The hand keypoints are detected using google's MediaPipe. These keypoints are then fed into receiver.py
. The tool recognizes two kinds of gestures:
Static gestures, by default, are mapped to all functionality relevant to the mouse, such as left mouse click, scroll etc. Combined with mouse tracking, this allows one to replace the mouse entirely. The mouse is tracked simply by moving the hand, where the tip of the index finger reflects the position of the cursor. The gestures related to the mouse actions are detailed below. To train the neural network to recognize static gestures, a dataset was created manually for the available gestures.
For more complicated gestures involving the movement of the hand, dynamic gestures can be used. By default, it consists of various other actions to interface with the system, such as modifying screen brightness, switching workspaces, taking screenshots etc. The data for these dynamic gestures comes from SHREC2017 dataset. Dynamic gestures are detected by holding down the Ctrl
key, which freezes the cursor, performing the gesture, and then releasing the key.
The project consists of a few distinct pieces which are:
receiver.py
, responsible for handling the stream and utilizing the following modules.mouse_tracker.py
, responsible for moving the cursor using the position of the index finger.recognizer.py
, takes in the keypoints from the mediapipe executable, and converts them into a high level description of the state of the hand, i.e. a gesture name.executor.py
, uses the gesture name from the previous module, and executes an action.executor.py
. Use the given find_keycode.py
script to find the keycodes of the keys used to change screen brightness and volumee. Finally, system shortcuts may need to be remapped so that the shortcuts work even with the Ctrl key held down. For example, in addition to the usual default behaviour of <Prnt_Screen>
taking a screenshot, you may need to add <Ctrl+Prnt_Screen>
as a shortcut as well.FAQs
Navigate Desktop with Gestures
We found that gestop demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 2 open source maintainers collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Research
The Socket research team breaks down a sampling of malicious packages that download and execute files, among other suspicious behaviors, targeting the popular Discord platform.
Security News
Socket CEO Feross Aboukhadijeh joins a16z partners to discuss how modern, sophisticated supply chain attacks require AI-driven defenses and explore the challenges and solutions in leveraging AI for threat detection early in the development life cycle.
Security News
NIST's new AI Risk Management Framework aims to enhance the security and reliability of generative AI systems and address the unique challenges of malicious AI exploits.