Forward Propagation, Backward Regression and Pose Association for Hand Tracking in the Wild (CVPR 2022)

Overview

HandLer

This repository contains the code and data for the following paper:

Forward Propagation, Backward Regression, and Pose Association for Hand Tracking in the Wild (CVPR 2022).

We develop a method to detect and track multiple hands in videos.

Installation

Install Torch 1.7.1

pip install torch==1.7.1+cu110 torchvision==0.8.2+cu110 torchaudio==0.7.2 -f https://download.pytorch.org/whl/torch_stable.html

Install other requirements

pip install -r requirements.txt

Install Detectron2 (version 0.4)

For CUDA 11.0 and Torch 1.7.x

python -m pip install detectron2==0.4 -f \
  https://dl.fbaipublicfiles.com/detectron2/wheels/cu110/torch1.7/index.html

Youtube-Hand Dataset

Please find the Youtube-Hand dataset in the project page.

Go to main folder

cd ./projects/HandLer

Model Weights

Please find the pretraind model weights here:

https://drive.google.com/file/d/1eaZn9E7lHvXY3Fh90f2eIK1P2yZwNfnP/view?usp=sharing

Please find the weights trained on Youtube-Hand training set here:

https://drive.google.com/file/d/1sK5bRTHt5zLOVPruUt-zyZ_AOrBvGQ9-/view?usp=sharing

Hand Tracking Demo: Run HandLer on custom videos

python demo.py --cfg configs/yt_trk.yaml --output output_vid.mp4 --weights /path/to/weights --input input_vid.mp4

Training on Youtube-Hand Dataset

For training on single GPU:

python train_trk.py --config-file configs/yt_trk.yaml

For training on multiple GPUs:

python train_trk.py --config-file configs/yt_trk.yaml --num_gpu 4

Testing on Youtube-Hand Test set

python tracking.py --cfg /path/to/cfg --out_dir /path/to/results/folders --weights /path/to/weights --root_path /path/to/dataset

Evaluating on Youtube-Hand Test set

python evaluations.py /path/to/results/folders

Improving Hand Tracking using Hand-Body Association

By associating hands with human bodies and linking human bodies across frames, we can establish correspondence between detected instances of the same hand across different frames, reducing identity switches in tracking. To know more about this, please check our work Whose Hands Are These? Hand Detection and Hand-Body Association in the Wild

Citation

If you find our code or dataset useful, please cite:

@inproceedings{handler_2022,
      title={Forward Propagation, Backward Regression and Pose Association for Hand Tracking in the Wild},
      author={Mingzhen Huang and Supreeth Narasimhaswamy and Saif Vazir and Haibin Ling and Minh Hoai},
      booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
      year={2022},
  }

@inproceedings{bodyhands_2022,
      title={Whose Hands Are These? Hand Detection and Hand-Body Association in the Wild},
      author={Supreeth Narasimhaswamy and Thanh Nguyen and Mingzhen Huang and Minh Hoai},
      booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
      year={2022},
  }
Owner
[email protected]
Computer Vision Lab at Stony Brook University
<a href=[email protected]">
Tools ini hanya bisa digunakan untuk menyerang website atau http/s

☢️ Tawkun DoS ☢️ Tools ini hanya bisa digunakan untuk menyerang website atau http/s FITUR: [ ☯️ ] Proxy Mode [ 🔥 ] SOCKS Mode | Kadang Eror [ ☢️ ] Ht

Bandhitawkunthi 9 Jul 19, 2022
A simpler way to make forms, surveys, and reaction input using discord.py.

discord-ext-forms An easier way to make forms and surveys in discord.py. This module is a very simple way to ask questions and create complete forms i

thrizzle 16 Nov 06, 2022
Coinbase Listing Sniper

Coinbase Listing Sniper Script that listens to the @CoinbaseAssets twitter to find information about new Coinbase listings, and automatically buys 100

4 Oct 26, 2022
An API wrapper around Discord API.

NeoCord This project is work in progress not for production use. An asynchronous API wrapper around Discord API written in Python. Features Modern API

Izhar Ahmad 14 Jan 03, 2022
A chatbot on Telegram using technologies of cloud computing.

Chatbot This project is about a chatbot on Telegram to study the cloud computing. You can refer to the project of chatbot-deploy which is conveinent f

Jeffery 4 Apr 24, 2022
Tools used by Ada Health's internal IT team to deploy and manage a serverless Munki setup.

Serverless Munki This repository contains cross platform code to deploy a production ready Munki service, complete with AutoPkg, that runs entirely fr

Ada Health 17 Dec 05, 2022
Complete portable pipeline for masking of Aadhaar Number adhering to Govt. Privacy Guidelines.

Aadhaar Number Masking Pipeline Implementation of a complete pipeline that masks the Aadhaar Number in given images to adhere to Govt. of India's Priv

1 Nov 06, 2021
A modified Sequential and NLP based Bot

A modified Sequential and NLP based Bot I improvised this bot a bit with some implementations as a part of my own hobby project :) Note: I do not own

Jay Desale 2 Jan 07, 2022
Its Is A Telegram Maths Basic Calculator Bot

Its Is A Telegram Maths Basic Calculator Bot

ANKIT KUMAR 1 Dec 26, 2021
Robot Swerve Test Public With Python

Robot-Swerve-Test-Public The codebase for our swerve drivetrain prototype robot.

1 Jan 09, 2022
A collection of automation aids to connect various database systems into Lookout for Metrics

A collection of automation aids to connect various database systems into Lookout for Metrics

AWS Samples 3 Apr 28, 2022
A discord bot that utilizes Google's Rest API for Calendar, Drive, and Sheets

Bott This is a discord bot that utilizes Google's Rest API for Calendar, Drive, and Sheets. The bot first takes the sheet from the schedule manager in

1 Dec 04, 2021
Telegram Bot Repo Capable of fetching the following Info via Anilist API inspired from AniFluid and Nepgear

Telegram Bot Repo Capable of fetching the following Info via Anilist API inspired from AniFluid and Nepgear Anime Airing Manga Character Scheduled Top

Rikka-Chan 2 Apr 01, 2022
A fully decentralized protocol for private transactions FAST snipe BUY token on LUANCH after add LIQUIDITY

TORNADO CASH Pancakeswap Sniper BOT 2022-V1 (MAC WINDOWS ANDROID LINUX) ⭐️ A fully decentralized protocol for private and safe transactions ⭐️ AUTO DO

Crypto Trader 2 Jan 06, 2022
Osmnx-examples - Usage examples, demos, and tutorials for OSMnx.

OSMnx Examples OSMnx is a Python package to work with street networks and other spatial data from OpenStreetMap: retrieve, model, analyze, and visuali

Geoff Boeing 1.2k Jan 03, 2023
An open source raffle bot made to increase the chance of winning limited sneaker raffles by automating entries.

🚀 SyneziaRaffles An open source raffle bot made to increase the chance of winning limited sneaker raffles by automating entries. 🏄‍♂️ Quick Start Pr

Alexis M. 29 Dec 22, 2022
The Fastest multi spambot of Telegram 🤞 🤞

Revil Spam Bot The Fastest multi spambot of Telegram 🤞 🤞 𝚂𝚄𝙿𝙿𝙾𝚁𝚃 🖤 ᴄʀᴇᴀᴛᴏʀ 🖤 ⚡ 𝓡𝓮𝓿𝓲𝓵 𝓗𝓾𝓷𝓽𝓮𝓻 𝔐𝔲𝔩𝔱𝔦 ẞø✞︎ ⚡ 𝓐 𝕾мοοτн 𝓐и∂ 𝕱

REVIL HUNTER 4 Dec 08, 2021
An Amazon Price Tracker app helps you to buy which product you want within sale price by sending an E-Mail.

Amazon Price Tracker An Amazon Price Tracker app helps you to buy which product you want within sale price by sending an E-Mail. Installing Download t

Aytaç Kaşoğlu 2 Feb 10, 2022
Jackrabbit Relay is an API endpoint for stock, forex and cryptocurrency exchanges that accept REST webhooks.

JackrabbitRelay Jackrabbit Relay is an API endpoint for stock, forex and cryptocurrency exchanges that accept REST webhooks. Disclaimer Please note RA

Rose Heart 23 Jan 04, 2023
Github repository started notify 💕

Github repository started notify 💕

4 Aug 06, 2022