VASLs: a tool set for annotate ASL video sequence

VASLs is the byproduct of our automatic ASL video sequence recognizer. It is a set of tools that can help annotate ASL video sequences. The purpose of building these tools is to facilitate us to generate large amount of ASL training data. The main task of these tools is to segment the videos into ASL sentences, label each frame in the sentences as their corresponding signs, extract signs from the sentences, build candidate hand sets, extract the hand and face areas for training purpose.

We have tested it on Microsoft Windows XP and Intel Pentium III

Related Publications:

  • R.D. Yang. , S. Sarkar, B. Loeding and A. Karshmer, “Efficient Generation of Large Amount of Training Data for Sign Language Recognition: A semi-automatic tool”, International Conference on Computers Helping People with Special Needs (ICCHP’06), 2006.
  • R.D. Yang. and S. Sarkar, "Gesture Recognition using HMM from fragmented observation ", IEEE International Conference on Computer Vision and Pattern Recognition 2006.

VASLs is based on the Intel OpenCV Library:

VASLs used the meanshift algorithm source code from Robust Image Understanding Laboratory

The core algorithm of VASLs is developed and discussed by the ASL Group at USF

VASLs was written and is now being updated by Alan Yang.

Special Thanks to Melinda Black for testing the software and editing the documentation.

 

HandGenerator

This tool can help generate skin likelihood image, motion likelihood image of the sentences/signs video sequences. also it can generate a lot of candidate hands, which facilitate the step of manual hand segmentation. This tool is automatic.

HandSelector

This tool uses the candidate hands generated from HandGenerator, providing tracking, region of interest selecting functionality to segment the hands out of the scene.

VideoSegmentor

 This tool helps segment a long video into ASL sentences. Virtualdub is needed for the final segmentation. .Net Framework is required to run this program.

Framelabler

This tool helps label each frame in one ASL sentence either as one sign or coarticulation. The output is a text file with the corresponding labels. 

SignExtractor

This tool is a simple matlab function. Given the sentences and labels from the above tools, it automatically extracts signs and stores them into different directories.