By the end of my PhD research, the prototype of the DSW system for dynamic hand gesture and movement tracking and recognition has been successfully developed. As you may see from the attached video file, with the hand posture
data gathered from the ShapeHand data gloves as well as the hand orientation and movement trajectories acquired by the InterSense wrist trackers, the DSW system is shown to be able to produce the corresponding SW symbols by recognising dynamic and complex
hand gestures automatically. These gestures and movements contains different hand configurations.
For hand gesture, the fingers configuration can be classified as Close/half-close/open and abduction/adduction; hand orientations can be classified as left/right hand being placed on floor/wall plane and on each plane it has eight principal directions, which
are horizontal/vertical/diagonal. For hand movements, they can be classified as: left/right hand movements on floor/wall plane,
on each plane it has eight principal directions, which are horizontal/vertical/diagonal. The movement can be straight/curved movement paths, clockwise/anti-clockwise
movements, and single/repeated movements.
Furthermore, the DSW system includes a special visual interface to enable the user to see his/her hand movements as well as the SW symbols produced in each stage of the signing sequence.
To test the accuracy and effectiveness of the system, some example gestures have been selected and tested, which are taken from the first page of the book of "Frosty the Snowman”.
The SignWriting symbols can be found on
Although there are still some errors for the gesture and movement recognition, and many technical difficulties need to be overcome, I reckon it provided a novel idea for automatic
direct sign writing based on dynamic hand gestures and movements, and it forms the basis to develop a fully human gesture recognition system based on all human gestures.
Above just a brief summary of the work we have done.
Finally, I would like to take this opportunity to thank all of you, especially Val and Adam, who have given me many useful help and suggestions throughout this research. Thanks very much to you!