Choreography for Pure Dance Movements
A Thesis
submitted to Goa University for the award of the degree of
Doctor of Philosophy by
Sangeeta Chakrabarty
under the Guidance of Dr. Jyoti D. Pawar
Department of Computer Science and Technology Goa University
Taleigao Plateau, Goa
May 2017
Declaration of Authorship
As required under the ordinance OB-9.9 of Goa University, I, Sangeeta Chakrabarty, state that the present Ph.D. thesis entitled, ‘Automation of BharataNatyam Choreography for Pure Dance Movements’ is my original contribution carried out under the supervision of Dr. Jyoti D. Pawar, Associate Professor, Department of Computer Science and Technology, Goa University and the same has not been submitted on any previous occasion. To the best of my knowledge, the present study is the first comprehensive work of its kind in the area mentioned.
The literature related to the problem investigated has been cited. Due acknowledgments have been made whenever facilities and suggestions have been availed of.
(Sangeeta Chakrabarty)
Certificate
This is to certify that the thesis entitled ‘Automation of BharataNatyam Choreography for Pure Dance Movements’, submitted by Ms. Sangeeta Chakrabarty for the award of the degree of Doctor of Philosophy in Computer Science, is based on her original studies carried out under my supervision. The thesis or any part thereof has not been previously submitted for any other degree or diploma in any University or Institute.
Dr. Jyoti D. Pawar
Department of Computer Science and Technology Goa University, Taleigao Plateau
Goa-403206, India.
“Creativity is contagious. Pass it on.”
Anonymous
Automation of BharataNatyam Choreography for Pure Dance Movements
by
Sangeeta Chakrabarty Abstract
Choreographers invent and design dance moves co-ordinated with music. They may work with dance groups or individuals. It may even sometimes take several weeks to choreograph and perfect a move for a small dance choreography of few seconds. The cre- ativity of an individual makes a major impact on the choreography.
In this research we have attempted to enhance the creative aspect of a choreographer for pure, rhythmic dance movements called as Nritta using Information Technology. Automation of creative aspects like painting, music composing or even dance choreogra- phy is a challenging task. Indian Classical Dance (ICD), specially BharataNatyam (BN) is a very stylized and formalized sequence of movements following the ancient manuscript on dance and dramaturgy called as Natyashastra (NS). Dance choreography has movement sequences planned and executed to perfection. Nritta is purely technical dance which does not convey any meaning. The accepted norms for bestNrittachoreography in BN areadavuswhich are usually practiced and perfected by all choreographers and dance students. The novelty in choreography would be to experiment with the existing basic set of 13 adavusand their variations and discover some new moves within the BN framework.
Dance choreography requires expertise with years of practice.
The choreographers usually tend to select the same sequence of adavus that was taught to them. This happens due to two main reasons: first being, comfort and convenience of what is readily available and secondly the lack of creativity and innovation along with the risk to avoid breaking tradition. Although dance choreog- raphy is a very intuitive domain, the use of computational power for such a creative aspect is less heard of. However several researchers have used Information Technology for several aspects of dance like dance education, assisting during live stage performances,etc. in Indian as well as Western Classical Dance forms.
In our research work, we have generated the required data sets, by modeling the 6 major body limbs (head, hands (right and left), waist and legs (right and left)). We did not use any data capturing methods like video capture or Kinect Cameras and sensors for data set generation. The reason was to innovate new choreographic skills and not capture the existing choreography. We represented the human body by using the 30 attribute Dance Position (DP) vector after carrying out a detailed dance literature survey. The combi- nations of the major body limbs were exponential in nature and
hence we used a Genetic Algorithm (GA) approach to choose the best move from amongst lakhs of possibilities. This GA approach, was designed with the help of a unique fitness function that could generate altogether new dance poses, with the help of existing adavus.The 30 attribute DP vector generated was in a numeric form and for deciphering the poses, we had to draw the Stick figures for the poses manually. We automated this time-consuming process, for stick figure generation by using GNU Octave. Using these Stick figures, we requested a dance expert to pose and captured the pictures one by one for the system generated poses. These pictures were used to assign the ratings for the newly generated poses, with the help of 10 dance experts. These experts are performers at the State, National and International Level. The ratings obtained from these dance experts were used to train a classifier considering them as gold standard data.
Next we experimented with Rough Set Theory (RST) to check the possibility of selecting the most significant attributes out of the 30 attributes used in DP vector. Using Rough Set Theory (RST) we could reduce the dimension of the data to 8 attributes from 30 attributes. However we noticed that it resulted in reduced classifier accuracy from 76.8% (with 30 attributes) to 56.4% for 8 attributes.
Hence we continued to experiment using 30 attributes for the DP vectors. The procedure to obtain the dance expert ratings was a time consuming process. Hence an attempt also has been made to measure the aesthetics of theadavusand the newly generated dance poses by using Fractal Dimension (FD). This helped to some extent in the automatic classification of the dance poses. The FD results are encouraging for carrying out further research in this direction.
Finally, N-Beat dance poses were automatically generated to support Multi-Beat Choreography using several filters proposed by us based on the BN dance literature survey.
The output of this doctoral research has been integrated to develop an user-friendly, interactive tool called as ArtToSMart(Art ToSystemModeled art). Using ArtToSMart the user can choose the starting pose and the system generates a user specified number of dance poses which are unique and aesthetically pleasing.
Acknowledgements
I would like to express my profound gratitude to my advisor, Dr. Jyoti D.
Pawar, Associate Professor, Dept. of Computer Science and Technology, Goa University for her guidance and constant support. Without her constant persuasion in her own patient and sweet way, it would’nt have been possible for me to complete this task.
University Grants Commission, New Delhi for granting funds for the Major Research Project under the XIth Plan from 2011 to 2014. Project Fellow, Mr. Clelo Andrade, who agreed to work with me at the most crucial final phase of the Project when we had very little time left.
I am deeply in debt to Dr. Manish R. Joshi, Associate Professor, Department of Computer Science, North Maharashtra University, Jalgaon, India, for having confidence in me and rendering this unique project with useful assistance throughout its tenure and making it a huge success.
I would also like to thank Dr. M. Sasikumar, Associate Director at CDAC, Mumbai, for his immense support and continuous encouragement.
A big thank you to Mrs. Sapna Naik, BharataNatyam lecturer, dance expert at Kala Academy, Panaji-Goa, for analyzing our project outputs, so that the project would be as perfect as possible. She has also spent several pain-staking hours in modeling and recreating our outputs for clicking the perfect pose.
Many dance teachers from Hyderabad, Pune, Bangalore, U.K, Den- mark, Goa and several renowned researchers and their students have contributed knowingly and unknowingly to our work. I would like to thank all of them for their profound interest and encouragement.
My Yoga therapist, Dr. Siddharth Sawaikar and Yoga Guru from Sports Authority of Goa, Mr. Suresh Kumar who helped me survive the most physically challenging phase of my studies through proper yoga training.
If it was not for them, I wouldn’t have been able to balance my life properly.
Thanks for making me pain-free!
Last but not the least I would like to thank my friends and fam- ily members. My loving parents; Mr. and Late Mrs Chakrabarti and my dear twins; Siddhanth and Siddharth Jadhav for their continuous encouragement and putting up with me during some strenuous times.
They never demanded anything and always supported me unconditionally.
Contents
Declaration of Authorship i
Abstract iv
Acknowledgements vi
Contents vii
List of Figures xi
List of Tables xiii
List of Algorithms xiv
List of Abbreviations xv
1 Introduction 1
1.1 BharataNatyam . . . 1
1.1.1 Introduction . . . 1
1.1.2 Structure of BN . . . 2
1.2 Motivation and Research Objective . . . 3
1.3 Contributions . . . 4
1.4 Thesis Organization. . . 4
2 Related Work 6 2.1 Dance notation and choreography . . . 6
2.2 Dance Capturing . . . 7
2.2.1 Using Sensors . . . 7
2.2.2 Motion capture . . . 7
2.2.2.1 With Monocular Vision. . . 7
2.2.2.2 With Multi view Vision . . . 8
2.3 Processing Captured Data . . . 8
2.3.1 Dance Semantics . . . 8
2.3.2 Annotation . . . 9
2.3.3 Ontology. . . 9
2.3.4 Dance Grammar (Verbs) . . . 10
2.3.5 Graph . . . 10
2.4 Dance generation . . . 10
2.4.1 Animated dance steps . . . 10
2.4.2 Computer aided choreography . . . 11
2.4.2.1 Fully automated . . . 11
2.4.2.2 Semi-automated . . . 12
2.4.3 Image generation . . . 12
2.5 Dance Processing Approaches . . . 12
2.5.1 Evolutionary Approach . . . 12
2.5.1.1 Genetic Algorithms . . . 12
2.5.1.2 Flock. . . 13
2.5.2 Classification . . . 13
2.5.2.1 Neural Networks . . . 13
2.5.2.2 Support Vector Machine(SVM). . . 13
2.5.3 Graph Based algorithms . . . 13
2.5.4 Image Processing through Gesture recognition . . . . 14
2.5.5 Corpus Based . . . 14
2.5.6 Multi agent system . . . 14
2.6 Conclusion . . . 15
3 Dance Pose Representation 16 3.1 Labanotation. . . 16
3.1.1 Basics of Labanotation . . . 16
3.1.2 Data Representation, Editing and Animation . . . 18
3.1.3 Use of Labanotation for BharataNatyam. . . 19
3.2 Capture Data through videos or Kinect Camera. . . 19
3.3 Modeling the Human Body for Pose Notation. . . 20
3.3.1 The initial stages of codification . . . 20
3.3.2 Modeling . . . 21
3.3.2.1 Bartenieff Fundamentals . . . 23
3.3.3 Orientation . . . 24
3.3.3.1 Head Orientation . . . 24
3.3.3.2 Waist Orientation . . . 25
3.3.3.3 Hands Orientation . . . 25
3.3.3.4 Legs Orientation . . . 27
3.3.4 Pose Notation for representingAdavus. . . 28
3.3.4.1 Adavus. . . 28
3.3.4.2 Coding of complicatedAdavus . . . 29
3.3.4.3 Coding of various Leg positions in KuchipudiDance . . . 32
3.4 Similarities and differences between DP vector and Labano- tation . . . 34
3.5 Conclusion . . . 35
4 Generation of Innovative moves for BN Choreography 37 4.1 Methodology . . . 37
4.1.1 Brute Force Approach . . . 37
4.1.2 Genetic Algorithm (GA) Approach . . . 38
4.1.2.1 Related work in GA used for choreographic creativity . . . 38
4.1.2.2 Factors affecting GA . . . 39
4.1.3 Designing of fitness function . . . 40
4.2 Summary. . . 48
4.3 Results . . . 49
4.4 Enhancing Robustness of the Results . . . 50
4.5 Conclusion . . . 50
5 Stick Figure Generation for Visualizing the Generated Poses 52
5.1 Related Work . . . 53
5.2 Stick Figure for BN Dance . . . 53
5.2.1 Mapping Dance Attributes . . . 53
5.2.1.1 Head Module: . . . 54
5.2.1.2 Hand Module: . . . 55
5.2.1.3 Torso Module: . . . 57
5.2.1.4 Leg Module: . . . 57
5.2.1.5 Stick Figure for Multiple beats:. . . 59
5.3 Experimental Results . . . 59
5.4 Conclusion . . . 60
6 Automatic Classification of the Generated Dance Poses 62 6.1 Classification. . . 62
6.2 Rough Set Theory (RST) for Dimension Reduction . . . 64
6.2.1 Why and how was RST used for our work? . . . 64
6.2.2 Reduct and Core . . . 68
6.3 Conclusion . . . 69
7 Aesthetic Evaluation of Poses through Fractal Dimension 71 7.1 Introduction . . . 71
7.1.1 Why was it necessary to evaluate the poses? . . . 71
7.2 Related Work . . . 71
7.2.1 Fractal Analysis used for Western Dances . . . 72
7.2.2 Box Counting method . . . 72
7.3 Experimental Results . . . 73
7.4 Conclusion . . . 80
8 Choreography using Innovative Moves 84 8.1 N-Beat Choreography . . . 84
8.1.1 Filters for Multi-beat choreography . . . 84
8.1.1.1 Hand Mudra Filter: . . . 87
8.1.1.2 Leg Filter:. . . 90
8.1.1.3 Fitness function Filter: . . . 92
8.2 Hardware and Software used . . . 97
8.3 The ArtToSMart Tool . . . 97
8.3.1 Result Evaluation . . . 98
8.3.2 Results . . . 98
8.4 Conclusion . . . 102
9 Conclusion and Future work 103 9.1 Conclusion . . . 103
9.2 Future Work . . . 104 A Tables that are used for N-Beat choreography 105
B Hand Gestures of BharataNatyam 106
C The Leg, Head and Neck Positions in BN 108
D Terminology Used 110
E Infeasible Dance Steps: The Elimination File 111
F The codifiedAdavulist 114
G Publications 115
Bibliography 117
List of Figures
3.1 A Hand-drawn Stick figure. . . 16
3.2 Labanotation Symbols . . . 17
3.3 The three Body Planes showing the Orientation of Hand and Leg in X,Y and Z axis . . . 23
3.4 An Adavu depicted through Vector. . . 29
3.5 Adavudepicted through Vector showing waist twist and a slight bend.. . . 30
3.6 Adavudepicted through Vector showing waist twist. . . 30
3.7 The dancer has turned completely back which is depicted through the highlighted hands and feet position. . . 31
3.8 Dancer turned completely back and then twisted (at 90◦) and bent (at 45◦) to look front. . . 31
3.9 Leg folded in the front inKuchipudidance.. . . 32
3.10 Leg folded in the front like a triangleKuchipudidance. . . 32
3.11 Leg sideways up inKuchipudidance. . . 33
3.12 Leg diagonally up inKuchipudidance. . . 33
4.1 The Dance Pose Representation Module.. . . 39
4.2 Block Diagram of GA Module for Generation of Innovative BN Poses.. . . 41
4.3 The Single Beat Activity Diagram. . . 41
4.4 The Normal Distribution Curve. . . 46
4.5 A Snapshot of the Dance Position Vectors generated.. . . 49
4.6 Few examples of generation of Innovative Moves . . . 50
5.1 A sample of Manually drawn Stick Figure. . . 52
5.2 Hierarchy Chart of the Stick Figure Generation Module.. . . 53
5.3 The Starting BN dance pose:Natyarambhe. . . 54
5.4 The head positions through the Stick Figure Module looking straight, side and tilt. . . 54
5.5 The Stick Figure depictingNatyarambhe. . . 55
5.6 The distinction between Right and Left Hand. . . 56
5.7 The loss in Z Dimension of the Hand. . . 56
5.8 Our Scheme for distinguishing between both the Hands. . . 57
5.9 The knee positions in BN. . . 58
5.10 6 poses Generated for 6 beats. . . 58
5.11 A system generated pose and it’s corresponding stick figure. 59 5.12 A Stick Figure which is assigned weight 1 . . . 60
5.13 A Stick Figure which is assigned weight 0.8 . . . 60
6.1 Screen-shot of accuracy for 224 instances through RSES. . . . 65
6.2 Screen-shot of 1866 rules for 224 instances through RSES. . . 65
6.3 Screen-shot of 4742 rules for 501 instances through RSES. . . 67
6.4 The 8 reducts obtained from RSES for 501 instances. . . 68
6.5 Core obtained from the reducts for 501 instances.. . . 69
7.1 Image J used for picture conversion . . . 73
7.2 An adavu image with Log-Log Plot and FD= 1.45. . . 73
7.3 ArtToSMart image with Log-Log Plot and FD= 1.60. . . 75
7.4 Expert 1 preferences with highest ratings . . . 78
7.5 Expert 2 preferences with highest ratings . . . 78
7.6 A common picture liked by all Experts . . . 79
7.7 Expert 3 preferences with highest ratings . . . 79
8.1 Overall Hierarchy Chart. . . 85
8.2 A 3 beat movement without applying any filter. . . 85
8.3 A Snapshot of the MultiBeat Traversal Program. . . 87
8.4 MultiBeatHierarchy Chart.. . . 88
8.5 The Multi- Beat Module. . . 88
8.6 A Snapshot showing the Selection table. . . 90
8.7 Snapshot of a section of the goodvector table stored in MYSQL workbench. . . 94
8.8 Snapshot of a section of the choosevector table stored in MYSQL workbench. . . 96
8.9 Snapshot of a section of the chooseclosest table stored in MYSQL workbench. . . 96
8.10 Snapshot of a section of the choreography table stored in MYSQL workbench. . . 97
8.11 Screen shot of User Interface. . . 98
8.12 Display of a single 4 beat Sequence.. . . 99
8.13 Display of a single 5 beat Sequence.. . . 100
8.14 Display of Multiple Choices for a 5 beat Sequence. . . 100
8.15 Display of Multiple Choices for a 7 beat Sequence. . . 101
B.1 Double Hand Gestures.. . . 106
B.2 Single Hand Gestures. . . 106
B.3 The Hand Mudras with Code. . . 107
C.3 The Head and Neck Positions.. . . 109
List of Tables
3.1 Initial Codes Designed . . . 21
3.2 The dance movement details . . . 22
3.3 Final Codes . . . 24
3.4 Final Codes with Index position of Vector . . . 24
3.5 Head Orientation . . . 25
3.6 Waist Orientation . . . 25
3.7 Hand Orientation . . . 26
3.8 Leg Orientation . . . 28
6.1 A Sample of Different Expert Ratings for the Same pose . . . 63
6.2 Comparison of Accuracy: RSES v/s WEKA . . . 64
7.1 FD ofAdavusCalculated . . . 74
7.2 A Sample for FD calculated for three expert ratings . . . 76
7.3 Overlapping FD for the averaged expert ratings . . . 77
7.4 FD for pics liked by more than 50% experts . . . 77
7.5 Statistically Calculated FD . . . 80
7.6 Statistically estimated FD for the BN poses . . . 80
7.7 Confusion Matrix for Expert 1 . . . 81
7.8 Confusion Matrix for Expert 2 . . . 81
7.9 Confusion Matrix for Expert 3 . . . 81
7.10 Confusion Matrix for Expert1 with Total . . . 82
7.11 Confusion Matrix for Expert2 with Total . . . 82
7.12 Confusion Matrix for Expert 3 . . . 82
7.13 Parameters of "Good" Classification . . . 82
8.1 The chart for possible handmudrafor next beat . . . 95
List of Algorithms
1 Algorithm for Single beat Choreography. . . 40
2 N-Beat Generation Algorithm . . . 86
3 Algorithm for Hand mudra Filter . . . 89
4 Algorithm for Leg Filter . . . 91
5 Algorithm for Vector Difference Calculation. . . 92
6 Algorithm for Limb Difference Calculation . . . 93
7 Algorithm for Calculating the Next Dance Pose. . . 94
List of Abbreviations
Absolute Vector Difference AVD Absolute Difference AD Bartenieff Fundamentals BF
BharataNatyam BN
Dance Position DP
Fitness Function FF Fitness Function Value ffv Fractal Dimension FD Genetic Algorithm GA
ICD Indian Classical Dance
Limb Difference LD
Limb Variation Count LVC
Natyashastra NS
Normal Distribution ND
Rough Set Theory RST
RSES Rough Set Exploration System
Vector Difference VD
WEKA Waikato Environment for Knowledge Analysis
Dedicated to my mother late Mrs Biva Chakrabarti.
Maa, here is your dream come true. . .
Chapter 1
Introduction
Dance Choreography is a unique technique in which dance moves are plan-ned and executed to perfection. Dr. Maya Angelou, the American poet, memoirist, and civil rights activist who has received more than 50 honorary degrees said that "You can’t use up creativity, the more you use, the more you have." A choreographer faces several challenges of how to make his choreography unusual by breaking the typical movement mold, introducing enough varieties, avoiding repetitions and checking for seamless transitions from one movement to another. He has to not only execute his unusual moves but also remember them in a set order. He must be able to convey his creative ideas either with the help of some existing notations (in which he has to be well-versed) or possibly with an easier tool like a computer. This is the age of Information Technology (I.T) and it has been used successfully in all aspects of life. I.T provides innovative solutions without compromising on the comfort. This research work has focused on using the power of I.T to address some of the challenges faced by a choreographer.
The term dance technology refers to application of modern I.T in ac- tivities related to dance like dance education, choreography, performance, and research (Fügedi, 1998). The use of computers in choreographic creativity is less heard of. We have attempted to use the power of I.T to aid the choreographer with new innovative suggestions for a dance move.
Based on these poses, he can generate Multi-beat choreography. Thus a choreographer can be suggested with new creative ideas. S(h)e will be able to get a choice ofvariouspossible moves for thesamestarting dance pose and then choose and try out these sequences accordingly. We have experimented with several methods and algorithms for the Indian Classi- cal dance, BharataNatyam. Using the data representation technique for a dance pose and the algorithm designed for the choreographic assistance for pure dance movements, a tool has been developed. The tool named as ArtToSMart (Art To System Modelled art) helps a choreographer in suggesting varieties of innovative movements for N beat dance sequences.
1.1 BharataNatyam
1.1.1 Introduction
BharataNatyam (BN), an Indian Classical Dance (ICD) is the most popular, ancient and authentic Indian dance form. BN is a solo
dance, with two aspects, lasya, the graceful feminine lines and move- ments, and tandava (the dance of Shiva), masculine aspect, which is identical to the Yin and Yang in the Chinese culture (Bharata Natyam http://community.worldlibrary.in/Articles). It has received universal acknowl- edgment as one of the subtlest expressions of Indian culture. BN literally can be broken into its syllables: Bha-bhava or expression, Ra- raga or melody, Ta- talaor rhythm and Natyammeaning dance. BN includes all forms of dances and dance - dramas as laid down by the sage Bharata in accordance with Natyashastra. Many research scholars agree that BN is a comprehensive word for various other classical dance forms of India like Kathakali, Manipuri, Kuchipudi and its likes (Kothari, 2007). Hence the choice for the oldest dance form of India, BN was done.
Sage Bharata’s Natyashastra (NS) is an encyclopedic work on the theater art and the oldest text on the subject in the world. It is believed to have been written during the period between 200 BCE and 200 CE and consists of thirty-six chapters written in Sanskrit language and from the style and syntax resembles Vedic literature. It’s origin is shrouded in the yuga puzzle by many scholars and Dr. Padma Subrahmanyam, an eminent dance research scholar and an expert has enough reasons to believe that the present text ofNatyashastrabelongs to the Pre-epic period (Subrahmanyam,2003). TheNatyaShastra, as written byBharata Muni, does not mention the names of any classical dance forms recognized today, but it lists the fourPravrittis(trends) asDakshinatya(Southern),Audramagadhi, Avanti, and Panchali. Bharatanatyam, Kuchipudi, and Mohiniyattam evolved from thePravritti form calledDakshinatya(Indian Classical Dance http://community.worldlibrary.in/Articles). Thus we can say that BN, KuchipudiandMohiniyattamhave similar dancing styles.
This traditional dance form is passed on by the “guru” (dance teacher) to the “shishya” (disciple) in a traditional method of rote learning. In this method, memorization of the dance steps is done with the help of regular practice. Since its glorious past, this tradition has undergone several changes and unlike the Western dances, doesn’t have a notation system of its own. Many of the magnificent temples built in South India during the fourth century AD – twelfth century AD has sculptures and paintings which depict the dance poses from the NS. Along with the ancient temple carvings for notation, various research scholars have tried to attempt the same with the help of Western staff notation system (Annemette P. Karpen, 1990; Pandya, 2016). The details of these papers are explained in the coming chapters. With the absence of a standard dance notation for this ancient form of art, most of its well-renowned dance choreographies are bound to be extinct very soon.
1.1.2 Structure of BN
The three main aspects of BharataNatyam are Natya, Nritya and Nritta.
In Natya, the dancer has to convincingly portray the situation through the facial expression and bodily actions and it is the art of acting out the meaning of a poem in a stylized form. It is the most advanced aspect of
Indian dance that requires long training, knowledge of Indian mythological and devotional background and lots of experience in order to capture the attention of the audience. Only a highly trained dancer with years of practice is capable of doing so. Nritya is the combination of Natya and Nrittaand makes it a perfect balance between the two aspects.
Nrittais purely technical dance based on music alone with no meaning;
the dancer interprets the rhythm through executing geometrical patterns in space. It can also be described as a series of poses connected by cer- tain pattern of movement. The unique technical aspect of BharataNatyam easily attracts viewers and does not require any background knowledge in order to appreciate it. Hence, Nritta is the most popular aspect of BharataNatyam for the audience as well as for the performer. The training in BharataNatyam starts from mastering the pure technique with the help of systematized dance movements or steps that are called“adavus”. These adavusare a combination of various poses. A pose uses major limbs of the body e.g. head, hands, waist and legs. All combinations of these different poses and movement sequences are used to form a dance item. Thus we have modeled these major limbs of the body to define a pose. The 4 most popular schools of BN are Mellatur, Padanallur, Vazhuvoor and Kalakshetra.
We have used the Kalakshetrastyle of BN dance which is a simplified ver- sion of thePadanallurstyle. It is the most beautiful of all styles of BN. Also it has to be noticed that theadavusare common amongst all these schools of dance and have minor variations.
1.2 Motivation and Research Objective
ICD has been the slowest to adopt technology. Although choreography is mainly a domain of creativity, computers can help a lot to ease this process especially for rhythmically oriented, aesthetically pleasing “Nritta”. Since these movements are geometrical patterns and do not convey either any meaning or any expression, its easy to experiment for computational creativity. This work will be useful to enhance the learning/teaching experience of BharataNatyam (and other dance forms). It can also be used for animated choreography of newly discovered pure dance movements.
Treatises like NS codify all the constraints and rules in a human under- standable form. This research attempts to model these in a computational framework, using practical experience and existing literature. This would help evolving better teaching programs, better understanding of the dance form, enhanced ability to compare one dance form to others in India and abroad, use ICT in composing and designing dance programs, and so on (Jadhav and Sasikumar, 2010). Enhancing teaching skills with variety in choreographic pattern is a major objective of this research. The system acts as an enhancer to the creativity within and the thinking process. It does not by any chance replace the task of a choreographer but acts as a catalyst to the thinking process by suggesting innovative moves which the choreogra- pher may not have even seen or thought of.
1.3 Contributions
The following are the main contributions from this research whose ob- jective was use of I.T to suggest innovative and aesthetically pleasing BharataNatyam poses for pure dance movements,Nritta:
1. Dance Pose representation through a computational model.
2. Designing an Evolutionary algorithm for suggesting new poses.
3. A Stick figure generation Module to visualize the generated poses.
4. Automatic classification of the generated dance poses.
5. Use of Fractal Dimension as one of the aesthetic parameter for these generated dance poses.
6. Application of Rough Set Theory for dimension reduction from 30 attributes to 8 attributes.
7. Algorithm for generating N-Beat Sequences using several filters.
8. An easy interactive tool which we called as ArtToSMart (Art To System Modeled art) to aid the choreographic process by incorpo- rating all the algorithms.
1.4 Thesis Organization
The rest of the thesis is organized in the following way:
• Chapter 1 introduces the concept of dance technology and the moti- vation behind the research.
• Chapter 2 deals with all the related work like dance notation and choreography, various dance capturing techniques, processing these captured data, dance generation and so on.
• Chapter 3 mentions the data capture and modeling technique used by us and why we did not capture the existing data through various other methods like kinect camera and sensors.
• Chapter 4 details the generation of innovative poses for BN choreog- raphy through an evolutionary algorithm.
• Chapter 5 explains how we experimented with GNU Octave for visu- alizing the generated poses through Stick Figures.
• Chapter 6 is focused on the automatic Classification of the generated dance poses and also on how we reduced the dimension from 30 at- tributes to 8 attributes.
• Chapter 7 explains why we needed to evaluate the aesthetic quality of the poses and how Fractal Dimension was used as one of the measure of aesthetics.
• Chapter 8 deals with the final algorithm. It also explains theArtToS- Marttool in detail along with the results generated by it.
• Chapter 9 concludes the thesis and points towards the future trends and directions that are possible to work on.
Chapter 2
Related Work
This Chapter deals with all the research related to various techniques in Dance Capture, notation, automation, processing, animation and so on.
Since there are very few papers that have attempted automation in Indian Classical Dance, hence we have included all other styles of dances like Bal- let and traditional Japanese dances like Soran Bushi, Noh and Butoh too.
To make it simpler to read, we have used the bold font forBNpapers and Italics forOdissi,Kathakand other Indian Classical Dances.
2.1 Dance notation and choreography
Automating dance choreography and capturing the movements have been done previously. Dance Representation can be done using existing notation called as Labanotation which is a notation system for recording and analyzing human movement that was derived from the work of Rudolf Laban (Wikipedia, 2016b). A lot of research work can be noticed dedicated especially in the area of dance notation and its standardization especially, Labanotation followed by Latin American dance styles and then Indian Classical dances. Latin American dance styles like Ballroom, Foxtrot, Waltz has been used successfully for several research attempts while Indian classical dances like BharataNatyam, Kutchipudi, Oddisi, etc.
have relatively fewer research attempts. Most of the works mentioned in this section have experimented with Western Dances except for few.
Ebenreuter (Ebenreuter,2006) has attempted to design an interface for Western Dance to facilitate the exact documentation of dance notation while in her paper Karpen (Annemette P. Karpen, 1990) has tried to solve the problem of notation (refer Chapter 1) for BN by using Labanotation. She claims that it could not be completely successful due to the fine movements of BN which could not be captured by Labanotation. LabanDancer system developed by Calvert et al. (Calvert et al., 2005) helped to translate the recorded labanotation scores into 3-D human figure animations. Choreo- graphic process was enhanced by Nahrstedt et al. (Klara Nahrstedt et al., 2007) using a 3D tele-immersive (3DTI) room surrounded by multiple 3D digital camera and a remotely placed dancer with a remote 3DTI room in a joint virtual space. Laban Movement Analysis (LMA) provides a model for observation, description and notational system for human movements.
Implementation of the same in a computer has been done through Bayesian approach (J. Rett, J. Dias, and J.M Ahuactzin,2008) and also by 3DTI (Klara Nahrstedt et al.,2007). Choreographic Language Agent (CLA) (deLahunta, 2009) helped to bridge the gap between the notations, sketches, diagrams
and text done by the choreographer on a notebook and his thinking pro- cess. Thus a unique method was used to augment the thinking process of the choreographer. Based on Newton’s Law, Hseih et al. (Hsieh and Lu- ciani, 2005) generated a dynamic model according to dance verbs jump, flip, etc.
2.2 Dance Capturing
Various techniques have been used for capturing dance movements of different styles. The most common being motion capture techniques and use of sensors. Dance motion can be captured using multi-view vision camera or monocular vision camera. The obvious difference between usages of both is the additional cost factor and also the various dimensions of the object.
2.2.1 Using Sensors
Using 41 markers on dancers Qian et al. (Qian et al.,2004) have developed a real time gesture driven interactive system. This marker based motion capture systems were used to provide real-time marker positions in global space. Paradiso et al. (Paradiso et al., 2000) have used a sensor system along with a small microprocessor and wireless transmitter in a pair of dancing shoes to capture the various degrees of freedom for the feet which can be made applicable for a series of computer-augmented dance perfor- mance. Aylward et al. (Aylward and Paradiso,2006) have used a compact , wearable sensor system which enabled real time collective activity tracking for interactive dance. The dancer wore a wireless sensor at the wrists and ankles. Takasi et al. (Takahashi and Ueda, 2009) used a Motion Capture system called as Eva Hires which is an optical motion capture system introduced in the Hiroshima city university for synthesizing dance data automatically for a user given music, even if the user has no knowledge of music at all. A multimedia system (Barry et al.,2005) uses a novel motion classification scheme. This is a wearable computer which aims to achieve a 3-D dance style classification for Butoh dance. This is a contemporary dance improvising method originating in Japan.
2.2.2 Motion capture
2.2.2.1 With Monocular Vision
Chen et al. (Chen et al.,2005) have used a single camera without any mark- ers for Motion Capture to obtain 3D motion parameters of a human figure.
This tracking of human figure on video has used image silhouettes. Paul et al. (Paul, Sinha, and Mukerjee,1998) have used a single camera to metamor- phose the user into a virtual person who can be an off-site coach using low band-width joint motion data to permit real time animation. This metamor- phosis involved altering the appearance of the person in this case akathakali dancer since this dance style has elaborate costume and make-up which is
very time consuming. The user did not have to wear any hardware de- vice and the paper aimed at making gesture tracking simpler, cheaper and user-friendly. Using a single camera for motion capture of BharataNatyam (Mamania, Shaji, and Chandran, 2004), a semiautomatic method was de- veloped to track the body parts using skin color detection. The authors restricted their model up to upper body part only. A Kinect camera has been used to capture aKathakdance teacher’s movement. The student can use a vision tool called PoGest (Gupta and Goel,2014) to compare and get a score of their own movements. This analysis helped a student learn at their own pace without a teacher.
2.2.2.2 With Multi view Vision
Lapointe et al. (lapointe_choreogenetics) have used genetic algorithm for real time generation of human computer choreography through chore- ogenetics algorithm. Using motion capture with 8 cameras and 8 PCs, Nakazawa et al. (Nakazawa et al.,2002) have used motion analysis method for recognizing the structure of human dance motion and motion primi- tives for a Japanese folk dance “Soran Bushi”. Concatenating these primi- tives they have generated new dance motions using inverse kinematics and dynamic balancing techniques.
Brand et al. (Brand and Hertzmann,2000) have generated stylistic mo- tion sequences from motion patterns through motion capture sequences.
Data was captured by physical markers placed on human actors, over short interval of time, in motion capture studios. Qian et al. Qian et al.,2004have tackled motion capture’s marker occlusion problem by developing a real- time marker cleaning algorithm. Using 8 camera VICON systems for train- ing, the multimodal feedback engine produced visual and audio feedback to the performer.
2.3 Processing Captured Data
Capturing and modeling of all the dance movements correctly results in efficient processing too. Several attempts have been made by researchers in various ways to process these captured and modeled data. Some of them are Evolutionary approaches using Genetic algorithms (Nakazawa and Paezold-Ruehl,2009), (lapointe_choreogenetics), (Hagendoorn,2002) and Multi agent system (Bechon and Slotine,2012), (Dubbin and Stanley,2010) optimization Algorithms, Classification using Neural networks (Qian et al., 2004) and Support Vector Machines (Hagendoorn,2002), Image Processing for gesture recognition (Hariharan, Acharya, and Mitra, 2011), ( Bradley, 1998), Corpus Based (Barry et al., 2005) and using Multi-agent system (Bechon and Slotine,2012).
2.3.1 Dance Semantics
Several techniques have been applied to understand the semantics of dance data from annotation, ontology, dance grammar or dance verbs, graph to vector space. These techniques used to interpret dance semantics have
helped in the process of automating choreography.
2.3.2 Annotation
Puig et al. (Puig et al., 2010) have experimented with Thierry de Mey’s (a musician and filmmaker from Belgium) Project for gestural annotation of dance video recordings. The grammar used by Mey to identify the different choreographic moves within the show and the annotation was processed through IRI’s software. To each identified gesture of the dancers, corresponding moves of the observer’s two hands were recorded through a multi-touch sensitive surface. Cabral et al. (Cabral et al., 2011) have designed a video annotator for tablet PC using touch input interfaces.
This is used for contemporary dance as a creative tool by choreographer to improve his work during rehearsal, live performances for later review or sharing notes with performers which can in future be also used for general web-based archive. Mallik et al. (Mallik, Chaudhury, and Ghosh, 2011) have automatically annotated new instances of digital heritage constructed ontology for Indian Classical Dance BharataNatyam and Odissi to train multimedia data. E-dance project (Helen Bailey, Michelle Bachler, and Simon Buckingham, 2009) showed how grid-based hyper- media and semantic annotations were used for capturing and rendering the choreographic practices. For content based multimedia access, concept recognition using ontology is used by Malik et al. (Mallik, Chaudhury, and Ghosh,2010). The automatically annotated new instances enabled creation of semantic navigation environment in a cultural heritage repository.
Video annotation by Malik et al. (Mallik and Chaudhury, 2009) using the power of MOWL has provided an effective video browsing interface to the user through a Bayesian Network for Indian Classical Dance such as BharataNatyam, Odissi, Kutchipudi and Kathak including music perfor- mances like Hindustani and Carnatic music. 200 videos of about 10 to 15 minutes duration were used of all these Indian Classical Dances for experimentation.
2.3.3 Ontology
Mallik et al. (Mallik, Chaudhury, and Ghosh, 2010), (Mallik, Chaudhury, and Ghosh, 2011) have constructed ontology for Indian Classical Dance BharataNatyam and Odissi to train multimedia data and automatically annotate new instances of digital heritage. The domain knowledge has been encoded in ontology and has provided methods to co-relate this to the audio-visual recordings and other digital artifacts. An ontological frame- work for Indian Classical dance by Malik et al. (Mallik and Chaudhury, 2009) offered a robust ground for several multimedia search, retrieval and browsing applications. The system was self enhancing where ontology was refined from annotated data and data annotation was improved based on fresh, refined knowledge from the ontology.
2.3.4 Dance Grammar (Verbs)
A gestural annotation of dance video recordings to codify Mey’s (Puig et al., 2010) own movies and musical pieces in which he has elaborated a grammar of gestures. A multi touch sensitive surface records the observers’
two hands for each identified gesture which can help identifying all similar clips in the movie. Hsieh et al. (Hsieh and Luciani, 2005) have used Newton’s law for presenting a set of dynamic models according to some dance verbs for contemporary dance. This paper was done with an aim to assist in computer-aided choreography and overcome the difficulty of dynamic based animation. Bradford et al. (Bradford and Côté-Laurence, 1995) has described a program that uses Artificial Intelligence for dance.
An if-then rule driver approach was used which described motion to create a sequence of dance phrases and this was used to generate choreography for multiple dances. A complex set of rules are thus created, executed and evaluated for patterns of dance.
2.3.5 Graph
A graph based algorithm was proposed for reconstructing 3D model for BharataNatyam dance from tracked data. The authors Mamania et al.
(Mamania, Shaji, and Chandran, 2004) proposed to visualize a pose as a node and a transition between two poses as an edge between corre- sponding nodes. Sugathan et al. (Sugathan and R, 2014) has extracted features from an attribute relation graph for the upper body poses of basic Bharatanatyam steps which can be useful for classification and annotation of dance poses.
2.4 Dance generation
Dance generation using computer system has been attempted by several researchers and some have been successful in fully or partially automating the process. We enlist a few methods of automated dance choreography by generating dance steps or poses through 3D, using robot motion to gen- erate images. We have identified three major aspects of dance generation namely, animated dance steps, computer aided choreography and Image generation.
2.4.1 Animated dance steps
Nagata et al. (Nagata et al., 2004) obtained Latin Dance movements us- ing Motion Capture and attempted to extract the difference in the character of movement of experienced people (Latin people) as well as inexperienced (Japanese people) for Latin dance. Thus after the extraction of natural dance movements, animation was carried out to make use of the outcome. Using advanced 2D research techniques, they have confirmed a phase difference in the movement of shoulders and hips considered to be a characteristic movement of experienced Latin dancers. They claim that Japanese people are unfamiliar to a motion called ”isolation”. Wilke et al. (Wilke et al.,2005)
have developed the LabanDancer to translate Labanotation dance scores into 3-D human figure animation for a wide variety of different movements.
The main reason being fewer dancers can read notation and even fewer are capable of producing scores. This can be a teaching tool for dance chore- ographers and students. Animation was done using Life Forms software by Sukel et al. (Sukel et al.,2003) and all the participants performed three formal Ballet movements in same order and they filled out a demographic sheet The authors felt that computer animation provided a better joint seg- mentation thus improving learning as compared to videotapes. Mazum- dar et al. (Majumdar and Dinesan,2012) have generated a library of body movements based on rules ofBharataNatyamand human body constraints which are further used to generate the dance steps for pure dance move- ments.
2.4.2 Computer aided choreography
Computer aided choreography is further sub categorized into fully auto- mated and semi automated. Computer aided choreography is discussed in following subsections.
2.4.2.1 Fully automated
Hagendoorn has transformed a dance studio (Hagendoorn,2002) into a lab- oratory for studying complex systems for dance choreography. He has tried to find different equivalence relations and classes as they apply to dance so that fascinating patterns emerge within a group of dancers. He has for- mulated a set of what he called as nature inspired rules that determined the interaction among a group of dancers. The system results in patterns that indicate how dancers should move on stage. Although the focus of the work is limited to movement of dancers on stage, the system generates fully automated choreography. Nakazawa et al. (Nakazawa and Paezold- Ruehl, 2009) have used genetic algorithms for a fully automated system of waltz choreography. They used mutation and crossover for exploring possible solutions to obtain a global optimum. The system generated satis- factory results by using majority of the stage, keeping partners facing each other and dancers on stage. The authors claimed that the system resulted within 10 % of the optimal choreography. Using the Choreogenetics algo- rithm (Lapointe,2005), choreographic variants were obtained for five basic movements. These were selected based on aesthetic criteria. Lapointe et al. (Lapointe and Epoque,2005) have shown that the best mutants closely match with the virtual dancer and thus the duets generated by the algo- rithm are not entirely random. Takahashi et al. (Takahashi and Ueda,2009) proposed a dance synthesis system that utilizes the motion capture data with Computer Graphics software Maya according to the impressions of music. Yu et al. (Yu and Johnson, 2003) tried to demonstrate the worka- bility and usefulness of computer generated choreography by using swarm toolkit and Life forms software with multi-agent system. The swarm toolkit was used to generate a sequence of dance steps which was later animated and expert evaluation is sought.
2.4.2.2 Semi-automated
Stuart et al. (Bradley, 1998) have developed and used corpora of human movements comprising of ten Balanchine ballets to select a movement se- quence that would naturally occur between a given pair of body postures.
They have applied techniques from graph theory, Artificial Intelligence and statistics to the above corpus of movement sequence. Interpolation meth- ods are described in this paper to automatically construct interpolation se- quence that suggest moves from one specified body posture to another in a physically and stylistically coherent fashion. Curtis et al. (Curtis et al., 2011) has designed a system that could be trained to learn dance move- ments through visual and haptic cues. With human assistance the robot could learn and perform dance steps. Bradford et al. (Bradford and Côté- Laurence, 1995) utilized rule driver embodying a heuristic algorithm for choreography of dance. They have designed a system CorX which may prove valuable to choreographers as an aid to the creative process but many of the details of interpretation are left to the human choreographers.
2.4.3 Image generation
Pattanaik (Pattanaik, 1989) has tried animating a few BharataNatyam karanas using stick figure model initially and finally a stylized volumetric model was used which could convey the position of the body efficiently and correctly.
2.5 Dance Processing Approaches
Capturing and modeling of all the dance poses effectively ensures efficient processing to obtain automated dance movements. Several attempts have been made by researchers in various ways to process these captured and modeled data. These approaches and corresponding research work are ex- plained in later subsections. The approaches discussed in this Section are as follows: Evolutionary Programming using Genetic, Flock and Ant op- timization Algorithms; Classification using Neural networks and Support Vector Machines; Image Processing for gesture recognition; Graph based algorithms; Corpus Based and Multi agent system.
2.5.1 Evolutionary Approach
This approach works on the powerful principle of evolution i.e. survival of the fittest (Dipankar and Zbigniew, 1997) which models natural phenom- ena like genetic inheritance and Darwinian strife for survival using heuris- tic search.
2.5.1.1 Genetic Algorithms
A Genetic Algorithm based paper (Lapointe,2005) that generates aesthetic choreography through mutations and selection and a new algorithm is ap- plied to simulate the evolution of a sequence of dance movements. Us- ing GA to create human computer choreography for real-time performance
environments, Lapointe et al. (Lapointe and Epoque, 2005) have created human computer duet by using motion capture technique on actual per- formers and coded the same with LIFE animation software to create a vir- tual vocabulary of four movements: run, jump, turn and fall. Nakazawa et al. (Nakazawa and Paezold-Ruehl, 2009) have used genetic algorithms for a fully automated system of waltz choreography. The fitness function was designed with respect to following equally weighted factors like the couple’s position, use of stage, step sequence, closeness to ideal steps, mea- surement of stage and so on.
2.5.1.2 Flock
Hagendoorn (Hagendoorn,2002) has used flock technique for generation of enticing patterns of dance. He says that within a flock only nearest neigh- bors are visible and hence it is used for self- reinforcing of dance patterns.
Rules governing the behavioral pattern of agents are listed out and most of the rules are inspired by the flock of birds or swarm behavior.
2.5.2 Classification
Classification is a technique where the user knows ahead how classes are defined. It is necessary that each record in the data-set, used to build the classifier. already have a value for the attribute used to define classes.
Dance Processing can also be done using various classification techniques like Neural Networks and Support Vector Machines.
2.5.2.1 Neural Networks
Dubbin et al. (Dubbin and Stanley, 2010) presented a program that takes advantage of interactive evolutionary computation and Artificial Neural Networks to train virtual humans to learn to dance. The dancers were con- trolled by ANN. They have efficiently solved the problem to parse sound in a way that ANN could interpret it.
2.5.2.2 Support Vector Machine(SVM)
Using k-means clustering algorithm Mallik et al. (Mallik, Chaudhury, and Ghosh, 2010) have trained an SVM classifier for classifying the media pat- terns. A training set of video segments were labeled by domain experts which helped in creating the multimedia enriched ontology and use of ma- chine Learning algorithms re validated the same with the use of low level media features and SVM classification. This was further used to interpret the media features extracted from a larger collection of videos to classify them into semantic groups. Sharma (Sharma,2013) has used an action clas- sifier for the basicBharataNatyamdance moves called asadavususing Sup- port Vector Machine.
2.5.3 Graph Based algorithms
Bradley et al. (Bradley,1998) have designed interpolation algorithm for a Ballet dancer’s body postures. These movements remain consistent from one prescribed form to another. The use of transition graphs for capturing
transition probabilities for every body-joint and generating a small corpus for ballet sequences and finally interpolation sequences and applying A*
search has resulted in learning the grammar of dance. The graph-theoretic methods learn the grammar of joint movements in a given corpus.
Feature extraction was done by Hariharan et al. (Hariharan, Acharya, and Mitra,2011) by generating a feature vector for distinguishing between different gestures of aBharataNatyamdancer’s single hand gestures. The silhouette was extracted followed by the generation of the corresponding skeleton and the evaluation of the gradients at its end points. Morpholog- ical operators are used to obtain a skeleton which corresponds to graph called as connectivity graph. Several such connectivity graphs for different hand gestures are shown. Sugathan et al. (Sugathan and R,2014) has pro- posed a graph based model for identifying and classifying the 2D poses of aBharataNatyamdancer’s upper body.
2.5.4 Image Processing through Gesture recognition
Quin et al. (Qian et al.,2004) have proposed a gesture recognition engine which provides real time recognition of the performer’s gesture, based on the 3D marker co-ordinates. Hariharan et al. (Hariharan, Acharya, and Mi- tra, 2011) developed a prototype for the recognition of the 28 single hand gestures ofBharataNatyamcalled as Asamyukta Hastas in a 2D space us- ing image processing techniques whereas Saha et al. (Saha et al.,2014) have used boundary of the hand gesture and texture based segmentation to sort out the flaws for recognition of the same single hand gestures. Emotions of the Indian Classical Dancer have been captured through the Kinect Camera by Saha et al. (Saha et al., 2013b) for creating a gesture recognition algo- rithm and in (Saha et al., 2013a) for automaticBharataNatyam hand ges- ture recognition. Sharma (Sharma,2013) has used the Kinect camera in his M.Tech. thesis, to capture and recognize the basicBharataNatyamadavus.
2.5.5 Corpus Based
Bradley et al. (Bradley, 1998) have developed a corpus- based interpola- tion algorithm for movement sequences to achieve a ballet dancer’s move from one body posture to another for computer animation. Bull (Bull,1996) has designed a corpus of Aerobic Dance Exercise routines for analysis us- ing linguistics techniques. He has named it ACCOLADE: A Computerized Corpus of Legal Aerobic Dance Exercises. Later on he used the real time animation of dancers with the NUDES system developed by University of Sydney. This was successfully applied to his Aerobics project.
2.5.6 Multi agent system
The prototype system proposed by Hagendoorn (Hagendoorn, 2002) is based on the concept of modern multiagent system. This term (multiagent system) is not explicitly used by the author but each dancer is assumed to be an agent and determines his/her behavior by sensing the state of other agents (individual or as a group).
In Chapter 15 of the google e-book by (Adeoye,2015), Iqbal et.al have surveyed in detail about Augmented Reality (AR) towards education sector. They have specified the advantages of dance education through Kinect sensors and have shown the cyclic taxonomy of AR in the field of Dance especiallyOdissiandBNfor posture estimation.
2.6 Conclusion
In this chapter we have segregated various research papers for different aspects of dance. We began with dance notation and choreography techniques. This was followed by various techniques in dance capturing with the help of sensors and motion capture. The captured data was processed through various methods and this lead to the dance generation through animation, computer aided choreography and image generating techniques. The dance processing approaches were further segregated into evolutionary approach, classification techniques and graph based algorithms. Finally we have also surveyed the papers image processing through gesture based and corpus based approaches followed by multi- agent systems.
Our system differs from all the existing work since it is focused in the area of Indian Classical dance, BharataNatyam and this system is designed to generate novel BN steps as a choreographic tool. Till date this work has not been approached by anyone, to the best of our knowledge.
Chapter 3
Dance Pose Representation
In this chapter we have discussed the method of representing our data. It demanded a detailed study of the ancient dance scripturesNatyashastraand AbhinayaDarpana. We also involved several dance experts from Goa and other states for the need to understand and represent the major limbs of the human body, so as to generate various combinations permissible within the dance framework. The following subsections explain as to why we didn’t choose Labanotation (a notation system for recording and analyzing hu- man movement that was derived from the work of Rudolf Laban in 1928 (Wikipedia,2016b) or video capture methods for the same and how we ar- rived at our method of modeling the body.
3.1 Labanotation
ICD especially BN does not follow any notation system and usually its learnt from the teacher to the student under the guru-shishya parampara i.e. teacher-student traditional learning system through regular practice.
Students use non-standardized stick figures notation to depict and remem- ber the sequence of choreography as shown in Figure 3.1. The obvious dis-advantage to this system are many which was found through our experiments in (Jadhav et al.,2014). For example the projection of 3D poses onto 2D always created a loss of dimension. The intricate and finer hand movements of BN called asmudraswere lost. The finer expressions of eyes or lips could not be captured through stick figures and hence the same dance could be interpreted in different ways by different dance teachers.
3.1.1 Basics of Labanotation
Western dances like Classical Ballet, Ballroom ( Tango, Cha-Cha and Two- step) and Folk Dances (Cherkessia, Israeli Dance) follow Labanotation, see Figure3.2afor a basic cha-cha movement. It is difficult to notate a dance
FIGURE3.1: A Hand-drawn Stick figure.
(a) A Basic Cha-cha movement.
(b) Direction symbols.
(c) Parts of the body.
(d) Staff symbols.
FIGURE3.2: Labanotation Symbols
movement through this notation until and unless one has gone through rigorous training as can be seen in the notation system from the Figures 3.2b, 3.2c, 3.2d. The technical standards and education for Labanotation are provided by several organizations. For example, the International Council of Kinetography Laban / Labanotation promotes standards and development for Labanotation (Wikipedia, 2016b). Also the Department of Dance, Ohio State University has designed a website to teach the basics of Labanotation (Dance, 2014). Several websites are dedicated to teach Labanotation.
This is a Dance Notation system derived from the work of (Austrian- Hungarian) Rudolf von Laban (1879-1958) and further developed by noted dance notation historian, Ann Hutchinson Guest and others. It is a sys- tem of analyzing and recording of human movement (Wikipedia, 2016b).
Labanotation uses abstract symbols to define the:
• Direction and level of the movement
• Part of the body doing the movement
• Duration of the movement
• Dynamic quality of the movement
3.1.2 Data Representation, Editing and Animation
When Labanotation was designed, there was no intention of computeri- zation. Several researchers have worked towards representation, editing and animation of Labanotation. One of the oldest papers date back to 1978 (Smoliar, 1978) where Stephen W. Smoliar from the University of Pennsylvania developed a program to compile the Labanotation move- ments into graphic simulation of human movement. He has used ordered pair to represent Direction (DIRECTION, direction-indication) and Level- indication (HIGH, MIDDLE or LOW). The data structure was represented in a page as variable number of staves and floor plans. Each staff was associated with 17 columns of objects.
Life Forms (1996), the animation program was developed by Dr. Tom Calvert (Calvert and Mah, 1996) from the Simon Fraser University stem- ming from an earlier work in 1977. It had begun as a visualization tool for generating choreographic ideas in 1993. LabanWriter was developed by the Department of Dance at Ohio State University by Dr. Tom Calvert (Calvert et al.,1991). This system was only supported on a Macintosh platform and suitable for preparing Labanotation scores and recording them in digital form. LabanDancer (Wilke et al.,1932) was a tool for animating the inputs from LabanWriter. Today Life Forms is being commercially marketed by Credo Interactive, a Vancouver- based animation company. The dance figures can be manipulated through key framing techniques and also uses motion capture inputs.
Some researchers from Japan, Kojima et al. in 2002 (Kojima, Hachimura, and Nakamura, 2002) and Nakamura et al. (Nakamura and Hachimura,
2002) have developed LabanEditor which helped to input / edit the Laban- otation scores and display character animation. Choensawat et al. in 2010 (Choensawat et al., 2010) had introduced a dynamic template technique for notating Japanese traditional performing arts called as Noh by using fundamentals of Labanotation. Later they have generated a tool called as GenLaban in 2015 (Choensawat, Nakamura, and Hachimura, 2015) for automation of Labanotation scores from Motion Capture data. They claim that their work is more comprehensive than others due to use of whole body posture analysis for generating the Labanotation scores. However they clearly stated that the Labanotation symbols are being continually introduced and hence will be incorporated in the tool as time permits, indicating that the entire symbols have not been coded yet. In the year 2005 Chinese researchers X. Shen et al. (Shen et al., 2005) removed few problems with LabanEditor about the quantization of front direction in an animation sequence and introduced new ideas. Thus in 2006 (Nakamura and Hachimura,2006), Nakamura and Hachimura introduced LabanXML which was an XML representation of the Labanotation.
3.1.3 Use of Labanotation for BharataNatyam
Annemette P. Karpen, a BN dancer of Russian origin has attempted in her paper at the 11th European Conference of South Asian Studies (Annemette P. Karpen, 1990) to use Labanotation, for BN. She concluded that the finer hand gestures couldn’t be captured successfully with the same. So she devised her own method of notation for BN. Ami Pandya, from the Department of Dance, Maharaja Sayajirao University of Baroda, in her Doctoral Thesis (Pandya,2016) has also used Labanotation for BN. She has added several columns on to the original Staff of Labanotation and success- fully notated with this system for Nritta Choreography especiallyAdavus to the minutest detail. She adopted the Laban fundamentals to suit BN movements for e.g the high, medium, low symbols were used for standing, half-sitting and sitting movement of BN. Divya Venkatesh (Venkatesh, 2016) has presented a graphic design project in her experimental work on BN notations through comparing and contrasting on three different notation systems: Benesh System, Beauchamp-Feuillet and Labanotation.
Since Labanotation was extensively used by several researchers earlier, we ruled out modeling the body with Labanotation .
3.2 Capture Data through videos or Kinect Camera
Few researchers have worked in the area of capturing BN dance through Kinect camera. Sharma (Sharma, 2013) has used an action classifier for adavus using SVM. Kale et al. (Kale and Patil, 2015) have used the kinect camera to recognize 5 adavusof BN which would help in recognizing the meaning of dance and display it to the audience. We couldn’t use any of the above existing methods mainly due to one main reason. We didn’t want to use any of the existing choreography but generate altogether new steps and
suggest the same to the choreographer. This generation of steps also had to follow BN norms and be consistent enough to be accepted by dancers. Thus capturing dance which was already existing would not add any novelty to our research at all. We also wanted various techniques of pruning the un- wanted data and displaying those poses that were feasible and totally in- novative. Thus capturing or recording existing sequences would not help us in our innovative technique.
3.3 Modeling the Human Body for Pose Notation
We have attempted to model the human body for Indian Classical Dance in a totally different way. There aretwo assumptionswhich we followed.
These are as follows:
1. Movement of the limbs are represented with respect to the dancer.
2. Movement of the dancer is taken with respect to facing the audience which is the default position of the dancer.
After an extensive literature review of ancient dance books like NS, Ab- hinayaDarpana (AD), CDs/ DVDs, several discussions with dance experts from all over India and abroad and the author’s personal experience of BN;
we noted down all the position of major limbs. These included the head orientation, the hand movements along with the dancemudras forNritta, the waist position and the leg positions.
3.3.1 The initial stages of codification
This involved the use of codes from NS (Bharata, 1996) and AD (Ghosh, 2002, Ghosh,1975) which were in Sanskrit along with their orientations in English. For example following are the codes for Shiro Bheda (SB) (head movements) written in brackets. The pictures can be seen in AppendixB.
Shiro Bheda(SB) total - 9 movements.
Samam(sm) :straight
Udhvaahitham(ud) :head with eyes facing up aalolitham (al) :rotating the head
adhomukham(ad) :face is cast down
dhutam (d) :shake head from side to side kampitham(kam) :shaking head up and down paraavritham(prv) :looking to one side utkshiptham (ut) :semi circle movement
parivaahitham(pvh):shaking from one side to the other.
Similarly the Neck called as GreevaBheda(GB) and Eye movements called as DrishtiBheda(DB) were also constituting of 4 and 8 movements respectively. The Hand movements are divided into Double hand Ges- tures called as (Samyukta Hasta) (SH) and Single hand Gestures called as (Asamyukta Hasta) (AH) or mudras.The AH has 28 mudrasand SH has 23
mudras. The Leg movementsPada Bheda(PB) has total 39 movements in all.
Thus initially, we used the following Table3.1 for codifying the Body movements. Head (2 attributes), Neck(3 attributes), Hands( Double hand Gesture (Samyukta Hasta) and (6 attributes) each for Right and Left Hand) and Legs ((6 attributes) each for Right and Left Leg along with Strike and Jump).
SB Ori GB Hori Verti SH AH-R
Hori-R Verti-R El-O-R P-twist-R P-bend-R AH-L Hori-L Verti-L EL-O-L P-twist-L P-bend-L PB-R Hori-R Verti-R
PB-L Hori-L Verti-L Str Jump
TABLE3.1: Initial Codes Designed
Ori/O: Orientation, Hori: Horizontal (if orientation is in front or back of the body),Verti: Vertical (if orientation is along the body i.e. not in front and back),R:Right,L:Left ,El:Elbow,P:Palm,Str:Striking or tapping Leg.
These were 26 attributes in all and later we realized that a mixed data type of string and numeric will not give good results with (Waikato Envi- ronment for Knowledge Analysis) WEKA, the Machine Learning tool. The results of WEKA couldn’t be interpreted and also the combinations gener- ated with these attributes were very large. Although it was easier for us to decipher the positions but the system wasn’t able to handle these mixed data, due to the specific techniques used by WEKA. Thus we finally agreed on numeric data types for all the codes. Also the intricate Hand Gestures of BN, SH and AH were combined into Single Hand Gesture representation only. Most of the SH convey meaning (like shankha, chakra, shivlinga, etc.) however few of these gestures while being used forNrittacan be easily de- picted with the help of AH and hence they are not shown explicitly. For example Anjali Hasta(thenamaskarposition) which is a SH can be shown with the help of Pataka Hasta (which is a AH) of both hands at the chest level. Also our Dance Position Vector representation has each hand (right and left) modeled separately.
3.3.2 Modeling
We chose to fuse the Neck and Eye movements with the Head since these three work in unison and are related to each other always. The Head and Neck movements are detailed in the Appendix C. Only the circled ones are used by us and rest are discarded since we wanted movements for Nritta only. Eye movements are not considered since they are very fine movements used to express feelings like anger, happiness, etc. For Nritta we use the eye movements as per theshlokainNatyasastra(Bharata, 1996) as follows:
“yato hasta tato drishti; yato drishti tato manah; yato manah tato bhava; yato bhava tato rasa” ||
Meaning: Where the hand is, eyes should follow there, where the eyes are, the mind should be there, where the mind is, there goes the expression,
