This paper examined whether hand movements are responsible for expression in a piano performance. A player played a chord 100 times each with 12 different performance expressions, which consisted of three articulations (tenuto, heavy staccato, and light staccato) and four-level dynamics. The landmarks’ coordinates of her right fingers, wrist, elbow, and shoulder estimated as she played the chord (12 × 100 times), judged by MediaPipe Pose and Hands, were used for machine learning training and testing. In the results for the learning model, the testing accuracy rate was 0.99. In each performance expression, F1-scores were 0.94–1.00. This suggested a relationship between performance expressions and hand movements. Moreover, when the player happened to play a different, unintended type of expression, her landmarks’ coordinates were close to those when she had aimed exactly to play that type of performance expression.
CITATION STYLE
Oshima, C., Takatsu, T., & Nakayama, K. (2024). Examining the Relationship Between Playing a Chord with Expressions and Hand Movements Using MediaPipe. In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) (Vol. 14691 LNCS, pp. 118–131). Springer Science and Business Media Deutschland GmbH. https://doi.org/10.1007/978-3-031-60125-5_8
Mendeley helps you to discover research relevant for your work.