Ph.D. of Kwon-Young Thomas Choi

Combination of unsupervised generative models and a syntactical method for music symbol detection with few annotated data

Defense on June 30th 2021

Abstract

In this work, we study the detection of music symbols in images of complex, historical, dense, noisy and damaged printed music scores through the use of Deep Learning detection models. We propose a comparative study of multiple state-of-the-art detection models applied to music symbols as well as a new architecture based on the Spatial Transformer for a very focused music symbol detection task. Although we explored an original detection approach with this new architecture, we obtain 94.81% of mAP while the best state-of- the-art method obtains a mAP of 98.73%. Since the use of Deep Learning methods requires a huge amount of annotated data, we present the Isolating-GAN, a novel unsupervised music symbol detection method based on Generative Adversarial Network (GAN). Using only isolated music symbols and no symbol level detection annotations, we build an encoder-decoder generative model able to filter and isolate music symbols from background noise and shapes and train the model using a hybrid adversarial and image reconstruction loss. Symbols isolated by the generative model are then detected using a small detector pre- trained using isolated symbols in empty white images. With this approach, we obtain a mAP of 82.5% for a detection task with three accidental classes. We also demonstrate that using the Isolating-GAN to filter and isolate symbols before the detection operation reduces the number of false positives from 2,696 to 57. The method was applied on 1,774 pages of historical music scores and was able to detect 38,908 new accidentals.

Jury composition:

  1. ÉGLIN Véronique, Professeur à l’INSA de Lyon
  2. PAQUET Thierry, Professeur à l’Université de Rouen et Normandie Université
  3. FORNÉS Alicia, Senior Research Fellow at Universitat Autònoma de Barcelona
  4. FUJINAGA Ichiro, Associate Professor at McGill University
  5. MOUCHÈRE Harold, Professeur à l’Université de Nantes
  6. COÜASNON Bertrand, HDR à l’INSA de Rennes
  7. RICQUEBOURG Yann, Maître de conférence à l’INSA de Rennes
  8. ZANIBBI Richard, Professor at Rochester Institute of Technology

Comments are closed.