PhD offer (Sept. 2024): Semi-supervised learning of an intelligent tutorial system for e-education through the production of drawings/sketches

Supervisors:

Location: IRISA laboratory, Shadoc, Rennes

Keywords: Intelligent Tutorial Systems, Artificial Intelligence, interpretation of semi-structured handwritten documents, two-dimensional visual grammars, sketching, pedagogy, e-education, pen- based tablets

The Shadoc (formerly IntuiDoc) research team (https://www-shadoc.irisa.fr/) at IRISA is working on the analysis and recognition of handwriting and gestures on 2D surfaces: tablets and touch screens. We are particularly interested in the design of shape recognition engines and new uses for gesture interaction on tactile surfaces. The team is working on the development of innovative digital environments on stylus tablets for education and has piloted several recent projects on learning handwriting for primary classes [4], and on producing geometry diagrams for secondary school classes [3].

This PhD is part of the theme of societal issues with AI for education. It follows the research work carried out on the design of Intelligent Tutorial Systems (ITS) to aid learning through drawing. Our previous work has focused in particular on the pedagogical themes of aid for learning geometry in secondary schools and on anatomy diagrams in health courses. This work is based on studies that have shown that introducing scientific drawing activities into courses [5, 6] improves student learning performance. Intelligent tutorial systems make it possible to develop highly effective personalized learning strategies by automatically producing appropriate corrective or guiding feedback.

Intelligent tutorial systems [7, 8, 9] are the result of a combination of two fields: artificial intelligence and e-education. The principle behind their design is to model the expert knowledge that will enable the system to automatically analyze the learner’s actions. The analysis concerns both the recognition of semi-structured handwritten drawings and the analysis of the validity of the action in relation to the constraints of the problem (problem-solving protocol, drawing steps).

In this research work, we will explore a new challenge which consists of working on a module for the automated generation of expert rules (author mode) to understand the structural (and compositional) modelling of schemas. The ambition is to be able to take structured schemas (as for geometry) as well as semi-structured schemas, such as anatomy schemas, as input for learning the ITS to generate rules. By using semi-supervised learning to facilitate the creation of knowledge models of ITS, we will be able to extend their scope of application to other disciplines, for example to diagrams describing processes or scientific diagrams (chemistry, biology, physics, etc.).

Today, this knowledge is defined by rules made explicit empirically by an expert by formalizing them in a two-dimensional visual grammar (GMC-PC) [2]. The aim here is to extend the formalism so that new rules can be inferred dynamically by learning from a reference schema. The challenge will be to infer grammatical composition rules dynamically and semi-supervised from a teacher’s reference sketch, based on intelligent interaction with the teacher. This will automatically deduce the structural and compositional constraints to be incorporated into the rules so that the learner’s composition of the sketch can then be analyzed.

This structural grammatical knowledge will be combined with evolutionary AI systems for incremental recognition [1] capable of learning handwritten forms from few examples, which will make it possible to adapt to different styles of composition. We will also study the ability of fuzzy landscapes [10] to model and learn the spatial context in order to characterize the relative positions of schema elements.

For the evaluation phases of these intelligent tutorial systems and for measuring the impact of their use on the learning of pupils and students, this thesis work will be supported by collaboration with the Psychology: Cognition, Behaviour, Communication (LP3C) laboratory at the University of Rennes 2. In parallel with this thesis, we will experiment under the supervision of Professor Eric Jamet of the LP3C, in particular through the master’s courses in Ergonomics and Human Factors Psychology (EPFH).

Finally, these intelligent tutorial systems are intended (particularly for geometry and anatomy) to be transferred and deployed in colleges and universities in the medium term.

References:

  • Almaksour, A. & Anquetil, E. (2011), “Improving premise structure in evolving Takagi-Sugeno neuro-fuzzy classifiers”, Evolving Systems, vol. 2, no. 1, 25-33.
  • Macé, S. & Anquetil, E. (2009). Eager interpretation of on-line hand-drawn structured documents: The DALI methodology. Pattern Recognition, Volume 42, Issue 12, 3202-3214.
  • Krichen, O., Anquetil, E. & Girard, N. (2020). IntuiGeo: Interactive tutor for online geometry problems resolution on pen-based tablets. European Conference on Artificial Intelligence (ECAI), Santiago de Compostela, Spain, 1842 – 1849.
  • Simonnet, D., Anquetil, E. & Bouillon, M. (2017). Multi-Criteria Handwriting Quality Analysis with Online Fuzzy Models. Pattern Recognition, 69, 310-324.
  • Alsaid, B., & Bertrand, M. (2016). Students’ memorization of anatomy, influence of drawing. Morphologie, 100(328), 2-6.
  • Joewono, M., Karmaya, I. N. M., Wirata, G., Yuliana, Widianti, I. G. A., & Wardana, I. N. G. (2018). Drawing method can improve musculoskeletal anatomy comprehension in medical faculty student. Anatomy & Cell Biology, 51(1), 14-18.
  • Nkambou, R., Mizoguchi, R. & Bourdeau, J. (2010). Advances in Intelligent Tutoring Systems, t. 308.
  • Mitrovic, A. (2010). Modeling Domains and Students with Constraint-Based Modeling, in: Advances in Intelligent Tutoring Systems, edited by Roger Nkambou, Jacqueline Bourdeau and Riichiro Mizoguchi, Berlin, Heidelberg: Springer Berlin Heidelberg, 2010, 63-80.
  • Aleven, V. (2010). Rule-Based Cognitive Modeling for Intelligent Tutoring systems, in: Advances in Intelligent Tutoring Systems, sous. sous ladir. de Roger Nkambou, Jacqueline Bourdeau et Riichiro Mizoguchi, Berlin, Heidelberg: Springer.
  • Adrien Delaye, Eric Anquetil. Learning of fuzzy spatial relations between handwritten patterns. International Journal of Data Mining, Modelling and Management, 2014, 6 (2).

Comments are closed.