We build upon the fully convolutional masked autoender ConvNeXt-v2 to introduce the first line-level self-supervised framework for Handwritten Text Recognition. The standard masking strategy used for natural images is inadequate for text recognition needs so we propose a strategy dedicated to HTR. We further propose to model language during pretraining through a two-phase curriculum pretraining.
The code will be available soon on this page.