User Tools

Site Tools


public:evaluations

Evaluations

A2iA regularly evaluates its technology in international competitions. These competitions usually attract the best research teams and provide a state-of-the-art of the current technology.

Handwritten French recognition

The international evaluations for French handwriting recognition have been conducted on the Rimes database. This database is composed of free form handwritten letters.

Handwritten English recognition

The ICFHR 2014 Handwritten Text Recognition on the tranScriptorium Dataset (HTRtS) competition was organized on dataset consisting of a series of manuscripts written by the philosopher Jeremy Bentham (1748-1832) himself over a period of sixty years, as well as fair copies written by Bentham's secretarial staff.

  • A2iA achieved the lowest error rate with 8.6% Word Error Rate

The ICDAR 2015 Handwritten Text Recognition on the tranScriptorium Dataset (HTRtS-2015) competition was organized on an extended dataset compared to the ICFHR 2014 competition.

  • Unrestricted task: A2iA achieved the lowest error rate with 27.9% Word Error Rate
  • Restricted task: A2iA achieved the second lowest error rate with 31.6% Word Error Rate

Handwritten Arabic Recognition

Evaluations for Arabic handwriting recognition were first conducted on an isolated word database (IFN-ENIT) and then on a database containing realistic full handwritten pages for the OpenHaRT evaluations operated by the NIST.

Multi-lingual Recognition

The Maurdor campaign (2013) presented a challenge to recognize complex documents in French, English and Arabic with images comprising printed and/or handwritten text from a variety of types:

  • Blank or completed forms;
  • Printed, but also manually annotated business documents;
  • Private and handwritten correspondence sometimes with printed letterheads;
  • Printed, but also manually annotated business correspondence;
  • Other documents such as newspaper articles or blueprints, etc.

The campaign presented many modules involved in document processing. In the “Module 4: Writing recognition or transcription”, A2ia ranked first in all three languages.

Handwritten historical documents

Historical documents often contain handwritten information interesting for scholars or genealogists. Several campaigns have been organized to evaluate the readiness of the handwriting recognition technology on such documents.

Local Attribute Detection for Improving Handwriting Recognition

The ANDAR-AD-2016 competition was organized during the International Conference on Frontiers in Handwritten Recognition (ICFHR). The goal is to detect the presence (or absence) of certain attributes that are useful in transcribing historical documents. Examples of those attributes are: presence of content ink, is text machine-printed or handwritten, orientation of the text, if there are strike-through markers present, is the text illegible, etc.

public/evaluations.txt · Last modified: 2018/05/16 10:10 by messina