New Program Enables Computers to Learn to Read and Write Handwriting

The majority of us take the "Three Rs" of education for granted, but within the field of artificial intelligence, computers have only shown an aptitude for ‘Rithmetic, something they were specifically designed to do to begin with. Conversely, Reading and ‘Riting have proven to be a difficult hurdle for AI researchers to overcome, as the sheer complexity of how the human brain recognizes, interprets and reproduces handwritten text is still largely a mystery. However, a team of researchers have developed a way to teach a computer how to both read and write, and to do so as quickly and efficiently as a human.

The researchers, from the Massachusetts Institute of Technology, New York University, and Canada’s University of Toronto, drew inspiration for their new program from a previous study on human recognition of handwriting. That study noted that people who were learning new characters from an unfamiliar alphabet tended to remember them as a series of pen strokes, rather than as a full character, as would be the case after the character is properly learned by the individual.

The researchers used this concept to design a new algorithm that allowed the computer to learn the new characters as pen strokes as well: traditionally, computers learn new characters based on recognition of the entire character, and this often requires thousands of examples for the computer to do so. The new program, however, was able to learn new characters by looking at only one example.

The new algorithm was also able to reproduce the handwritten characters, but as interpreted by the program, and to not simply copy what it had previously seen as an image. The program succeeded at this task, and was able to produce handwritten characters that were slightly unique each time, with the result that the researchers had trouble distinguishing between the computer’s handwriting, and that of a human’s.

"We think in some form, this corresponds to what the human mind does," says co-author professor Joshua Tenenbaum, from MIT’s Center for Brains, Minds and Machines.

The researchers hope this new technique will also create advances in other fields of AI research, such as speech recognition, and to move beyond concepts like handwriting, to include recognition of new objects and physical human gestures.