Handwriting Recognition

Handwriting recognition (HWR) is the ability of computers to recognize handwritten texts on any medium such as paper, photographs, documents etc. This is also often referred to as Handwriting Text Recognition(HTR). Optical Character Recognition(OCR) is the electronic conversion of handwritten of typed texts whether from scanned documents or photographs. HWR is a new research field of OCR, which many researchers and tech giants are trying to address. According to reports published by marketsandmarkets in 2021 [1], the global market size of OCR handwriting recognition was estimated to be USD 1,039.3 Million in 2016. It has an expected CAGR of 15.7% from 2017 to 2025. HWR market is expected to grow along with it and also show tremendous improvements on the current accuracy levels.

Handwriting recognition has a large number of industrial use cases from health care industry and pharmaceuticals to banking and insurance. HWR technologies can be employed for various reasons like – reducing labor costs, saving time invested on manually digitizing handwritten records, enhancing the user experience of customers, etc. HWR may also lead to automation of various labor intensive processes.

The varied handwriting style, lighting of an image, separation of text in cursive handwriting have made handwriting recognition a difficult problem to achieve good accuracy. However, the ongoing research which deploys state of art deep learning architectures have made considerable strides in improving accuracy. These prove to be far superior than earlier used machine learning algorithms where features to train the machine learning model were human defined.

Current Status and Future Predictions

[1] According to a new market research report published by Credence Research “Handwriting Recognition (HWR) Market (By Type – Online and Offline; By Application: Automotive; Education and Literature; Enterprise and Field Services; Healthcare and Others) – Growth, Future Prospects, Competitive Analysis and Forecast 2017 – 2025”, the global handwriting recognition (HWR) market was valued at US$ 1,039.3 Mn in 2016 and is expected to grow at a CAGR of 15.7% during 2017 to 2025.

HWR technologies can be classfied into two types – online and offline methods. Online methods correspond to extracting machine readable texts from strokes on touch screens. Offline methods refer to extracting machine readable text from paper, journals etc.

Fig. Global Handwriting Recognition Market Revenue by type

The major players of this market face high competition. The key for surviving in this market is better efforts towards service enhancement so as to address the changing regulations and economic conditions in the global market. Along with it improving accuracy of HWR software is also necessary. The key players of handwriting recognition (HWR) system market include MyScript, Nuance Communications, Inc., SELVAS AI, Inc., Hanwang Technology Co., Ltd., Paragon Software Group, PhatWare Corporation, SinoVoice (Beijing Jietong Huasheng Technology Co. Ltd.), and Sciometrics, LLC.

A ton of apps for mobile, tablet and web platforms are also available for online methods. The table below is summarizing some of most popular handwriting recognition apps.

Table 1. Software for handwriting recognition

Major cloud services like AWS (Amazon Textract), Google Cloud (Google Vision) and Azure (Microsoft Azure Vision) also provide APIs for handwriting recognition.

Handwriting Recognition Use Cases

Healthcare industries

We have all heard jokes about doctors’ unrecognizable handwriting and also faced this issue first hand. Jokes apart, according to July 2006 report from the National Academies of Science’s Institute of Medicine (IOM) found doctors’ s unrecognizable handwriting kills more than 7,000 people annually. Using a HWR technology could easily save people’s time, money and lives. A lot of healthcare institutes and hospitals are implementing data strategies to combat loss of life arsing from illegible script. These include Electronic Health Records(EHR) which have been adopted by 71% physicians as claimed by a 2013 survey.

These technologies also help mitigate issues such as dependence on paper, regulatory violations, compliance issues, forgeries and frauds. HWR technologies can be further used to digitize patient entry forms and handwritten records. Digitizing will also give access to huge amount of data which can be used for data analytics to gain meaningful insights.

Education

There is an ongoing effort to introduce children and educators to computing devices early on. Google for educators is one such campaign which focused on schools to embrace technology. With the increasing use of tablets in schools and colleges HWR technology is bound to grow in education sector. This is a powerful tool which can enhance students learning experience. As an example student can make handwritten notes on their iPad which helps in better understanding. These notes can also be converted to computer readable text using handwriting recognition techniques. This technologies have been augmented in various note taking apps, apps to deal with complex mathematical sums and also on music apps. For example mathematical apps can convert handwritten questions and equations into neat computer readable digits and text which can be used to crunch desired answers. Other uses include turning scrawled diagrams into digital documents, music composition and even streamlining the process of adding references to research papers by including highlight and search capabilities. Thus, handwriting recognition technologies can benefit the educator, and students at all levels be it a kindergartner or a senior year college student.

Banking

Fig. Application of handwriting recognition in Banking Applications

Currently cheques deposited to banks are manually analyzed and then entered on the computer. Bank employees also have to manually verify the signature and date of the cheques. This takes time and manual effort also delays reflecting of balance on the benefited bank account. Handwriting recognition technologies can be used to read these cheques and other bank documents such as forms, demand drafts etc at a much faster pace.

Online Libraries

A large number of historical books and journals have been digitized to make it accessible to entire world. However, most of these efforts are restricted to photos or scans of books. This effort can become even more useful if the text on these historical books could be parsed and queried and indexed by web crawlers. Handwriting recognition plays a key role in bringing alive the medieval and 20th century documents, postcards, research studies etc.

Consumer

There has been an increasing demand for OCR and related technologies by consumers. This has fueled the adoption of such technologies in tablets and smartphones. The demand has surged owing to increase in number of devices to capitalize on. OCR technologies including handwriting recognition have been around for a decade. However the wide spread reach of touchscreen mobile devices have increased the adoption of such technologies in day to day life. HWR technologies might provide a viable alternative to use of digital keyboards in near future. Whether it is within mobile applications, leveraged in your smart watch, included as an alternative to your smartphone keyboard, or integrated within the dashboard of your car, HWR technology is gaining momentum.

Techniques used

Handwriting text recognition can be categorized in two categories – offline method and online method.

Handwriting recognition offline method and online method.

1. Online method – Online method involves touch screen devices, light pen, stylus etc. This method has access to information such as stroke and direction. They also don’t face the issue of noisy backgrounds. They can be recognized with pretty high accuracy as compared to offline methods.

2. Offline method – This method refers to text written down on a non digital media such as a paper.

In this, information such as stroke, directions are not available like in case of online methods. Apart from this method might also have issues of noisy background.

The ongoing efforts are more concentrated towards handwriting recognition in offline method which still has a long way to go before it achieves a respectable accuracy. Also offline method does not require extra devices such as light pens and stylus.

Early attempts at handwriting recognition utilized machine learning algorithms such as Hidden Markov Models(HMM), Support Vector Machines (SVM) etc. These techniques involved input of per-processed text, feature interaction to identify key elements such as loops, aspect ratio and inflection points. These generated features are then fed into machine learning algorithms such as HMM and SVM. Such methods had limited accuracy due to need of identifying features by humans. Also, this method is not scale-able as it cannot cover different languages.

Deep Learning solves this issue with use of state of art technologies together such as – Convolution Neural Networks (CNN), Recurrent Neural Networks (RNN) and Connectionist Temporal Classification [3].

Convolution Neural Networks are used for feature extraction from input images. The convolution layers contains three main operations – use of convolution (filter kernel) to the input image, a non-linear ReLU function, and finally a pooling layer to downsize the output of CNN. Recurrent Neural Network (RNN) is a class of artificial neural networks where connections between nodes form a directed graph along a temporal sequence. LSTM implementation of RNN is used as it facilitates long term propagation of information, thus making the trained model better. Finally Connectionist Temporal Classification (CTC) layer is used to classify the images. The CTC is given the RNN output matrix and the ground truth text as input using which it computes the loss value. The CTC gives the final text as output.

Fig. Overview of Neural Network Architecture for handwriting recognition

Limitations

One of the main struggles of HWR technologies is accuracy. A lot of HWR technologies cannot predict many handwriting making them unreliable to use in real life scenarios. They also require huge datasets and huge computing power to train models. As a result of this implementing HWR on industry level might be quite expensive. Apart from this, the recognizing capability of the model depends a lot on the data used for training the model.

Conclusion

The future of HWR is hopeful, with recent strides in accuracy level, mass adoption of mobile devices and a push for paperless operations. A ton of softwares are available for handwriting recognition with varied accuracies and price points.

Some popular handwriting recognition softwares are –

Amazon Textract
Microsoft Azure Vision
Google Cloud Vision API
Infrrd.ai
MicroBlink
MyScript

While utilizing handwriting recognition software for business, you should ponder on factors such as – character recognition accuracy, word recognition accuracy, computation speed in case results need to be delivered real-time, continuous learning capabilities, user-friendliness of the interface if the interface will be used by humans and the price point carefully.

HWR technologies will play are huge role in the upcoming years. So, is your company ready to reap its benefits?

Need help with Emotion recognition and detection software?

If you want to utilize Handwriting recognition technologies and need help, advice, or developers, feel free to contact us at contact@quantiantech.com or on our LinkedIn page or visit our company website, www.quantiantech.com

References

Leave a Reply Cancel reply

Services

Contact Us