Google Lens: will read the indecipherable handwriting of doctors
From today, the Mountain View house wants to attempt the impossible, read doctors' handwriting with Google Lens . For the uninitiated, this service combines different systems present in computer science, including the potential of artificial intelligence. The declared goal is “ to connect the physical world around you and the digital universe on your device”. Indeed Lens allows you to read text and identify objects both within images and from reality. A seemingly hidden function in our smartphone device, which can save your efforts and time. And who now wants to engage in deciphering the nightmare of every patient dealing with doctors: their handwriting.
The new challenge of Google Lens
The American company has announced that it is working on the project with the help of pharmacists. Always protagonists of memes and jokes, the latter are giving a hand to what for now is only a prototype. Therefore, through the application, which uses a camera, it will be possible to take pictures of the medical prescription. Goolge's system will then process the image to avoid possible errors deriving from incorrect medical treatments . But from Moutain View they are clear:
no decision will be made based only on the result of this technology
Google post India
The role of technicians, between health specialists and pharmacists themselves will still be fundamental for examining handwriting.
How Google Lens works
The google app came out five years ago, first supplied separately, then integrated into the Android Camera. Just frame an object to start the Google lens system; a system that is precisely able to carry out searches based on what is seen with the camera . The service also provides information by reading labels or written text. In fact, an example is reading the name and password of a WiFi network: having read the two elements, the lens traces the scanned network.
Not only that, the app is also able to calculate tips and split the bill on a menu. Or recommend items from the menu itself once recognized. It can even describe the steps of a recipe starting from the written text (perhaps through vocal synthesis).
An interesting feature is what makes it the Shazam of fashion, namely Style Match . In fact, this option works in a way that is as simple as it is surprising and effective. Once again, by framing an item of clothing, the Google lens system will provide information about it. Useful information such as the price or the shop where it is sold . Because in fact he is able to do his job even with an instagram image, the photo on a blog or magazine. And it can do it with furniture too! If it does not find the specific article, the app refers to a series of similar products.
Finally we come to what was previously announced and that is the Smart Text Selection. This feature allows you to copy-paste text taken with a camera from real or digital documents. Hence the idea of tackling the “medical handwriting” challenge . But how does Google Lens interpret information from the physical world through the use of a single lens? If you answered or thought about artificial intelligence, well you guessed it!
The AI behind the Google app
The Google Lens app makes heavy use of so-called CNNs or Convolutional Neural Networks . These form the backbone of many computer vision-based applications. Lens uses CNNs to detect blocks of coherent text such as columns or text in a style or a uniform color.Then, within each block, it uses the text alignment, language, and geometric relationship of the paragraphs to determine their final reading order.
A bit like the human brain then, which makes simplifications to allow us to recognize objects. And like the latter, CNNs divide their work into several steps, each one specializing in a task. A convolutional neural network therefore splits into an input block, the hidden layers and the output block that provides the result . To activate the hidden levels there are the activation functions (e.g. RELU) which allow the former to carry out the calculations.
As can be seen from the "direction" of the data, CNNs are feed foward networks, ie with forward flow; connections between nodes do not form loops, which differentiates them from recurrent networks. But CNNs are also different than the feed forward networks themselves. In fact, it is precisely the levels of convolution that characterize them .
Convolution layers are like “Zooms” within the networks themselves. That is, between one intermediate level and another, they extract information from the image by selecting a particular characteristic . This is thanks to the use of special "filters". Depending on the type of filter used, it is therefore possible to identify different things on the reference image. The outlines of the figures, the vertical lines, the horizontal lines, the diagonals, are all examples of what can be focused on.
Possible developments of this technology
One can therefore imagine the potential and methods of application of Google Lens tools in the medical field. In addition to the question of our doctor's handwriting, Google Lens could give important results in the diagnostic field.
But on the other hand, the use of such a technology could also arouse a lot of controversy and concern from the healthcare world. Once again, the word belongs not only to AI specialists but, as Google said, also to a full collaboration with doctors.
The article Google Lens: It will read doctors' indecipherable handwriting was written on: Tech CuE | Close-up Engineering .