Social media is an extremely visual platform, whether it’s Facebook, Instagram or Twitter. An estimated 1.8 billion photos are uploaded to these platforms every day. For people who are blind or partially sighted, this can be extremely difficult because they have no way of telling what an image is.
But that’s all beginning to change. On Tuesday (April 5), Facebook launched a service that uses artificial intelligence (AI) to decode the content of an image that can be read by a screenreader.
People who are blind or visually impaired use navigation software called screenreaders to be able to read the content on a computer. The screenreader turns what is on the screen into speech output or braille.
Facebook has trained its software to recognize around 80 objects and activities, with the software learning more and more.
The individual behind this amazing development is Matt King, an engineer who lost his eyesight due to a condition called retinitis pigmentosa which destroys light sensitive cells in the retina.
Using Facebook’s in-house recognition software, King and his team have trained the software to recognize items such as food and vehicles.
According to BBC, King says,”On Facebook, a lot of what happens is extremely visual…And, as somebody who’s blind, you can really feel like you’re left out of the conversation, like you’re on the outside.”
King hopes to break that obstacle by enabling those with impaired vision to be described the content of the images on social media. King and his team hope to eventually use the technology to also have facial recognition, but some resist this idea due to privacy. For King, it’s a matter of fairness. If sighted people are able to identify people in a photo without them being tagged, visually impaired people should have the same privilege.
Some of the objects the software can recognize and describe are:
Transport – car, boat, aeroplane, bicycle, train, road, motorcycle, bus
Environment – outdoor, mountain, tree, snow, sky, ocean, water, beach, wave, sun, grass
Sports – tennis, swimming, stadium, basketball, baseball, golf
Food – ice cream, sushi, pizza, dessert, coffee
Appearance – baby, eyeglasses, beard, smiling, jewelry, shoes – and selfie
To see how this AI technology works check out the video below:
All images courtesy Facebook