Today, Microsoft announced a new collaboration with Be My Eyes to bring high-quality, disability representative data to help train AI systems. This work with Be My Eyes will help make Microsoft AI models more inclusive for the 340 million people in the world who are blind or have low vision and is the latest step in our commitment to build inclusive technology.
AI requires large amounts of data for training and utility but too often disability is underrepresented or incorrectly categorized in datasets. In Microsoft Research’s most recent paper on AI performance for describing images from blind or low vision individuals, disability objects, like a braille device, were included less frequently in popular large-scale image-text datasets, leading to recognizing those objects ~30 percent less accurately. This disability data desert can limit the utility of a technology, strengthen existing stereotypes and magnify bias. As we continue to discover the opportunities for AI and accessibility, inclusive data through these types of partnerships and community collaborations are paramount to building inclusive AI.
How we do this matters. Transparency and user control are the guiding principles for data privacy in this agreement. Be My Eyes will provide video data sets, including unique objects, lighting and framing that realistically represents the lived experience of the blind and low vision community. Personal information will be removed from metadata by Be My Eyes prior to sharing and the company continues to work transparently with its members to provide a clear process and options to opt out of data sharing. Microsoft will then use the data to improve the accuracy and precision of scene understanding and descriptions with the goal of increasing the utility of AI applications for the blind and low vision community.
Today’s news is the next chapter in our collaboration with Be My Eyes, which began in 2017 when their app was integrated into the Disability Answer Desk support experience to give customers more efficient technical support. Last year, Microsoft was the first to pilot their new ‘Be My AI’, with AI helping to solve technical support issues. Be My Eyes also recently developed a Windows app, available in the Microsoft Store. Collaborations like these allow us to improve representation and diversity of AI data and drive more authentic inclusion of disability in our technology.
Responsible AI is inclusive AI
In the last 18 months, we have been focused on understanding how generative AI can directly impact disabled people, but our history here goes back even further. We are principled in our approach to make sure we extend those benefits and mitigate potential harms of AI. Microsoft’s Responsible AI framework is how we do this. It includes fairness, reliability and safety, privacy and security, transparency, accountability, and inclusiveness including accessibility.
AI innovations can reduce cognitive load through summarization, automate meeting notes, and provide detailed image descriptions. These innovations have so many positive implications for neurodiverse, deaf, and blind communities and more, and with 1+ billion people on the planet who experience some type of disability, the disability data desert is a crucial issue to address. 
Speech is one example of where we have been investing, and since 2022, we are proud to work with the University of Illinois Urbana-Champaign (UIUC) along with Apple, Google, Meta, and Amazon on the Speech Accessibility Project. The UIUC works with nonprofit organizations to collect data to improve speech recognition for individuals with diverse speech patterns such as those with ALS. This data helps AI models learn a broader understanding of the diversity of speech and expands scenarios where this technology can bring benefits – including accelerating non-standard speech data accuracy in mainstream speech platforms, like Azure Speech. The project is enhancing AI systems for people with disabilities by integrating diverse data and perspectives into the development process.
Our commitment to inclusive AI
At Microsoft, we are committed to building inclusive AI that is representative of all who use it, while also protecting marginalized members of society from proliferated bias that could impact education, employment, and civic engagement. We believe accessible technology is a fundamental right that can unlock opportunities in every part of society, and generative AI is one of the most powerful tools we have today to deliver on that potential when designed inclusively and responsibly. We are also committed to transparency and providing robust data protection for every individual. Today’s news with Be My Eyes builds on these commitments, and we will continue to seek out partnerships across the industry and community to create a more inclusive future.
If you have questions or feedback on Accessibility at Microsoft, let us know at the Disability Answer Desk or try out the Bing AI-powered Ask Microsoft Accessibility tool.
To learn more about Accessibility at Microsoft: Our Accessibility Approach | Microsoft Accessibility
Read more on Be My Eyes privacy commitments.
