December 9, 2024 |

CCNY Develops AI Technology to Improve Accessibility and Disability Challenges

By Emily Blackburne

What if AI could empower a blind person by helping them safely cross the road on their own? The merging of artificial intelligence (AI) and accessibility technology is making large strides forward, with research at the City College of New York (CCNY) offering life-changing assistance and solutions for individuals with disabilities. At the forefront of this movement is CCNY and CUNYGraduate Center distinguished professor ��� a leader in AI-driven assistive technology. Tian also directs , made up mostly of CCNY���s Grove School of Engineering graduate and undergraduate students. The lab develops AI technology to improve the lives of people with disabilities, advancing practical applications like AI powered gadgets alongside groundbreaking scientific research.

Tian���s focus in computer vision���a field that uses collected image and video data to interpret environments��� is paired with applications that have real societal impact. Supported by the Department of Transportation and teaming up with other experts like CCNY professors and ��the team has developed wearable mobile devices with cameras built in to assist visually impaired individuals. Using simultaneous localization and mapping (SLAM) technology to plan paths and guide users, the device���s cameras collect visual data from the environment, surrounding object orientation, distance, and sudden environmental changes. The AI computer model then interprets this data and communicates it auditorily in real time to users.

The developed AI-based navigation and wayfinding device has successfully demonstrated at Department of Transportation a blind participant was instructed to go to a specific office located randomly in the building and she successfully navigated to that office from the first floor to the eighth floor, guided entirely by AI auditory feedback.

“It���s pretty cool to see our work making such a tangible difference,” Tian remarks.

��This technology has since been developed further to address specific challenges faced by blind or low-vision individuals, like reading visual signs, packaging labels, currency, or using AI gesture-detecting technology to interpret non-verbal body language cues, including subtle facial expressions. Going forward Tian hopes to incorporate in this AI interface, to make the process even more user friendly.

Collaborating with CUNYHunter and CUNYQueens College, the lab also addresses auditory and speech disability challenges using AI. CUNYGraduate Center Computer Science Ph.D. alumna Dr. Longlong Jing (now a researcher at Waymo) and (now a data scientist at Microsoft), applied AI technology to understand highly nuanced American Sign Language (ASL) in real time for those with auditory or speech impairments. “American Sign Language isn’t just about hand gestures���it involves facial expressions and body movement to convey questions or emotions,” Tian informs.

Funded by the National Science Foundation (NSF), the lab employs research participants and collects video and image data of them doing sign language gestures. This data is used by the team to program AI computer models to recognize and translate these gestures into speech.

The work is groundbreaking, but it also faces limitations. “No matter how advanced AI is, recognizing subtle human gestures remains a challenge,” Tian notes, highlighting the complexity and variability of human expressions and movements. Therefore, the team focused on technology to check grammar mistakes made by ASL learners and provide them with immediate feedback without recognizing each individual sign.

The lab���s research focus on accessibility needs reflects a broader ethical mission becoming clearer in the AI community��� creating tools that practically improve well-being and quality of life.

���Technology is just technology unless it meets the user’s needs,��� Tian says.

Her team also collaborates with leading disability organizations like and New York State Commission for the Blind, ensuring that their AI developments effectively prioritize the everyday challenges faced by those with disabilities, and is not confined to the lab.

In tandem with disability and accessibility improvements, Tian���s team also employs AI in broader healthcare practices, particularly medical imaging, focusing on radiology and cancer detection. Partnering with healthcare institutions like Memorial Sloan Kettering Cancer Center, the team utilizes AI to avoid unnecessary and costly testing for detection, such as those for renal and kidney cancers. Another focus explores ways to create virtual contrast images for CT scans, providing an alternative to the traditional approach that involves injecting dyes, which can cause allergic reactions in some patients. The alternative enhances accessibility in medical testing to accommodate individual needs.

Tian envisions AI as an inevitable and transformative force, similar to the advent of the internet.

���No matter how much people debate, it���s a tool we must embrace��地nd guide ethically,��� she stresses. “When used the right way, it can have a huge positive impact.”

For the research happening at CCNY���s Media lab, this impact gives those with disabilities the chance for a more autonomous and enriched life.