If the decision of local TV station GMA-7 to use AI sportscasters for its NCAA coverage has elicited a largely negative sentiment among the public, the reaction is quite different at the ongoing Asian Games in Hangzhou, China where an AI-powered virtual sign language interpreter is providing support to ensure individuals with hearing impairments could participate in the regional sports competition.
Alibaba Cloud, the digital technology and intelligence unit of tech giant Alibaba Group, developed the digital avatar named “Xiaomo” who serves as a sign language interpreter for the different sports events at the Games.
Xiaomo is expected to extra useful when she is deployed at the Asian Para Games to be held between October 22 to 2 also hosted by Hangzhou.
According to the World Health Organization (WHO), currently more than 1.5 billion people (nearly 20% of the global population) live with hearing loss.
Alibaba Cloud integrated Xiaomo into a mini-app on the payment platform Alipay, supporting two-way translation between sign language and Chinese spoken language.
Users can then input voice messages, which the virtual character will translate into sign language for hearing impaired individuals.
Conversely, the mini-app can interpret sign language into spoken language, facilitating seamless communication between the hearing and hearing-impaired individuals.
Sign language, with its unique blend of gestures, expressions, and body movements, has its own phonetics, vocabulary, and grammar that differ from spoken language.
To accommodate these complexities, Alibaba Cloud compiled a Chinese sign language translation dataset that contains 25,000 signs gathered and annotated from sign language practitioners and hearing-impaired people across Zhejiang province in China.
Alibaba Cloud leveraged its advanced visual recognition algorithm and its motion-tracking capabilities to capture the movements of the hearing impaired and translate sign language into natural language with a state-of-the-art language translation model.
To convert natural language into sign language, Alibaba Cloud also developed a deep neural network and a lip movement generation module, enabling the virtual character to perform sign language gestures accurately.
“Through the integration of natural language processing, computer vision and machine learning technologies, we have developed a sign language translation system. This innovation aims to make the Asian Para Games more inclusive and accessible to those with hearing loss,” said Matt Zhang, algorithm engineer at Alibaba Cloud.
Xiaomo was also incorporated into the news channel of the Asian Games to enhance the accessibility of its announcements.
During the Asian Para Games, Xiaomo can assist participants with hearing impairments in interacting with volunteers at the venues for various assistance, including asking for directions, seeking medical help, and getting assistance with game viewing.
At the same time, Alibaba Cloud also launched a sustainable lifestyle Web application that helps participants of the Asian Games adopt low-carbon behaviors and reduce their carbon footprint through digital engagement.
“Cloud-based technology helps drive the sustainable and inclusive development of the Asian Games. We want to leverage the power of technology, in collaboration with our partners, to drive more individuals to participate in our efforts to make the Games greener and more accessible to all,” Jingren Zhou, CTO of Alibaba Cloud Intelligence, said in a statement.
The interactive web application rewards athletes, journalists and staff members of the Asian Games with carbon points for making eco-friendly lifestyle choices, such as taking public transport, waste recycling, and reducing food waste.
Across the Asian Games villages, participants can scan a QR code to record their low-carbon activities in the Web application, ranging from opting out of using plastic bags at the supermarkets, to taking a photo of their clean plates at the dining hall in an effort to reduce food waste.
Behind the Web application is a system that translates individual low-carbon activities into carbon points. Participants can accumulate the points and redeem for limited-edition Asian Games-themed pins and low-carbon products.
The Web application has attracted over 310,000 visits to participate in low-carbon activities across the Asian Games villages, and it has recorded over 7 tons of carbon reduction through the activities since the opening of the Asian Games villages on September 16.
Alibaba Cloud also tapped its AI-powered carbon management tool Energy Expert to measure and optimize the carbon footprint for the manufacturing of the Hangzhou Asian Games’ mascots.
For example, according to the assessment from Energy Expert, one of the three mascots, Congcong, has a cradle-to-gate carbon footprint of 1.59 kg carbon dioxide equivalent (CO2e), with the emission from three major sources, including electricity use, packaging and raw materials.
By advising the factories to increase the use of solar power and adopt more eco-friendly designs, the sustainability platform helped slash the product’s carbon footprint by 0.15 kg CO2e per item.
To encourage a wider scale participation, Alibaba Cloud has partnered with 10 companies and brands, spanning across retail, transport to banking, to go sustainable inside the Asian Games villages.
Its sustainability platform Energy Expert provided carbon footprint measurement, carbon neutrality calculation, and certification services for the pin exchange center and low-carbon stores with energy-saving installations in one of the Asian Games villages.
As part of the sustainability initiatives during the Asian Games, Energy Expert helped Chinese dairy company Yili to identify the sources of the carbon emissions from its store at the Asian Games village and offered optimization plans for its electricity use to reduce its carbon footprint, including increasing the use of solar power and installing energy-saving lights and air-conditioner at the store.
It also advised Yili to build a dedicated section for low-carbon products and roll out a recycling machine for milk cartons.
