By Rachel Lerman
San Francisco - Chrissy Marshall has spent hours and hours scrolling through TikTok videos, deliberately liking and commenting on videos from other deaf creators and clips with small text bubbles. The 21-year-old is trying to train the app to only show her videos with captions or that use American Sign Language.
It's mostly working. Her TikTok homepage is so curated that Marshall, who has 1.2 million followers on her account ChrissyCantHearYou, only has to skip pastabout one in 10 videos.
"It really depends on who you are following and how you force the algorithm to work," she said.
This isn't an easy feat. TikTok, by design, is a place for millions of people to upload their own videos, without any requirement or even official suggestion to use captions. Videos include people dancing to music, ranting about their jobs, showing off new recipes and lip-syncing to the soundtrack of TV shows such as "The Office" or "Keeping Up With the Kardashians." Even if users want to caption their videos, TikTok's app doesn't have a way to automatically recognize voice patterns and automate text to use.
That makes the wildly popular app - used by nearly 100 million people in the United States each month as of last June - and many other social media apps moderately usable and sometimes frustratingly inaccessible for millions of Americans who are deaf or hard of hearing, several deaf creators said.
Others rely on old-fashioned social pressure to get people to amp up their accessibility game. A comment on a recent TikTok about customer service horror stories asked the creator, "could you put text/lyrics in there for the deaf People?" The TikToker obliged in a follow-up post, pointing out it was her first time using captions.
"I've learned to navigate it a lot more now, but definitely for general accessibility, it's a hard app to navigate as a Deaf person," Marshall said.
Social media apps have carved out prominent space for video for years, and in the past 12 months short video clips and audio-only messaging have hit the mainstream in a way they never have before - from TikTok's meteoric rise during the pandemic to upstarts such as audio-focused Clubhouse. That has forced a harsh spotlight - and provoked several instances of backlash - on the capabilities and shortcomings of these features on TikTok, Instagram, Twitter, YouTube and other popular apps.
The call for captions has a broad base of support. Up to 48 million Americans have a degree of hearing loss in one or both ears, according to a 2011 estimate cited by Johns Hopkins Medicine. Disability advocates also point out that many people without hearing loss use captions regularly to follow along on social media and TV shows.
A lack of captions was an inconvenience before the pandemic, said Berlin-based artist Christine Sun Kim, who is Deaf. But as work and socializing turned digital and generations flocked to TikTok and other apps during quarantine, the need to be able to follow along during Facebook Live events or live-streamed conferences became paramount.
It's not a new issue. But as an increasing number devices such as AI-powered smart speakers, doorbell security cameras and fitness-tracking watches transform everyday life, it's become even more critical to ensure everyone can access and use these tools.
Many of the tech giants have added features to their devices and products to make them more accessible for users who are hard of hearing or have low vision. But many deaf or blind people say there's still a long way to go.
Apple debuted VoiceOver, its screen reader feature that navigates Web pages audibly, on some iPhones in 2009. The iPhone maker also built in Braille capability on its products, and has controls for hearing aids and other accessibility devices. Facebook introduced automatic text using artificial intelligence to describe photos in 2016.
Instagram Threads, a messaging app from the company, has auto-captioning capability on short videos. Mike Shebanek, Facebook's head of accessibility, said auto captioning, especially on live events, can be a "daunting task" because videos can have multiple voices, background noise or other hurdles.
"We are trying to cover as much as we can as fast as we can," he said. "It's still an evolving art."
Google has entire teams devoted to accessibility and is often lauded as a leader in the accessibility space for deaf and hard of hearing people - YouTube has been equipped with automated captions since 2009. Google launched a feature for Android in 2019 called Live Caption that provides text for audio on users' phones.
Now, social media influencers are catching on to the fact that including captions, alt-text that describes images and transcripts of videos will get them the broadest possible audience. Influencers often make money by promoting products or services on their social media channels and regularly have millions of followers.
But despite the advancements, advocates say there is still significant work to be done to make social media inclusive for as many people as possible.
And missteps are still common.
Twitter was forced to apologize last year when it rolled out a limited feature to allow people to "voice tweet" but with no captioning support. A public outcry on its site pushed the company to admit it was wrong to test the feature "without support for people who are visually impaired, deaf, or hard of hearing," Twitter said in a tweet last June. It started testing a feature for group audio chats in December 2020 with some support for captioning. Amber Brown, a group product manager at Twitter, said in a statement that the company's "goal is to have automated captioning systems for all audio and video content on Twitter."
Clubhouse, a popular new app that focuses entirely on letting people gather in audio "rooms," has not introduced any live captioning support. Clubhouse did not return a request for comment.
Bryan Bashin, CEO of the LightHouse for the Blind and Visually Impaired, a San Francisco organization working for people who are blind or have low vision, said his screen readers on his Windows desktop and his Apple products make it possible for him to navigate social media sites. Bashin, who is blind, uses them to keep up with city announcements, connect with family and friends, and order an Uber. But he also says these technologies aren't perfect or all encompassing.
"There's often a lag," he said, pointing out that new features will roll out and not yet be fully accessible. Sometimes he will download a buzzy new app, only to find that it hasn't been properly vetted by the app store and isn't fully accessible. "And in the churn of new platforms and new ideas, we often find we're behind the curve on the newest thing."
Accessibility could be greatly improved by companies prioritizing it as they build new features, said Blake Reid, a law professor at the University of Colorado.
"Some companies have made serious commitments to design their products and services to be accessible," he said. "And for some companies, accessibility is not on their radar at all."
TikTok has rolled out features that warn users of videos with flashing lights. But it doesn't have a built-in way to easily caption videos. And for an app that relies heavily on sound, be it music, effects or repeated phrases, that can be constraining.
"I like the idea of TikTok, but I wish it had an easier way to add captions to videos," said Deaf YouTube creator Rikki Poynter, who has 91,700 subscribers on her channel.
The issue often falls to public pressure to get companies to pay attention. Marshall has made several popular and often educational TikToks about accessibility and ASL. TikTok reached out to her and appointed her to a panel about diversifying the app. They've been paying attention, she said, and she's hopeful about what's to come.
TikTok did not answer specific questions about auto captioning technology. In an emailed statement, Joshua Goodman, its director of product, trust and safety, pointed out a feature the company added recently to automatically narrate text for those who can't or don't wish to speak on videos.
"We're continuing to develop new products to make TikTok more accessible for everyone," he said.
Federal laws require closed captioning - or subtitles you can turn on and off - to be offered on broadcast television, including on live programming. But those laws, like so many others, were crafted before social media exploded and have not been extended in most cases to the user-dominated Internet.
In 2012, Netflix agreed to a settlement after it was sued by the National Association of the Deaf and agreed to add captioning to all its streaming shows.
But social media is a different beast - one that is populated by content from the masses, rather than carefully scripted and produced.
It's a tough problem to solve. Human captioners or interpreters are not realistic at the scale that social media presents - millions of users constantly making videos on a variety of platforms. Artificial intelligence technology has improved, but it still lacks precision.
YouTube is one of the only social media sites that automatically provides closed captioning on many videos, and even its biggest critics admit (somewhat grudgingly) that the technology has significantly improved in the past couple of years.
Poynter, the Deaf YouTube creator, helped popularize the term "craptions" with the 2016 hashtag #NoMoreCRAPtions.
"I just would have given up and said I'm just going to watch without it," she said. Words were wrong or missing, timing was off, formatting made it hard to follow the lines of text and still watch the video.
But now, she acknowledges automated captions have come a long way and are "a necessary evil." They still have a long way to go to refine precise wording, grammar and formatting, she said.
Without automated captions, many creators are left to fend for themselves - if they even realize captions are necessary. That struck art curator and author Kimberly Drew when she gave a talk with Kim, the Berlin-based artist, three years ago.
The discussion, about feedback and noise, was interpreted by an American Sign Language interpreter and Drew came away with a new outlook on her own work.
"I realized in that moment that every talk I had done before that one, to some extent, was a failure, because I wasn't aware of my own limitations in terms of outreach, my own limitations in the ways I was talking," she said.
Drew now takes the time to caption all her social media videos. She's used online services that allow her to transcribe text onto her videos. She's also paid captioners to do the transcribing for her and add it to videos in an accessible format.
Some captioners "will just put in a one- or two-word description," she said in an interview through interpreter Jennifer Vold. "But I understand that sounds are a huge part of a viewing experience. They really influence a viewer's emotions."
The same goes for alt-text that describes images on social media and is often used by blind and low-vision users. More description leads to a richer and more equitable experience using apps, said Erin Lauridsen, director of access technology for LightHouse.
Family members are always trying to get her to use Instagram, said Lauridsen, who is blind. But the polished, photo-filled app doesn't hold the same appeal for her when she runs a screen reader through posts, listening to automated image descriptions read out loud. She has to explain to family members why she just isn't compelled.
"Because I don't want to read 'picture may contain dog,' 'picture may contain baby' 10 times a day," she said.
Instagram spokesperson Liza Crenshaw said it worked to improve the feature's capability in January and February.
The calls for greater accessibility are not about creating special accommodations for certain groups, Lauridsen says. It's not a novel thought, it's just about creating equitable experiences for as many people as possible.
"We think of disability as something we need to help or fix or be charitable of instead of just part of the human condition," she said.
The Washington Post