(Bloomberg) -- Listen and follow The Big Take on Apple Podcasts, Spotify or wherever you get your podcasts.
With tens of millions of users under the age of 13, Roblox has become the biggest online gaming playground for kids. The company says its combination of AI chat filters and human moderators makes the platform safe for users of all ages. But a Bloomberg Businessweek investigation suggests that Roblox may have striking vulnerabilities.
On today’s Big Take podcast, host David Gura speaks with reporters Olivia Carville and Cecilia D’Anastasio about the way predators have used the platform to groom children — and what Roblox is doing to keep young users safe.
Here is a lightly edited transcript of the conversation:
David Gura: Roblox is one of the largest online gaming platforms in the world. And Cecilia D'Anastasio, who covers video games for Bloomberg, explained to me how it works.
Cecilia D'Anastasio: Think of it as a mall of video games, and there are all of these little enclaves in there. But it doesn’t charge admission to get into the mall, and you don’t have to pay to get into any of the enclaves.
Gura: And like the mall I remember, from my childhood, Roblox is a place a lot of kids go to to hang out. Worlds and characters on the platform look like a crossover between The Lego Movie and Minecraft: blocky, three-dimensional, and often colorful. And the platform has millions of user-made games.
D'Anastasio: There are games in Roblox where you pretend you're working at a pizza parlor, or where you fling yourself down a set of stairs repeatedly, or you raise virtual unicorns, and they're all very weird and whimsical and the kind of games that really appeal to kids.
Olivia Carville: When I started looking at Roblox, I was blown away by how popular it is.
Gura: That’s Bloomberg investigative reporter Olivia Carville. She says more than 77 million people sign on to Roblox every day. And more than 40% of them are under the age of 13.
Carville: This is where kids are today. Roblox welcomes children. It invites kids of all ages. You can create an account as a 3 year old, a 4 year old, a 5 year old, and play on this platform. And that really surprised me.
Gura: One thing that makes it so popular is it's really easy to access.
D'Anastasio: It’s really quite simple, and that’s why kids download it and play it — across iPhone, iPad, Android, PC, PlayStation, any device you can think of, it’s free there.
Gura: Signing up is simple. Cecilia says it takes about 60 seconds. You put in your birth date, pick a username and a password, and you can customize an avatar. Everyone on Roblox is anonymous. And once you’re online, you can interact with other players, and you can chat with them.
D'Anastasio: A lot of the trust and safety individuals that we spoke to for this story said that Roblox's open chat was one of its most dangerous features. You can create an account. You can talk to anyone. Anyone can talk to you.
Gura: Roblox uses a combination of human moderators and AI, and says it’s safe for kids of all ages. But as Olivia and Cecilia discovered, anonymity can be exploited. And it’s left some young users vulnerable.
Gura: Today on the show, a Bloomberg Businessweek investigation into how sexual predators have infiltrated the world’s biggest online playground, and the ongoing fight to keep users safe. This is the Big Take from Bloomberg News, I’m David Gura.
Gura: Before we get into the rest of today’s show, I want to let you know that we are going to talk about some tough subjects, including child abuse.
Gura: Roblox was launched in 2006, and while it grew steadily, Cecilia D’Anastasio says its popularity spiked during the pandemic, when kids were stuck at home.
D'Anastasio: Roblox just blossomed into this virtual cul de sac, and you know, when it went public in 2021, it had a $41 billion valuation.
Gura: Today, it has a market cap of about $25 billion and one way the company makes money is through a virtual currency you can purchase on the platform, called “Robux.”
D'Anastasio: Kids buy this currency. Kids beg for this currency from their parents. It’s huge. Roblox made $3.5 billion last year primarily through the sale of Robux.
Gura: Users on Roblox can spend that currency on virtual accessories like wings, hats, or costumes that make their avatars look like a cockroach, or like Peter Griffin from the show “Family Guy.” Established brands like Nike and Gucci sell virtual goods on the platform. But it's independent developers who make most of the items and games on Roblox. Cecilia says that in the world of Roblox, a game developer can have a huge following.
D'Anastasio: Roblox developers are, like, modern-day God kings on digital platforms. And that sounds like an exaggeration, but there are so many that have become so popular that they’ve had to completely anonymize themselves — lock down any even iota of some existence outside of an avatar and a username on Roblox. Because kids are, uh, ravenous for information about these people who are cultivating spaces where they forge their identities, they make connections, they grow up.
Gura: One of those developers who became wildly popular called himself “Doctor Rofatnik.”
That’s a not-so-subtle nod to the villain in the Sega game “Sonic the Hedgehog.” Olivia Carville says “Doctor Rofatnik” created what is basically a Sonic knockoff, called “Sonic Eclipse Online.”
Carville: Through Sega, you have to pay for the game. Here on Roblox, you’ve got access to it for free, and he knew that kids loved Sonic, so he created the space, and thousands of children came and played Sonic Eclipse Online.
Gura: And as his game became more and more popular, so did Doctor Rofatnik himself. Cecilia and Olivia told me his signature avatar would wander around the world he’d built, soaking up the adulation.
D'Anastasio: On Roblox, he was just wearing a tall white hat and a red tie and an American flag pin, and he looked debonair, professional — even political.
Carville: He said he was an industry visionary. That he was braggadocious. He claimed he was 28 years old, that he lived in California, that he was the younger brother of John Shedletsky, who was the platform’s first creative designer, who was an absolute legend among children.
Gura: “Doc,” as he came to be known, also told users that he had a, quote, “hot Spanish girlfriend,” That he drove a fast car. And he’d encourage Roblox users who’d chat with him to take those conversations with him to other online platforms.
D'Anastasio: You can click on a Discord link, or an X link, and your relationship — all of a sudden — is cross-platform, and there, it can blossom.
Gura: Soon, Doc was multi-platform famous.
Carville: So, he created a, you know, an online kingdom where children could look up to him, interact with him, talk to him regularly. Some of them said he was online 24-7. Anytime you reached out to him, he’d reply.
Gura: And what a lot of his followers found appealing about Doc, Olivia says, was his “dark and edgy” humor. But also, that he’d pay them — in Robux.
Gura: What was he paying them for? What was he asking them to do for that virtual currency?
Carville: He actually had dozens of kids working for him. They also, um, would design new characters for him, or new spaces inside the game, and he’d pay them for that.
Gura: Doc also encouraged bullying, Olivia says. He used racist and homophobic slurs. And in 2020, a group of teen vigilantes on Roblox decided they’d had enough. Just a heads up: what they sounded the alarm on could be troubling to hear.
Carville: These gamers go after Doctor Rofatnik, and what they they do is they post screenshots of a conversation he had with a young, 12-year-old girl, who worked for him, where he talked about kidnapping her, raping her He had very inappropriate, hyper-sexualized conversations with her, and all of those screenshots were uploaded into a document they dropped on Twitter.
Gura: One of those gamers took that document to the police, but that went nowhere.
But eventually, the mother of that 12-year-old girl saw those messages, and she contacted Roblox herself.
Carville: Her mom files a report to Roblox, and explains that, you know, there’s a really creepy, dangerous man, or a developer, going after her child, and asks the platform for help. So Roblox did receive a number of complaints from users about the Doctor Rofatnik account.
Carville: And four days after being notified of these messages that he had been exchanging with these young users, they blocked his account. They shut him down.
Gura: But that wasn’t the end of it.
Carville: It was kind of just the beginning.
Gura: We’ll be right back.
Gura: Roblox blocked Doctor Rofatnik’s account, but the man behind it was able to stay active on the platform. He simply made another account.
Before his main account was banned, Doc transferred ownership of his game, Sonic Eclipse Online, to a friend who kept running it on his behalf.
Roblox left it active because they said the game itself didn’t pose any safety concerns and told Bloomberg it continued to search for and ban any of Doc’s alternate accounts. So kids kept playing the game Doc developed. They kept paying him through their in-game purchases. And they continued to look up to him.
One of them was a teenage girl from Indiana.
Carville: In May of 2022, a 15-year-old girl goes missing. Her mom comes home, and sees that all of her electronics have gone from her bedroom. Her favorite blanket’s gone. Some of her clothing is gone. Her chargers are gone.
Carville: And she sees that she’s posted a photo onto Instagram, which shows her sitting inside a stranger’s car, looking out the window, and she just wrote, “Goodbye, Indiana.”
Gura: Law enforcement started to investigate what they assumed was a child abduction case, and a detective asked the girl’s family who she’d been talking to online.
Carville: She started to hear about this mystery figure who said he was a popular developer on this gaming platform, Roblox. That he lived in California. She liked to draw. And he actually called the mom at one point, and told her that he wanted to buy this teenage girl’s artwork.
Carville: After that, Amazon packages started arriving on the doorstep, and the girl would receive, you know, different tools or technology to help with her artwork. She received a teddy bear, takeout food. McDonald’s, Chinese noodles would come.
Gura: Again, that’s according to the detective’s account of her conversation with the victim’s family. The police got ahold of the girl’s private communications on social media, and they found she and a man had exchanged hundreds of sexualized messages.
Gura: The FBI took over the case, and agents traced those Amazon packages to an address in New Jersey.
Carville: Six unmarked cars pulled into the street for the stakeout of this house, and within minutes, the teenage victim rounded the corner with Arnold Castillo.
Gura: Arnold Castillo, or “Doctor Rofatnik,” had groomed the girl in the virtual world of Roblox, and then abused her in the real world.
Where he was not a 28-year-old brother of a famous game designer in California. He was a lonely 22 year-old man, whose mom — according to prosecutors — pulled him out of school in the seventh grade and told him to make money through developing video games. He still lived with her in a small apartment above a garage. Castillo put his victim in a room he’d rented in a building next door.
Carville: The prosecutor described the room that she was kept in as this teeny, tiny, dank room, where he’d thrown a twin mattress on the floor. He bought her hair dye, to change her appearance.
Gura: Olivia says she and Bloomberg gaming reporter Cecilia D’Anastasio managed to determine – and Roblox confirmed – that Castillo was active on Roblox on the day of his arrest. He was using a different username than DoctorRofatnik; but he was active on the platform all the same. He pleaded guilty to “Transportation of a Minor with Intent to Engage in Criminal Sexual Activity” and “Coercion and Enticement of a Minor,” and Castillo is now serving a 15-year prison sentence.
Gura: Police across the US have arrested at least two dozen people since 2018, who have abducted or abused victims they met on Roblox. That’s according to data compiled by Bloomberg Businessweek.
Gura: Olivia and Cecilia reached out to Roblox, for comment.
Carville: We received a statement from Roblox’s chief safety officer, Matt Kaufman, and he said that safety and civility are foundational to Roblox.
Carville: They said tens of millions of people of all ages have a safe and positive experience on Roblox every single day.
D'Anastasio: Kaufman also rejected the claim that child endangerment is widespread, or a systemic problem, on Roblox. He says that the company has strong systems and protocols that are designed to catch anyone circumventing their rules, including attempts to move conversations off Roblox for users who are under 13.
Gura: But Olivia and Cecilia say that, current and former employees they spoke with told them that before 2022, Roblox didn't have automated systems in place to proactively search for grooming behavior beyond basic text filters.
According to their sources, back then, the word “grooming” didn’t even appear in the company’s moderation guide.
Roblox has made some changes since Arnold Castillo was arrested:
The company has two new child safety investigators, and a child exploitation moderation team. There’s a child safety officer who reports to Roblox’s C-E-O.And on July 15th, after multiple inquiries from Bloomberg Businessweek, Roblox announced it would change its default settings. After this fall, users under the age of 9 will be automatically restricted to games that feature nothing more unsettling than mild violence and unrealistic blood.
Parents will have the option to allow their children to access more mature content. But the scale of this problem is only just coming into focus.
In 2022, Roblox reported nearly 3,000 incidents of child exploitation to the National Center for Missing and Exploited Children. In 2023, it flagged more than 13,000.
Gura: What really worries child-safety experts is Roblox’s ambition. It’s a publicly traded company that’s trying to get even bigger, and to do that, it’s aiming to have one billion users every day.
Gura: So, you’ve spoken with a lot of, I guess, past and present employees of the company. What do they say about the challenge of scaling up the company to that size, and also scaling up the safety side of things, in tandem?
D'Anastasio: It would be very difficult to find a platform out there where safety and growth were not at odds, and Roblox is one of those platforms. Sources I spoke with at the company said that artificial intelligence isn’t really capable of picking up on the subtle signs of grooming.
Gura: Eight current and former trust and safety workers told Bloomberg Businessweek that user growth at Roblox takes priority over child safety. They described calls for more resources going unanswered, resulting in a backlog of safety incident reports.
A Roblox spokesperson disputed the claims about resources and backlogs and says the company has a “robust pipeline” of safety features in development.
Roblox says that every second, 50,000 chat messages are sent on the platform. But it’s not just the volume that makes Roblox hard to police.
D'Anastasio: How do you tell if an avatar is simulating sex? How do you tell if an avatar is simulating rape? These are enormous challenges and ones that Roblox, I think, has not necessarily grappled with.
Gura: You've talked to online safety experts. What is the advice that they give to parents whose kids are in this virtual world and to kids themselves who are trying to navigate this at a very young age?
D'Anastasio: The question of child safety on all of these platforms is what can parents do? What can parents do? How can parents, you know, get into their kids’ devices and ensure that they're safe? And not what defaults must there be? What age-gating measures must there be? What chat filters must exist? What moderation protocols, what number of moderation should be employed by these companies?
And I think part of the issue is that there isn't the kind of literacy with these platforms that's necessary for parents to really advocate to them and to really make specific asks or to ask politicians to make really specific asks.
I'm not a parent myself, so maybe I'm speaking out of turn when, sure, they can disable a lot of settings, but, you know, we exist in a market where growth is what platforms desire and safety is at odds with that.
And there just aren't a lot of laws or policies that are forcing companies to adopt common sense measures that will protect children. That's something that safety experts have repeated over and over and over to us.
Carville: So much of this conversation has been so dark and so depressing, and while we don't want to deflect responsibility onto parents, Roblox should be doing more to keep its platform safer. They even say that themselves. They're continually working to improve their safety measures.
But there are things that parents can be doing, that they should know about. You know, there are safety features in the app. If you do some research, you can switch on these safety settings. They're not turned on by default, but you can go in and do that for your child,talk to them openly about their experiences in the game. Allow them to come to you to have those conversations.
And I think that, you know, parents should be proactively looking at what their kids are doing, because unfortunately, the platforms just aren't doing enough right now.
ia.
©2024 Bloomberg L.P.