How do you solve a problem like social media? Part I

It's all well and good saying we should regulate social media. But social media is already regulated. So why is it failing? And what could we do in response?

When I joined Facebook back in 2006, it was mostly about “throwing a sheep”, people untagging themselves from last night’s photos, and jokes about “fraping” that have not aged well. One of the few Facebook groups I joined was the “random early 90s footballer appreciation society” – Gunnar Halle, Efan Ekoku, that sort of thing. The Barclaysmen of their day.

I also remember a very early Facebook group dedicated to not talking about politics – which soon got derailed when someone decided to post about how great Israel is and a giant ruckus kicked off.

Still, nobody was actually trying to start a race war or build the Fourth Reich. So it had that going for it.

Twenty years later, social media is the fascist arsonist’s petroleum of choice. Facebook is now full of AI conspiracy slop and neighbourhood racists who want Efan Ekoku to eff off back where he came from (Cheetham Hill, since you ask). YouTube and TikTok’s algorithms rapidly open users’ doors to far-right propaganda videos. X is owned by the richest fascist in human history who uses it to advance that cause on our shores.

You know this already. So does the government.

Elon Musk unsticks his sweaty armpit in a regrettably Nazi fashion. (Credit: Angela Weiss/AFP/Getty Images)

The Ofcomfort zone

“We need to regulate social media!”

The thing is, social media is already regulated – but you might not realise it. Recent weeks have seen plenty of evidence that Britain’s Ofcom-led system of regulation isn’t working: racist pogroms openly whipped up on mainstream social media platforms, extreme racist abuse left unchecked on X.

I think one problem is likely to be attitude. Like many of Britain’s regulators, Ofcom appears to take a light-touch approach to certain aspects of its role, behaving more like a constructive “enabler” of providers than a watchdog. That approach falls down when dealing with bad faith actors. We’ve already seen this with Ofcom’s rank ineffectiveness towards GB News.

If that was all there was to it, this would be simpler – just give Ofcom a kick up the backside. But the system established by the Online Safety Act (OSA), through which Ofcom regulates social media, is itself proving inadequate.

First, it’s ponderous and detached. Over time, Ofcom publishes codes of practice. Over time, social media platforms carry out internal risk assessments on illegal content and consider how to mitigate those risks. Over time, Ofcom considers how effective these mitigation measures are.

Second, Ofcom gives providers wriggle room, which weakens accountability.

Two recent examples show how these two factors combine to create a cocktail of incompetence. One is think tank British Future’s research into how X handles reports of extremely racist posts. Researchers found the site only took action when a post was specifically reported to it as “illegal”, and not if it was reported as “hate, abuse or harassment” – even when posts were clearly in breach of hate laws, and even though hate, abuse and harassment are specifically listed as types of illegal content in Ofcom’s OSA guidance. The news came a month after Ofcom announced X had voluntarily agreed to remove illegal content within two days. X had taken a credulous Ofcom for a ride while doing the barest minimum.

The other example concerns an online suicide forum that Ofcom fined £950,000 in May, with a threat of legal action to shut it down in the UK if the forum fails to block UK users itself. At first glance, this seems like the kind of meaty regulatory clampdown Britain needs. But the fine only came at the end of a 13-month-long investigation, during which the forum – linked to more than a hundred UK deaths – was still visible to users in this country. Worse, Ofcom initially dropped enforcement action after the forum said it would block UK users voluntarily. The regulator only restarted proceedings after campaigners pointed out the forum had previously offered to block users when faced with scrutiny from other countries, only to U-turn once those investigations had closed.

I think this combination of dynamics is why regulation is currently failing. Normally the right move would be to give it time – the OSA’s provisions around illegal content only took effect last year. But with far-right race riots driven by social media suddenly a recurring feature of British life, we don’t have time to wait for a failing system to fix itself.

Ofcom’s London HQ, with its architectural bulbous salutation

But what is “social media”?

You can’t regulate something without first defining what it is. Helpfully, the Australians already have. Section 63C of legislation from 2021 underpins their more recent teen social media ban. Last month the British government effectively adopted Australia’s approach.

Excluded from Australia’s ban are messaging apps like WhatsApp and online gaming services such as Roblox, as well as Discord, the widely used messaging service originally built from gaming communities. Included are Facebook, Instagram, Reddit, Snapchat, Threads, TikTok, Twitch, X and YouTube, among others.

How regulations define “social media” should be guided by what those regulations are targeting. I’m not looking at the rights and wrongs of age-related bans here. I’m focusing on misinformation and extremism across our politics and society. That means social media regulation must continue to factor in YouTube and its huge user base.

Which means you can’t just “ban algorithms”.

Do algorithms dream up electric creeps?

YouTube has always used some form of non-chronological algorithm. Facebook and X could (and originally did) function chronologically, much as Bluesky does – just a timeline of posts by people you chose to follow. But YouTube would be little more than a video search engine without non-chronological algorithms.

The problem isn’t algorithms per se, so much as what they’re used for and the effect they have – after all, what’s wrong with recommending cookery videos to someone who watches cookery videos?

Simply making algorithms transparent doesn’t stop malpractice without proper rules and enforcement (as we see with political corruption – sorry, “donations”). But how do you write rules to specify that one algorithm is “good” and another is “bad”? Often the problem isn’t that algorithms are specifically designed to promote extremist content, but that they create filter bubbles based on prior usage, or promote material that is receiving high levels of engagement, with extremist content travelling widely as a side-effect.

Trying to directly regulate the algorithms themselves, with Ofcom deciding which algorithms are acceptable and which aren’t, would be a recipe for a huge mess.

At present, Ofcom requires social media platforms to use their OSA risk assessments to consider how their algorithms could lead to more illegal content. But despite this, algorithmic social media continues to foster hatred, abuse and extremism. Something has to change.

The baby and the bathwater

We should regulate social media’s output, not micro-manage its algorithmic process. But the OSA only does this in a hands-off way.

The Act requires social media providers to “swiftly take down” any illegal content once it becomes aware of it. But the OSA is weak at holding providers accountable for this. Ofcom’s guidance says it “may consider” whether a platform’s “illegal content judgements follow the approaches” set out under the OSA. That means a platform’s decisions on whether to remove certain posts may be taken as evidence of how well it is complying with the law – as long as Ofcom chooses to pay attention.

If that was proving effective, we wouldn’t be where we are now, Lisa Nandy wouldn’t be quitting X and I wouldn’t be writing this post.

Lisa Nandy’s banner picture on X with the possibly floating head of Hacker T Dog. Look, I’m running out of image ideas to break up the text and she probably won’t claim copyright

I’m not the first to suggest that we should treat social media companies as legal publishers of material posted on their platforms.

But doing so requires penalties for companies that fail to meet those regulations – and those penalties need to be big enough to actually incentivise compliance, rather than just being treated as a “cost of doing business”.

Changing the incentives for social media providers means changing their balance of risks in order to change their responses – but those responses won’t always change in the intended way. Severe penalties for breaking badly defined rules can lead to ultra-cautious overcorrection.

And the sheer volume of material on social media means regulation can’t work the way it does for traditional media. Traditional media providers operate at a small enough scale to allow pre-emptive editing and real-time moderation of everything in the publication. Contentious articles go through legal checks before they are published.

By contrast, social media sites are bottomless pits of content. A BBC investigation last year looked into the great unwatched mass of YouTube videos – snow falling outside a window, a six second video of a martial arts instructor, a guy who calls himself the “WooFDriver” and builds custom-made vehicles to travel around with his huskies. The site hosts literally billions of videos, with millions more added every day. No company can pre-emptively monitor all that.

Meet the WooFDriver - this particular video has so far had 110 views on YouTube. I assume it complies with the Online Safety Act but I haven’t checked

A system of regulation that held social media providers responsible for anything posted on their site from the moment it was posted would end social media in this country overnight. Maybe you’d like that. Most people wouldn’t. You can’t put this two-decade-old technology back in its box and burn it – not in a democracy anyway, and hopefully we still believe in that.

Therefore, responsibility should lie with the social media provider “as publisher” only where that provider has failed to swiftly remove harmful content that it has been notified of, or has failed to put systems in place to enable this.

That takes our current approach to regulation, but sharpens its teeth.

Tiers for fears

But how might such a system work? My view is that regulation should take different approaches to different harms on different platforms based on three factors:

• how serious the harm is

• how easily defined the harm is

• how culpable the provider’s systems are for enabling and spreading the harm

Let’s start with that last point. Some social media platforms pay users based on the number of views their posts or videos get. Such a platform is arguably more culpable for illegal content that spreads on its site – we know material that is bigoted, incendiary or sexually abusive “travels well” in the swamp of social media, so financial rewards for viral content create an incentive to produce harmful material.

A social media provider that uses non-chronological algorithms to determine what users see may also bear added culpability for online harms. Technically that depends on how the algorithm works, but as I said earlier, trying to nuance between different algorithms would create an unwieldy regulatory mess. Just treat all non-chronological algorithms the same.

Therefore I think regulation should be structured around “tiers” of culpability:

• platforms that financially reward content based on viewing figures go in the highest tier

• platforms that don’t do this, but do use non-chronological algorithms go in the middle tier

• other platforms go in the lowest tier

For all the criticisms of it, the existing Ofcom/OSA system is probably sufficient for sites that primarily operate chronologically* (see footnote) and don’t themselves pay contributors for content – the “lowest” tier, so to speak.

Take the most obvious example – Bluesky, whose main feed is chronological. The site does have an anti-Semitism problem, but such posts are not actively promoted by the way the site is designed. It’s reasonable to require such providers to formulate policies to clamp down on illegal material, and then be held to account by Ofcom for how effectively they do so, with potentially stiff penalties for persistent failure (Ofcom’s available penalties under the OSA are actually pretty meaty, including potentially huge fines). That does, of course, need Ofcom to pull its finger out more than at present.

The existing OSA regime should continue to apply to all social media providers. But those that use non-chronological algorithms should face additional duties – if such a platform is notified of harmful content and does not swiftly remove it, the platform should be considered the publisher of that content and thus become liable for it, alongside the individual who posted it.

To be clear, once notified of the content, it should be the site’s responsibility to correctly identify it as illegal. X should not be able to hide behind the defence that content had been reported as “hate, abuse or harassment” rather than “illegal content” – why should the burden of knowing if something is against the law lie with the member of the public reporting it instead of the giant corporation?

You pay? We’ll take it away!

The highest tier of culpability would lie with those sites that pay producers of content based on their traffic. Unlike non-chronological algorithms, I see no valid use for this. Popular YouTube content creators make most of their money through advertising** (see footnote) and Patreon-style fundraising – so payments by the platforms themselves are not required for that content creator economy to exist. Instead, these payments reward low effort, low quality material that goes viral, such as incendiary AI slop. These payments are why a devout Muslim in Pakistan is producing anti-Muslim content for British Facebook and Instagram users.

Just some of the thousands of pounds Restore leader Rupert Lowe makes from posting on X

I am exceedingly comfortable with killing off this business model – effectively driving it from these shores. Any social media site that pays posters of content based partly or wholly on their volume of views in the UK should be in the highest regulatory tier, regardless of whether the poster or content in question was actually paid or not.

This tier would be covered by the OSA and the regulations applying to the middle tier. But platforms in this highest tier would be considered the legal publisher of a much broader range of harmful content. In other words, the highest tier would carry more liabilities.

Analogue laws in a digital age

It would be easy to just throw the book at all social media. But if we accept that both social media and algorithms have legitimate uses, then we need to consider practicalities.

Not so much whether it’s reasonable to expect platforms to hire the staff to monitor and respond to reports of illegal content – most of the large sites are owned by multibillion dollar corporations; Elon Musk was a trillionaire for a week; like, come on.

Instead it comes back to the questions I raised earlier – how serious the harm is, how easily defined it is, and what are the risks of overreach when you shift platforms’ incentives.

Let’s take incitement to violence. This is very serious, especially at times of heightened tension. The OSA defines it using section 4 of the Public Order Act 1986, which covers the use of “threatening, abusive or insulting words or behaviour” in order to provoke the “immediate” use of violence (we can only hope material provoking “non-immediate” violence would be covered by other laws). In its guidance, Ofcom specifies that content “posted in the course of ongoing public disorder” could be inferred to be provoking immediate violence.

I’d lean towards placing incitement to violence in both the middle and highest tiers, as the potential severity of the harm is so great. If platforms in these tiers didn’t remove this content when notified of it, they’d be held responsible for publishing it.

Beyond that, things get more complicated. That’s because the OSA came to be based on the principle that it should only penalise content that is already illegal – you shouldn’t be allowed to say things online that you wouldn’t be allowed to say offline, but equally Ofcom shouldn’t ban you from saying things online that you can legally say offline.

Before it was passed into law, the Online Safety Bill did actually contain provisions against “legal but harmful” content. Platforms would have had to assess risks and set terms of service regarding content that had a “material risk” of causing “a significant adverse physical or psychological impact on an adult of ordinary sensibilities” (section 46 of the draft Bill here). The resulting terms of service might have led to such content being removed.

Given the Bill’s vague wording, it’s little surprise that concerns over free speech were “particularly focused on the government labelling legal content as harmful, and indirectly incentivising companies to remove this”, as Rishi Sunak’s government noted while announcing the “legal but harmful” provision was being axed. It didn’t make it into the final OSA.

But by only targeting content that is already illegal, the OSA ties the new regulatory system to existing laws that were often written decades earlier with criminal law processes in mind. Not only were those old laws drawn up before social media existed, but they were written based on the lengthy due process whereby criminal offences are investigated over a period of time, with charges only brought if evidential hurdles are cleared. They weren’t written for online moderators to make quick judgements about.

In addition, with content flagged for hate, threats, abuse or harassment, Ofcom’s guidance (section 3 here) tells providers to go through each offence in turn in order of “the simplest criteria for illegality” – so start with the offence with the simplest criteria, and if the content in question doesn’t meet that criteria, move on to the next offence, and the next, and the next. That’s potentially a lot of offences to judge each post or video against.

Take “harassment, stalking, threats and abuse”. Here is Ofcom’s definition:

That’s 12 different sections of legislation across three different jurisdictions (England and Wales, Scotland, Northern Ireland). But besides the sheer number of offences to go through, the bigger issue is the type of judgements we’re requiring social media providers to make.

‘Ooo remembers the Twitter joke trial?

For example, section 5 of the Public Order Act defines an offence as using “threatening or abusive words or behaviour, or disorderly behaviour … within the hearing or sight of a person likely to be caused harassment, alarm or distress thereby”.

Sometimes these judgement calls will be obvious; other times they won’t. Ofcom’s guidance tries to clarify the meaning of “distress” and “harassment” thusly: “Distress involves an element of real emotional disturbance or upset. The same is not necessarily true of harassment. A person may be harassed, without experiencing any emotional disturbance or upset. However, although the harassment does not have to be grave, it should also not be trivial.”

If we hold social media platforms responsible for individual posts and videos they don’t remove, and punish them for not removing them, they are liable to over-correct and overreach in response. We cannot assume they will employ moderators with extensive legal training – they may be inexperienced, with a brief crash course in the law and Ofcom’s 442-page guidance document. Would posting “Gary Sambrook eats big dinners” constitute abusive words likely to cause harassment or distress? No. Might an inexperienced put-upon moderator, working under threat of sanction if their employer got fined for not deleting an illegal post, decide the safe approach is removing it? Stupider things have happened.

The only thing Gary Sambrook is remembered for

Bear in mind the OSA doesn’t have a solid counterweight to this. While Ofcom’s guidance regularly refers to the importance of free speech and political expression, the OSA is underpinned by the principle that private companies are free to go further than the law in policing their platforms – they are allowed to block more than just illegal content if they want, because they’re private entities. Which is as it should be – some online spaces choose to be “safe spaces” with tighter moderation policies than strict illegality, and that is their right.

For these reasons, I’d place both hate speech and harassment/stalking/threats/abuse in the highest, paid-content tier. At present, that would still cover X, Facebook, TikTok and YouTube, as a reflection of the higher risk of harm their payment models carry, and perhaps as an incentive for them to remove them*** (see footnote). As for the other tiers, this content would still be covered by the OSA provisions – and, for the individuals posting such content, the criminal law.

There are two issues I haven’t covered here, and they’re big ones – misinformation and enforcement. I’ll look at those in part two later this week.

Footnotes:

* There’s a question over how you’d treat platforms that have both a chronological and a non-chronological feed – Bluesky does, as does X. You could perhaps base it on the “default” feed. Bluesky defaults to a chronological feed, while X defaults to the infamous “For You” torrent of sewage.

** This can be quite a technical area – some platforms take advertising revenue and divvy it up between content creators based on viewing figures, while others only pay ad revenue to those creators whose content accompanied each advert. I’d say the former structure counts as paying for views while the latter does not. As it happens, YouTube does both – the former payment structure for its quicker “Shorts” videos, the latter for its longer form content.

*** And if we really want to force social media platforms to stop paying creators for clicks, we could even consider the nuclear option – holding platforms in the highest tier as publishers under our onerous libel laws for defamation of individuals. That could make the paid-content model outright impossible to maintain.