How social ratings systems are going to ruin your life

Are you ready for a 'Black Mirror' dystopian reality?

There’s a scene in an episode of Black Mirror—‘No Name’, it’s called—where Lacie loses three ‘stars’ at an airport in California for ‘bad behaviour’. In Lacie’s dystopia society, this spells disaster: everyone has to rate and be rated according to their social interactions, class hierarchy and professional success.  A double demerit penalty after she swears and yells at a flight attendant, causes Lacie to drop from her position as respected middle-class citizen to social outcast.

But how far is fiction from our reality?  It’s already happening in businesses like Uber, Yelp and AirBnB.

The ‘gold star’ stickers of our childhood, shiny rewards for conformity and successful dictation, have been replaced with the virtual gratification of social media. It’s seen in the cultural practices of rating, liking, and swiping our way through social interaction. Most recently, these rating practices are morphing into complex systems of national security, with the Chinese Communist Party planning to implement a national Social Credit System by 2020. Under the proposal, people’s behaviour will constantly be tracked and monitored to manage a ‘citizen score’ where everyone is rated from 0-200. It’s a new kind of societal structure powered by technology and measured by perceived acts of ‘goodness’.

What’s perhaps most frightening about this technological conditioning is the way it blurs our actual and imagined lives to the point that they are, in some contexts, merged. One cultural opinion that, at least currently, curtails this expansive and pervasive mode of expression is the ability to choose to disengage. “Switch off,” they say.

Go on a phone detox, shut down your devices, turn to airplane mode. But when an entire authoritarian society accumulates your privileges and benefits through a government-regulated online sphere, it seems we might reach a point of absolute ubiquity, with no return. It’s not unreasonable to expect a blurring of the acts of general moral good, with practices of making good change only for the expectation of reward from the state.

Monopolising on these public displays of goodness may just make entire societies vacuous. Within China’s Social Credit Systems, points would be deducted for law breaking and added for charitable donations or volunteer work. The problem is, algorithms don’t possess nuanced understandings of context and circumstance when measuring what’s good and bad and its severity. If a person jaywalks all the time, does that make them a bad person?

Ultimately, there’s a chance this kind of controlled landscape will breed a general distrust amongst communities, with individuals encouraged to report on the ‘wrongdoings’ of their colleagues, peers and families. In China, this kind of system could see a return to the kind of oppression not seen since the days of Chairman Mao. But this time, it may well be more pervasive than ever because it relies on omni-present networks of global technological dependency.

“Social ratings systems position all social media users as unpaid critics,” said Dr Fiona Martin, Research Director and Senior Lecturer in convergent and online media at the University of Sydney. “As well as consuming products, we are now pushed to rate them and to give detailed feedback.”

We critique these technological transformations in China at a distance. But in reality, micro-rating and credit systems are popping up in all aspects of western lives. For example, take the rise of the social influencer, a position based on technological and social personality. It’s a career based on exploiting self-image and the advertising potential of social capital to eke money out of online consumerism. It is these permanent online reputations that provide little opportunity for moral development or critique to emerge, on a platform in which these influencers appear as friend not advertiser.

It’s obvious even now, in the increased demand of individual online personalities to receive jobs, organise social events and even legitimise our own self-belief through the number of likes on our Facebook profile pictures.

Surely, with the advent of social credit systems, the stress of consistently monitoring your scoring in real time—not to mention the tangible consequences of fluctuations in rating—leaves no room for human mistake. In the case of Black Mirror’s Lacie, the social fall from ratings of 5 to 1 may just be as devastating as its climb.