Cnut Songs #20: The Every (2)

I bought this for Louise on Mother's Day. He crushed it with his own feet. 

Someone at work asked, "Is Louise a big fan?" No. That's why it was funny. 

Earlier in the week, Martin blogged about his experiments with ChatGPT, the new AI chatbot that's about to replace human "experts" across the world. We all agreed how scary this was. I commented that it reminded me of Dave Eggers' firm belief that society is moving towards handing over freedom of choice in return for the safety of conformity.

Here's another extract from The Every, a novel which is best described as 1984 meets Catch 22 with a healthy dollop of Gulliver's Travels. Yes, it's satire... but so close to reality that it's hard to work out which bits are true and which bits Eggers has created for comic effect.

In the following extract, Delaney has gone to work for the HereMe department which creates Smart Speakers. Eggers uses this as an excuse to chronicle the secret history of Smart Speakers, which sounds like fiction… until you realise we’ve already lived through much of it.

Smart speakers had an awkward introduction to the world. They arrived in the 2010s to phenomenal sales, with hundreds of millions of households adopting them within the first five years. Before The Every entered the picture, the makers of the devices assured the buyers that the AI assistants were never activated unless their designated name or code word was spoken. This reassured the users that their private everyday conversations were not being heard, that only brief requests were audible, and even then, never stored. But a few months later, it was revealed that the smart speakers were in fact listening all the time, or could listen all the time. In fact, they could be activated by their manufacturers any time at all. For this, the manufacturers apologised; perhaps there had been some confusion, they said. Were we unclear?

The users, though momentarily upset at this foundational and central deception, were assuaged when they were told that under no circumstances were their conversations recorded. It would be, both users and manufacturers agreed, an egregious breach of trust to have a machine that a customer brought into their house – a machine, everyone noted, that was purchased primarily to play music and inform them of the current traffic – actually recording the conversations conducted in these private households. That would be unethical. And so it was assumed that no recording was being done by these home assistants, until one day the manufacturers admitted that they had in fact been recording just about every conversation every user had ever had, from the very beginning.

Again the makers were contrite. When you were asking before about whether we were recording conversations, they said, we didn’t quite understand what you meant. We thought you meant recording and listening to these conversations, and that of course we would never do. We would never. We record the conversations of hundreds of millions of users, yes, but no humans ever listen to any of these conversations. Conversations in the home, between family members, are private, are sacrosanct! they said. We simply record these conversations to improve our software, they said, to optimize our services, to better serve you, the customers, better.

And for a while the users, though feeling wary and burned by the series of revelations, looked askance at their smart speakers, wondering if the tradeoff was actually worth it. On the one hand, their private family conversations were being recorded and stored offsite for unknown future use by a trillion-dollar company with a limitless litany of privacy violations. On the other hand, they could find out the weather without ever looking out the window.

Fine, the users said sternly, fists on hips, you can continue to record everything we say, but – but! – if we ever find out that you manufacturers were having actual humans listen to our conversations, that will be one step over the line.

We would never! the manufacturers said, hurt by the inference, which, they felt, was offensive even to think about, given how open and transparent they had been from the start. Didn’t we reveal, they asked, after we were caught, that our smart speakers were turning themselves off an on at their own behest? And didn’t we admit, after we were caught, that we were listening to and recording anything we wanted at any time, anything that was said in the private homes of millions of users? And didn’t we reveal, after we were caught, that we were recording all the private conversations every user had in the privacy of their own homes?

After all this openness and contrition, they said, it stings to think that customers would wonder aloud if the other shoes might drop. No more shoes, said the manufacturers, would be dropping. We stand before you barefoot and humbled.

When it was revealed that the manufacturers had in fact hired 10,000 humans, whose only purpose was to listen to, transcribe and analyse the private conversations that had been recorded by these smart speakers, the manufacturers were amazed at the outrage, as muted as it was. Yes, they said, we have all along been recording and listening to your conversations, but none of these 10,000 workers know your names, so what possible difference would it make that we have all of your private conversations recorded, and that we could with one or two keystrokes de-anonymize your conversations at any time? And given the fact that every database ever created has been hacked, these recordings could be accessed by anyone at any time who had will enough to get them? What, the manufacturers asked, are you getting so worked up about?

In fact, no one got worked up at all. Lawmakers were mute, regulators invisible, and sales skyrocketed.

Eggers goes on to suggest the logical next steps for Smart Speakers... and they seem frighteningly prophetic. Go read The Every if you want to be really scared...

Miracle Legion came from Connecticut in 1983. They were on Rough Trade for a while, so the NME loved them. This is from their 1992 album Drenched. I'm a sucker for songs with "Ba ba ba ba baaa" choruses.