The Future of Privacy and the Internet of Shit

Introduction: IBM leaves the facial recognition business

KMOIf you speak the truth, have a foot in the stirrup. Hello, and welcome to another installment of the C-Realm Podcast. I am your host, KMO, and this is episode number 561, “The Future of Privacy and the Internet of Shit,” with guest Matthijs Pontier, prepared for release onto the World Wide Web on Thursday, June 11th, 2020.

So, as if the title didn’t spell it out clearly enough, in this episode I will be talking with my guest about privacy and the prospects for maintaining it, or reclaiming it as need be, in the digital era.

So everybody’s up in arms about race right now, which, for me — I get it. I mean, it sucks to be a young man not in the upper or upper middle classes in the United States when you have to deal with police. I have no doubt that it is worse for young Black men than it is for young white men, but it’s no picnic for anybody. The police are a problem. They have been a problem for a very long time.

So, on the topic of this week’s episode: IBM a few days ago announced that they’re relinquishing all their ambitions with regard to facial recognition technology. They’re not going to research it anymore. They’re not going to develop any products based on the technology. They’re just not pursuing it anymore. I’m going to read a couple of paragraphs to you from an article on Vox.com — yeah, I know. It’s called “Why it matters that IBM is getting out of the facial recognition business.” This is by Rebecca Heilweil.

This is not a quiet announcement by IBM. In a letter to members of Congress, IBM CEO Arvind Krishna said the company would no longer make general-purpose facial recognition and analysis software, citing concerns about the technology’s use by law enforcement agencies. He clarified that IBM “firmly opposes” the use of facial recognition “for mass surveillance, racial profiling, violation of basic human rights and freedoms.” The letter also outlined various efforts the company would take in response to ongoing anti–police brutality demonstrations, such as endorsing a federal registry for police misconduct.

The news follows extensive efforts by organizers and researchers highlighting how facial recognition can have baked-in racial and gender biases. Notably, in 2018, Joy Buolamwini and Timnit Gebru co-authored a widely influential paper highlighting bias in facial recognition systems. This groundbreaking research showed racial and gender disparities in the accuracy of AI-powered facial classification software, especially for women with darker skin. It specifically featured analysis of disparities in IBM’s technology among systems sold by other companies. Research from the National Institute of Standards and Technology, published earlier this year, has contributed even more evidence that the technology can be, and often is, biased, including based on gender, age and race.

IBM wasn’t super specific in its announcement. However, a person familiar with the matter said that IBM will limit itself to the development of visual object detection and no longer make APIs that can be used to power facial recognition available to outside or internal developers.

KMOAnd skipping ahead a little bit:

Deborah Raji, a technology fellow at the AI Now Institute, noted that the company somewhat quietly removed the ability to do face detection from its API last fall, while cautioning that IBM was far from the most dangerous provider of this technology.

Amazon’s Rekognition

KMOSo I’ll stop reading there, but I’ll summarize a few other pieces from the article. Basically, IBM’s not going to lose much revenue from this, because, as that last bit from the part I read indicated, they’re not really a leading provider of this technology. Tell you who is, though. Yep: Amazon.com. They have a product called Rekognition, with a K instead of a C.

And this article in Vox is from before Amazon made its announcement that it is going to withhold its Rekognition technology from police departments for one year. So before, I was reading from Vox.com; now I’m looking at an article on The Verge. The article is called “Amazon bans police from using its facial recognition technology for the next year,” with the subtitle “The product, called Rekognition, has been the source of controversy.” This is by Nick Statt.

So the previous article mentioned a paper from 2018 by Joy Buolamwini and Timnit Gebru about the biases inherent in facial recognition software. I say “inherent in the software” only because of the data sets on which the software was trained. They featured a lot of pictures of white men and fewer pictures of women and people of other ethnicities. And as you probably know, when you are training an algorithm with large data sets, you’re not giving it specific instructions on what to look for. You’re just feeding it lots of data and letting it make its own connections, and then either rewarding them or disincentivizing them, depending on whether the output is to your liking.

Well, when Buolamwini and Gebru published their paper, Amazon pushed back. Reading from this piece on The Verge:

Amazon tried to undermine the findings, but Buolamwini posted a lengthy and detailed response to Medium in which she said, “Amazon’s approach thus far has been one of denial, deflection, and delay. We cannot rely on Amazon to police itself or provide unregulated and unproven technology to police or government agencies.” Her and Raji’s findings were later backed up by a group of dozens of AI researchers who penned an open letter saying Rekognition —

KMO— again, that’s the Amazon facial recognition software —

— was flawed and should not be in the hands of law enforcement.

What is privacy?

KMOOkay, so I’m not going to [wording unclear; heard “pre-format”] this discussion any further, other than just to demonstrate: yeah, everybody’s going on about Antifa and police brutality and racism right now. And while I find the corporate media’s approach to this nauseatingly self-serving, it certainly has links — they’re not even very long tendrils that connect the different topics — to topics that I think are underappreciated in terms of the impact they’re likely to have on our lives going forward. And already.

So before I play the interview, as I like to do, I looked up the definition of a word. I already knew what it meant, but I couldn’t necessarily provide a definition this succinct right off the top of my head. What is privacy? Well, the official definition is “the state or condition of being free from being observed or disturbed by other people.”

Do we have a right to not be observed? Matthijs Pontier, who is a politician and a privacy advocate and an AI researcher in the Netherlands, thinks that yes, humans absolutely do have that right. And I’ll just say that late in the discussion, Matthijs makes an offhand reference to GDPR, and we don’t stop to define it. That is the General Data Protection Regulations in the European Union. The European Union is much more proactive in protecting the data security and privacy of its citizens than is the United States.

All right. And with those scene-setting comments out of the way, here’s my conversation with Matthijs Pontier.

Program identification: “You’re listening to the C-Realm Podcast. C stands for consciousness.”

Introducing Matthijs Pontier

KMOYou are listening to the C-Realm Podcast. I’m your host, KMO. And to introduce this episode’s guest, I am going to read from an email from Jasper. Jasper writes:

Matthijs Pontier has also been exploring consciousness for some decades, earning a PhD in cognitive science, focusing on emotion recognition in robots. He is an ardent advocate of drug legalization, and he’s been involved in many campaigns on that issue. He has since become a full-time activist and prominent member of the Netherlands Pirate Party. He’s campaigned for privacy rights, against the use of big data for profiling purposes, and for some version of universal basic income. He has also traveled the world extensively and is one of the most open-minded, most adventurous persons I’ve had the privilege of knowing.

KMOWell, with that excellent introduction: Matthijs, welcome to the C-Realm Podcast.

MatthijsThanks.

The Pirate Party

KMOAll right. Well, there’s a lot there in that introduction. I think let’s start with the Pirate Party.

MatthijsYeah.

KMOI’ve surely heard of it, but as far as I know there are no Pirate Party elected officials in the United States, so it’s a very European thing. What is the Pirate Party, and what’s it stand for?

MatthijsI think there are a few on the local level, actually. But it’s a party that started from the pirate movement. If I explain the ideology, I would always say it starts with empowerment — or, if you explain it from the other way around, it’s all about decentralization of power. So as a part of that, you want to protect basic human rights, because basic human rights protect us against power abuse. You also want transparent government, because you need transparent government to be able to control the government. We want democratic innovations, because if the citizens decide for themselves, that also prevents lobbying power from playing too big of a role in politics, as it does now.

And one of the main issues for the Pirate Party is also freedom of information. If you explain it like this: knowledge is power, so you want to share knowledge with everyone, and not, with a patent system, keep it only to some rights holders. Often patents are bought by big companies because these patents attack their business model, so they buy these patents to make sure they are not used. So they were invented to improve innovation, but actually they’re blocking innovation.

KMOSo the Pirate Party — let’s stick with that for just a second. You say it comes from the pirate movement. When I think of anything having to do with piracy, what leaps to mind for me is the logo of the Pirate Bay, which is a way for people to get intellectual property that they haven’t paid for. Is there any relationship between the scofflaws who ignore intellectual property law and the piracy movement generally?

MatthijsWell, they emerge from the same movement, but we don’t agree to everything the Pirate Bay does, and probably also the other way around. Actually, one of the founders of the Pirate Bay didn’t agree to what the Pirate Bay was doing later. So it’s a little bit complicated, but there is a relation — but definitely not an official relation. And we also don’t agree to everything [wording unclear; heard “the Pirate Age”].

Left, right and heterodox

KMONow, I think one reason — aside from the name, which I think would be pretty off-putting for a lot of Americans who are not going to really dig too deep into the underlying principles of a party — in the U.S., the Pirate Party, as you’ve described it, sounds like a mixture of right-wing and left-wing mentalities, which don’t go together here in the United States. You’re talking about empowering individuals: individual freedom, individual autonomy. That’s all very right-wing stuff. That’s all very libertarian stuff that Democrats don’t like. And anybody to the left of Democrats says that people invoking notions of freedom — freedom of speech, freedom of assembly, freedom of association — that’s all very suspicious. They don’t like that. That’s all right-wing stuff.

But I know you also advocate some form of universal basic income, which is anathema to the American right. So it’s the sort of safety-net aspects, which seem absolutely embedded in the European mentality toward government, and the decentralization and empowering of citizens. Those two things just don’t mesh very well in the contemporary American scene. People who would hold conflicting ideas like that here are called heterodox thinkers, and heterodox thinkers are very suspect.

MatthijsRight, yeah. Yeah, I think in the Netherlands it’s a little bit different — maybe also in the US, but you obviously know better than me. The parties on the left, they are a bit more toward supporting basic human rights, for instance. It’s a little bit complicated, because protecting human rights also has this egalitarian effect in a way, usually. But also individual liberty — it really depends, because in the Netherlands the right-wing parties are usually also the parties that are more of law and order. But then again, if I look to the US, from what I see, the Republicans are also more the law-and-order party, and willing to give up all kinds of individual liberties to have this law and order — although it’s actually a false dichotomy. So I’m not completely sure whether I agree to what you said.

But then again, I agree that there’s a weird mix of left-wing and right-wing people that support us. At the Pirate Party, we like to say that the way people look at left-wing and right-wing is a bit outdated, because you should always ask: what exactly do you mean with left-wing, and what do you mean with right-wing? But for us — or at least as I explain our ideology — it all depends on a decentralization of power, and a decentralization of economic power is a part of that, so it makes things more equal. But then again, you should also protect individual liberty, and the one doesn’t necessarily go against the other, I think — although a lot of people look at it that way in traditional views.

KMOSo one thing you said, just in passing — I took it as a courtesy — you said, “you know better than me.” And that’s not necessarily the case.

MatthijsI assume you know better than me, because I watch the news from the U.S. just a little bit. But I assume you know the situation of the U.S. better than me.

KMOWell, I would say 20 years ago, that would have been the case.

MatthijsOkay.

KMOBut now there is such a thoroughgoing divide between the ideological tribes here in the United States, and the two sides relentlessly hurl very ugly propaganda at one another. They’re not trying to convince the other side, and the lines have very much been hardened. And this is aided and abetted by social media companies like Twitter and Facebook, and also by the mainstream corporate news media.

We’ve come to a point in the United States where — I forget who said it, but somebody that I know on social media said: these tools were supposed to allow us to organize and share information, and really all they’ve done is allow powerless people to rage at one another.

And so I am one of those cursed heterodox thinkers. I don’t identify with the red or the blue tribe here in the United States, but most of my friends are blue tribers. And culturally speaking, particularly where I live now — I mean, I live around a lot of poor, uneducated, working-class people, which would be fine if I was in the American South, which is where I’m from, or the American Midwest. But here in the Northeast, it’s just different. It’s a different culture. And I have no seeming point of contact with the people all around me. So it’s a very frustrating situation, in that I live in a community, but largely I am informed by information coming in through channels which are highly distorting, which is to say social media and the corporate news media.

Surveillance capitalism

MatthijsOne of the difficult things — the internet started as a nice place where everyone could share information. But now big corporations and other powerful institutions are using their power to amplify certain sounds in the debates, and that makes it very difficult. Also, with increasing mass surveillance by governments as well as corporations, they gather lots of data about you. So they get to know you really, really well. If you look at certain specific things, they can even know you better than your partner, or even yourself. And they use this knowledge about you to target you with certain information, with the direct goal to manipulate you. This can be into buying something, or getting a service, or it can also be voting for a political party. So that’s really one of the big challenges of this time: how to deal with this, and how to protect people from this manipulation that largely also happens subconsciously.

KMOAre you familiar with Shoshana Zuboff and her book The Age of Surveillance Capitalism?

MatthijsYeah, definitely, yeah. So what I found — this is one of her major points, I think, right?

KMOIt is. Although a piece of it that you didn’t mention is that not only is the information gathered about us, and the profiles that they formulate with this information — not only do they use it to manipulate us, but I think more importantly, from a financial perspective, from the perspective of the companies doing the surveillance, they can make predictions about our behavior, which they can then sell to interested parties.

MatthijsExactly. Yeah. Yeah.

KMOThey’re not necessarily selling something to us based on what they know about us. They’re selling information about us, and predictions about our behavior, to other companies, like insurance companies, medical companies, or all kinds of people who want to sell us stuff.

Insurance and pooled risk

MatthijsYeah, yeah. And then these companies use it to sell us stuff, but they can also use it to not let us take their services.

KMORight — medical insurance in particular, here in the United States.

MatthijsYeah, that is also always one of the most important examples, I think, because the goal of an insurance is to share a risk together. If you individualize this to the max, then everyone is paying for their own risk, and then the whole point of insurance is gone. It leads to very bad situations.

There’s this student that I know here in Amsterdam — somehow, with his case, he gets on the television quite a lot too — but he’s a really good example. When he was already very young, he had a certain form of cancer, and he healed, and then he wanted to have a health insurance. In the Netherlands we have basic health insurance, which is obligatory for everyone, so insurance companies cannot say, “You cannot get this,” because everyone is supposed to have it; every health insurance company is supposed to offer it for everyone. But then there’s additional health insurance, which of course for him was a very smart idea. But every health insurance company, they forbid him to take their additional insurance, because they said, “Well, if we would be insuring you, we would be insuring a burning house.”

KMOThat’s very sensitive and humane.

MatthijsYeah.

KMOYeah. So I just want to expand on what you said a little bit. The whole point of pooling people, having them each pay in premiums for insurance, is to take a population with very unequal amounts of risk and sort of flatten things out, so that healthcare becomes affordable for everybody. And if the healthcare companies have very finely detailed and granular information about the entire population, then they can pick and choose who they are going to insure, or they can customize the insurance premiums based on risk. And then you’ve basically stopped pooling risk. You’re no longer taking advantage of the ability of large populations to pool risk and to help one another.

MatthijsYeah.

The pandemic and function creep

KMOSo let’s talk about the coronavirus pandemic.

MatthijsYeah, of course.

KMOYes. It’s a crisis. And for folks in power, the constant mantra is “never let a good crisis go to waste.”

MatthijsYeah.

KMOWhat opportunistic steps do you see proponents of centralized control taking in this moment?

MatthijsWell, there’s a lot of governments that are starting to have these corona apps, right, to gather location data from people. Well, if you look at that in the context of what we just talked about with surveillance capitalism, then that’s a very big issue. Also, governments are using it to extend other powers. Here in the Netherlands, they’re flying around with drones that are warning people — even in the open air, where the risk is not even that high. It’s very dystopian: there’s drones flying around giving people warnings about not being six yards away from each other.¹

We also have these scan cars. There’s cars driving around with cameras with AI object recognition. They were used for recognizing number plates, so they would automatically give fines to cars that are not parked in the right manner. I already thought these were sort of Orwellian machines, but now, with the corona crisis, they’re applying function creep. So in Rotterdam here, they are using the same cars, but now they’re applying a recognition of whether people are six feet away from each other. So yeah, I think this is also really bad.

In the Netherlands they’re also giving really high fines — 390 euros — if people are too close to each other in the open air, which also will give them a criminal record. So if they want to work in healthcare, they cannot do certain things for a period of five years, because this criminal record is visible for five years in the system.

So that’s one of the things I try to warn about: that governments are using this crisis to expand their powers. And even if it’s only for a limited time period, it can also make us used to this step toward a totalitarian government. And it gives a bad precedent, because if people get used to something like that, it’s also easier to take a step toward that in the future.

KMOSo these contact-tracing apps and other technologies that basically allow authorities to reconstruct somebody’s movements — what this allows them to do is map somebody’s social network. You know who the target individual came in contact with; you know who they come in contact with regularly. But the picture that these apps paint of somebody’s social network is really rudimentary compared to what Facebook knows about your social network, or what Twitter knows about the topics that you’re interested in and your more ideological and verbal behavior — which, going forward, seems to be as important as, if not more important than, people’s physical behavior when it comes to controlling them.

So it seems like the government is playing catch-up with private corporations. But at the same time, the government can simply demand that the private corporations turn over everything they know about any given individual. And it actually streamlines the process for the government: not having to do the research and the calculations themselves, just letting private companies — or publicly traded companies — do that up front, with people’s permission, because it comes with a lot of convenience. And then, when the government needs that data, just strong-arming the corporations.

And corporations come in a spectrum of their willingness to cooperate. At one end, we’ve got Jeff Bezos at Amazon.com, who is the government’s and police officers’ best friend. He is definitely on the side of his customers [wording unclear], and he’s on the side of agents of authority, not so much on the side of citizens. And at the other — I don’t know if it’s the extreme, but toward the other extreme — is Apple. Apple has pointedly refused to unlock iPhones that belong to people who had been murdered, when law enforcement officials wanted to get information about those people’s final hours and minutes, and Apple says no.

And some people are angry at agents at either end of the spectrum. Some people are angry at Jeff Bezos and his willingness to play ball with authority figures, and some people are mad at Tim Cook and his refusal to provide what seems to be reasonable help to law enforcement in very serious situations. I wonder what your position is, and maybe what the Pirate Party position would be, on issues like that.

Corporations, governments and whataboutism

MatthijsWell, we like to protect privacy. It’s one of our most important points. So we think every government should not follow citizens when this is not completely necessary, and for corporations we think the same. A lot of people, when we point to government surveillance, they say, “Yeah, but corporations are doing much worse.” But I think this is a form of whataboutism.

KMOYeah.

MatthijsAlso, with corporations, usually you are participating by choice — although often you can argue how far we have this choice. I tried to get off WhatsApp, but it really gave me so many problems, with not being able to talk to journalists or my family, that I installed it again. So how much of a free choice do you have? Sometimes you really have to make sacrifices if you don’t want to participate in these products or services that kill your privacy.

But yeah, in general, we think companies should always refuse to hand over information to governments unless there’s a really clear and specific goal. And if they want to get information about one person, then it has to be heavy enough, and the requests of the government must also be specific enough. A corporation is not part of the police; they cannot be used for that. I think only when there is a request for a really, really good reason, then of course in some cases it’s good to cooperate. But I think if you look at Apple, they have never refused information when the goal was really important enough, in which case they should.

And also, I prefer services that cannot give information to the government even if they wanted to, like it’s the case with Signal. At some point they got requests to hand over information, and, well, they couldn’t, because they don’t have access to it. And that’s how it should be, I think.

Amazon, Google and the libertarian blind spot

KMOEvery month Google sends me a summary of my physical movements. They know where my phone has been, and they send me a timeline and a map showing me where I’ve gone. The last three months’ reports have been very depressing [wording unclear; heard “very far at all”].

But Amazon.com — I’ve been an Amazon.com customer since 1996. They have a list of all of my previous addresses since 1996, and I have moved a lot. They have a list of all my past credit cards. And when it comes to — I’m self-employed, and here in the United States, doing taxes for a self-employed person is a Herculean task. It requires a lot of time and effort. At tax time, for me, my Gmail inbox, Amazon.com and PayPal are invaluable resources, because I can go back and reconstruct all of my financial movements of the past year, or many years past. So Amazon — while I was just running them down for cooperating with authority figures — they collect a lot of data on me, but they make it available to me. Whereas most of the companies gathering data on me don’t share it with me. They don’t even let me know that they’re gathering information on me.

So my concern, though, is — and I think I mentioned in our email correspondence that I voted libertarian for decades —

MatthijsYeah.

KMO— hardcore libertarian. But I’m dissatisfied with libertarian philosophy now, for a variety of reasons. And one of the [wording unclear; heard “purposes”] is that it thinks that government is evil in everything it does, but that private corporations doing the exact same things that a totalitarian government does are no problem, because we have a choice. We have a choice not to use the products or services. But once we get seduced into using them for whatever reason — and you brought up a great example, WhatsApp. It’s a Chinese app, and any information exchanged via WhatsApp is accessible to the Chinese Communist Party — especially — it’s part of Facebook. Ah, what am I thinking of? TikTok, I guess. Maybe.

MatthijsYeah, I don’t know. You’re right. But WhatsApp is not Chinese, I believe.

KMONo, you’re correct. The main point is that it’s part of Facebook now. Yeah, you’re correct. Facebook bought it, along with Instagram.

MatthijsInstagram, yeah, exactly.

KMOYeah. So the general thrust of my point here is that I think there can be totalitarian forces which are not government forces, and we enter into relationships with these forces voluntarily. But in the end, to me, it doesn’t matter so much how they got our data and how they got to know us so well, whether we agreed to every part of it or not. We can be seduced or lured into very abusive relationships with authority. And authority doesn’t have to be government authority.

MatthijsYeah, exactly. The whole thing, again, is about power distribution. It doesn’t really matter whether you have a really powerful company or a really powerful government. The tendency, I think, if you look at history, is that whenever there’s too much power concentration, it’s used for the worst at one point or the other. Of course, there’s positive examples where it doesn’t happen, but a lot of times it does. When a company or a government gets too powerful, at some point they will start abusing this power for their own good.

Is the cure worse than the disease?

KMOStaying on the topic of the pandemic, there are a variety of points of view. We’ve touched on some of them. One of the points of view that we haven’t mentioned is that this disease is not so deadly that it justifies the totalitarian measures which have been taken to contain it. That’s a very cold, clinical way of expressing something that I see people online saying in much more emotionally laden terms. But basically: the cure is worse than the disease.

MatthijsI think it depends also per country, but I can probably best start talking about the situation in the Netherlands. There, the lockdown wasn’t as bad as in some other countries. But the weird thing was that there were some simple measures, not so invasive, that you could take, that haven’t been taken. While there was a lockdown, everything was closed, and people on the street that are too close together are getting fines — which is really [wording unclear] totalitarian stuff, if you ask me.

And there were some simple things that you could do, like doing contact tracing, but then through humans, at a healthcare institution, and doing massive testing of people that come out of this contact tracing as contacts, and using a lot of protective gear — especially for health personnel, but also for other people that, for instance, have a job in which they are in contact with other people. In the Netherlands, hairdressers are often named, but also physiotherapists. If they wore protective gear, then they could probably just have continued working.

So a lot of simple measures that could have been taken have not been taken, and therefore more people have gotten sick than necessary. At some point, more than half of the infections were caused by health personnel that did go around lots of vulnerable people, and they went unprotected. And if you had taken all these measures to prevent spreading, then a lot of lockdown measures would have been unnecessary.

Death counts and healthy years

KMOI wonder if you’ve seen the meme. It shows a picture of a shark, and there’s a headline. It says, “Man attacked by a shark dies of coronavirus.” Have you seen that?

MatthijsNo, I didn’t see it.

KMOThe point being — I don’t want to be particularly prejudicial; I don’t have opinions that I hold strongly in this matter — but some people think that the reporting of the death toll from coronavirus has been vastly inflated: that people who died from other causes, because they tested positive for the virus, were listed as having been coronavirus victims.

MatthijsRight, yeah. Yeah. Well, for a big part, that is in a way true. I always like to look at the statistic of — in Dutch, oversterfte: more people dying than usual in this time of the year. And that was actually a little bit more than the corona deaths in our statistics. But if you look at which people died from corona, then in the Netherlands the average age was 81-point-something, I believe, and they always had, or almost always had, one or more underlying diseases. So it’s a really relevant question indeed: did they die from corona, or did they die from what they already had, and wouldn’t they have died otherwise anyway in the coming year? And many of them would probably have died in the coming year.

So also, if you look at measures you can take, and whether they are worth it, so to say, then I think it’s always good to look at the number of healthy years that people live extra because of these measures. Does it make sense?

KMOAbsolutely. Yeah.

MatthijsYeah. And then also, if you look at the lockdown measures that have been taken, for a lot of them the costs were amazingly high, and you can ask: were they all necessary? And if we had not taken these lockdown measures, but had invested extra in public health instead, wouldn’t we have saved more healthy years for people?

KMOIn very, very general terms, anytime you’re responding to a crisis — or in medicine, you’re treating symptoms, or you’re responding late in the course of a disease — the benefits you derive from your actions are going to be much, much smaller than [wording unclear] much less attention and much less resources devoted to prevention early on.

MatthijsYeah.

KMOOr, even more simply and in more political terms: it is easier to resist at the beginning than at the end.

MatthijsYeah, that’s true. Yeah, yeah. And also, one part of why it was difficult to respond in the Netherlands was the changes we had in our healthcare system. Because everything was focused on financial efficiency, there was no extra capacity, not of protective gear — a lot of people who worked in intensive care as nurses were fired, so there was a lack of personnel in intensive care. And a lot of things would have gone better if we had had a better healthcare system to begin with — if not so many cuts had been made in the previous years. Then we could have responded way better.

Also with the contact tracing: it’s really still, in a way, a surprise to me that really early in the pandemic in the Netherlands they gave up contact tracing, because they said, “Yeah, there’s too many cases to trace now; we cannot handle it.” If we had invested more in contact tracing, and started immediately with training people to do contact tracing, or instructing people how to make a list of the contacts they had — it’s not really that difficult, in a way; you just make a small diary of your contacts in the past week — that would have led to much better results, I think, in the Netherlands.

Masks and trust

MatthijsAnd one thing in general that I would say: in a liberal democracy, you don’t want to have all these repressive lockdown measures. You want to inform people. You want to tell them, “This is what is going on; this is what you could do to help prevent it.” And then you would see that all people, or at least almost all people, are very happy to protect themselves and protect others if they just know how to do it, and there’s not really any need for enforcing this.

And one good example in the Netherlands is the use of mouth masks. Because they were afraid of a run on mouth masks, they said, “Well, these masks don’t really help you, and they don’t help to control the spread of the disease.” Well, there was so much science that obviously says that mouth masks do help to spread disease.² They do not necessarily protect you, and definitely not 100% if you have basic masks. But if they stop, like, 60% of the spread coming from you, then it already plays a really big role in prevention of the spread for the whole population. So it’s more an altruistic thing than individual protection: if you wear a mouth mask, you protect others from getting sick from you.

And because they have been saying for such a long time that they wouldn’t help, now later — actually just since a few days — they are making them obligatory in public transport. And then also, they’re allowing only — not the best masks. If you wear the best masks, then you can actually get a fine in public transport now, because they want to keep them for healthcare personnel. So with things like this, you really start to lose trust from people.

Well, I think in a liberal democracy you want to show that you trust other people to protect themselves and each other, and you basically help them with making their own well-informed decisions to protect themselves and each other. And if you use repression, or you give information that is not based on science but is based on — well, in this case, the availability of masks in that period of time — then the population will also lose trust in the messages that come from the government. And they will also trust the advice less, and they won’t comply with the measures that, in most cases, of course, are taken with good intentions.

Machine learning and the panopticon

KMOLet’s talk about machine learning and artificial intelligence. In the Cold War, the East German state was notorious for keeping massive amounts of documented information on its citizenry. And not only that, but it had recruited a very large portion of the population to spy on the rest of the population. And this was all paper-based and human-facilitated work. Artificial intelligence and machine learning can create, as you know, a totalitarian or panopticon situation which makes East Germany look quaint. What are your concerns, and what are your proposals for not going there?

MatthijsWell, yeah, of course: basic privacy protection, from corporations and from governments, but also from each other. And the main difficulty is with press freedom, I think, because press freedom is really important, but so is personal privacy. So that’s where the main difficulties are — like, how far do you allow people to film on the streets? Or at least that’s where my personal dilemmas are on this point. But generally, you want good privacy protection to prevent this from happening.

David Brin and The Circle

KMOOkay. So you’ve said the word “privacy” many times now in this conversation. I think it’s a good time to bring up David Brin. David Brin has a book called The Transparent Society, which is several years old now, so in my mind it’s kind of out of date. But he maintains the position that if we pass laws ensuring privacy, that won’t do anything whatsoever to stop powerful actors from spying on normal people. We can pass all the privacy laws we want here in the United States; it’s not going to protect me from the NSA, and it won’t protect you from the CIA. But the privacy laws will be invoked to protect the privilege of the powerful, so that normal people won’t be able to spy on the powerful.

And David Brin’s position is that we should have a transparent society where everybody has access to all of the camera feeds, so everybody can spy on everybody else, and everybody will be held accountable. Even the powerful people will have to account for their actions to the masses. To me, that seems quixotic, but before I give my position, let me offer you the opportunity.

MatthijsWell, it makes me remind of this book, quite recent, by — what’s his name — David Eckhart?³

KMOI haven’t read the book.

MatthijsThe Circle.

KMOOh, yes.

MatthijsThe Circle. And basically, that gives this image of a society which is completely transparent. I think they use it quite literally in the book, and that really is a dystopian story. If everyone can watch everyone, it can lead to very bad things. In this book there are a few examples: people are accused of doing something bad, and then there’s public witch hunts in which, I think, people are lynched at some point. And also with political decision-making — because this tech company knows everything about everyone.

And it’s not just about having access to this information. Only the biggest companies have the power to extract certain information from all this data. So you still have this problem with some organizations having more power than others.

When you have no privacy at all and you’re watched all the time, it really changes your behavior. There’s also [wording unclear; heard “the friends”] — it was shown that after the Snowden revelations, if you just look at internet searches, then after the revelations there were fewer searches for terms like “whistleblower.”

KMOHmm. I would think there would have been more.

MatthijsInteresting, yeah. Yeah, so it was really interesting. People knew that what they did online was being watched in one way or the other, so they started to change their online behavior because of this.

KMOSticking with The Circle for a minute: I didn’t read the book, but I did see the terrible movie with Tom Hanks and Emma Watson — Hermione from Harry Potter.

MatthijsYeah.

KMOShe agreed to be completely transparent and to share her life with the world as part of this pilot project. But there were some barriers: the cameras couldn’t follow her into the toilet and watch her use the toilet. So she started doing more and more things in the toilet; that was her reprieve. And I could imagine a scenario where we have a supposedly transparent society where everybody is on stage and behaving well most of the time, but they disappear into the toilet for long, long periods, and they do more and more of their activity in there, just because that’s the one place where they’re not observed. And this might, over time, affect architecture — affect the way houses are designed, to make —

MatthijsThey would design really big toilets.

KMOIt’s where you go.

MatthijsYeah. And also, one thing that, at least in my mind, makes it pretty clear why we wouldn’t want this is that it’s basically a panopticon, right? With the panopticon, the idea is that if people are being watched all the time, they will probably behave well. But it’s a prison. And if we start to make society like that, we’re basically making society like a prison. And I don’t want to live in a prison. I want to live in a free space.

KMOOh, I don’t remember the celebrity — oh, I think it was Ellen DeGeneres — who, when she was in quarantine in her fabulous mansion, was complaining on a live stream that it was like being in prison.

MatthijsYeah, yeah.

KMOWhich I think for most people was rather offensive, because she’s living in luxury. She’s living much better than most people. And particularly here in the United States, prison, or any interaction with the criminal justice system, is a very humiliating, dehumanizing experience, and Ellen was definitely not subject to that.

MatthijsNo, no, no, that’s true. There was also a TV show by some people that started filming their life, all day, every day, as an experiment, and it also had a really, really bad psychological effect on them.

Intelligence agencies, law enforcement and drug policy

MatthijsBut back to machine learning and AI. So one point that maybe I want to make about what you said before — because at this point you said, “Yeah, the secret services, they can still do whatever the heck they want,” right? And that’s also true. We had a referendum on a dragnet surveillance law that gave a lot more power to the intelligence agencies here in the Netherlands. And insiders also said, “Yeah, intelligence agencies are breaking the law about what they should and what they shouldn’t do all the time.” But then again, if they do break the law, they cannot use the information they received in that way in court. So it still makes it less valuable to do all these things that they are doing anyway in a certain case. So it does help a lot.

And also, only the intelligence agencies can do that. Now basic law enforcement is also using all this same data, and these are different people with different goals, also with a different education, and they’re using the same information for really different goals. And that makes a very big difference in practice for people, because most people are not really of interest to the intelligence agencies, but they are to law enforcement, or they are to insurance companies. Maybe for most people the intelligence agencies are not the biggest risk. Of course, for some they are. For instance, democracy activists in countries that aren’t free have a lot to fear from intelligence agencies. But a lot of people also have to fear insurance companies and law enforcement.

And also, a lot of times they have to fear law enforcement for crimes that are not violent and not actually bad to society. A lot of activists have to fear law enforcement because they just want to stop their cause. Or [wording unclear; heard “they have also said something in the things about drug policy”] —

KMORight.

MatthijsI don’t think producing or trading drugs is bad per se. But still, a lot of people that do no harm to anyone have to fear a lot from law enforcement.

KMOWell, that is definitely true. That’s what drove me into the arms of the libertarians all those decades ago: my objection to the drug war here in the United States, which was so abusive, and remains so. Marijuana is legal in a variety of places, but lots of other drugs which are popular with people are not. And just legalizing marijuana doesn’t really diminish the power of the police, particularly here in the United States, to stop and search anybody at any time for really no good reason. Just the existence of illegal drugs makes everybody a suspect.

MatthijsYeah, yeah.

KMOWhen it comes to intelligence agencies versus law enforcement — yeah, you can say to law enforcement, “If you collect information that you want to present as evidence, but you didn’t abide by privacy restrictions, then that evidence is inadmissible.” But with the intelligence agencies, their primary objective is not to secure a criminal prosecution. In many instances, they just assassinate people or blackmail them. The CIA, for example — it’s well known that the CIA ran heroin operations during the Vietnam War, and in the ’80s they were facilitating the transportation of cocaine into the United States.

MatthijsYeah, they do this too. And there’s also economic warfare between countries that is supported by intelligence agencies, right?

KMOSo just saying that failing to protect people’s privacy in law enforcement will hamper law enforcement’s ability to secure convictions — it doesn’t seem like much of a shield to me when it comes to protecting people from the abuses of power.

MatthijsStill, there’s also activist groups being subject to surveillance by intelligence agencies, for instance, and terror laws are being misused, also against people that are activists, usually.

KMOYeah. Yeah. So I think these are specific examples of the general idea that tools that are introduced for one purpose will later get repurposed for activities which we would not have approved of in the first place. But by the time this repurposing happens, we’re basically up to our necks in quicksand.

MatthijsExactly. Yeah. Function creep.

KMORight. And particularly when you introduce machine learning, and you have basically automated cognitive tasks so that you can perform things with a very small staff that previously would have taken an army of people filling out paperwork and operating machines and sharing data and collating data and synthesizing data — if you can automate that and then just really increase the throughput of information, your function creep can turn bad quickly and on a grand scale.

Transparent, explainable algorithms

MatthijsYeah. I think one of the important things we can do to prevent this from happening is to demand a certain ethical standard for algorithms in general, and democratic control over the ethics that we give to these algorithms. So, by these standards, every algorithm should be fully transparent and explainable — also the algorithms by corporations. For instance, the Facebook algorithm that determines what you get to see in your timeline: you should be able to look at how this works. It should be well explained, so everybody can understand it, and not just some AI nerd. And I think you should also have a certain right, in many cases, to change this algorithm. And, like I said before, I think every algorithm should comply with certain ethics that we decide together. So there should be a democratization of technology.

KMOYou know, I think of something like Linux: open-source project, totally transparent. Anybody can dig into it. I don’t have the skills. I mean, you could tell me that a certain line of code in Linux is a totalitarian algorithm, and I couldn’t assess it.

MatthijsYeah, so that’s why only open source is not enough. The points of explainability and also accountability should be introduced next to that.

KMOSeems like a tall order. How do you get somebody like me, who’s interested in the topic but doesn’t have the background or the skills to get into code and look at actual lines of code and determine whether or not they support transparency and freedom, or whether they are corruptible and susceptible to totalitarian mission creep?

MatthijsYeah, so general people that are not into coding shouldn’t even need to look into the code, but just into the explanation. And then there would be an army of knowledgeable volunteers to make sure that the explanation is actually in accord with the facts. Yeah — because the code is open and the explanation is open, everybody can check whether the explanation matches the code. And everybody who wants to look at what this code is actually doing can look at the explanations. So you need a small group of knowledgeable nerds that are willing to do this, and then the general public can look at what these algorithms are actually doing, and whether we want that.

And also, in many cases there should be an office. If you decide together about certain norms — so for instance, you don’t want algorithms to be able to discriminate by ethnicity, explicitly or implicitly, by looking at a bunch of other variables so that you’re actually still discriminating against people from a certain ethnicity — if you decide this together, then I think in a lot of cases there should be a team with knowledgeable people that audits these algorithms, to make sure that they comply with the rules or norms that we decided together as a society.

Paperclips and burning buildings

KMOYou are 35, is that right?

MatthijsYeah, that’s right.

KMOI’m 52, so I’ve been observing trends a little bit longer than you. When I was in grad school in the ’90s, I was studying philosophy, and my area of emphasis was philosophy of science and philosophy of mind, and I was very interested in artificial intelligence at the time. But when I was thinking about it, I was thinking about it in the context of the philosophy of mind: what does it take to create consciousness, self-awareness — to take something which is not alive and is not aware of its surroundings and turn it into something which is aware of its surroundings, aware of itself, aware of its interactions with the world, and is in some sense alive? That was my interest at the time. But the intervening decades have made me realize that what’s really dangerous is algorithms that can perform cognitive tasks but which have no sense of self, no idea at all what they’re doing, much less any ability to take an ethical stand against being misused.

MatthijsThat’s exactly right.

KMOWell, let me have you elaborate on it, then, rather than me just talking.

MatthijsOh, well, yeah. In presentations on machine ethics, I sometimes use this example of the paperclip machine. In this thought experiment, there’s a superintelligent AI, but people are not allowed to attach it to the internet to get lots of information to use with this intelligence. And they have a machine that makes a lot of paper clips. So it’s intelligent in a way that it can produce lots of paper clips, and it has only one goal: produce as many paper clips as possible with as few resources as possible. And then, because there’s heavy competition, this one company says at some point, “Well, we’ll attach it to the internet just for one night. What’s the worst that could happen?”

And then the next day they come back into the office, and everything smells a little bit weird, and within a few moments everyone’s dead. It turns out the paperclip machine has found a way to use molecules in the human body to make loads of paper clips. And then it’s still connected to the internet, so it gets really smart and intelligent, and it just uses everything it finds in the world — destructs it to make a lot of paper clips from it, to make extra paperclip machines. And then it ends up with paper clips being made everywhere, and they’re invading the whole universe, building huge stacks of paper clips in the whole universe.

You don’t really need to program a bad goal into a machine to make it behave in a way that you really don’t want. And it doesn’t have to be self-aware. It’s just following the goal to make loads of paper clips, and it ends up killing everyone.

Another example that I like, just to show that it’s really difficult to give a clear instruction to a machine: if your grandmother is in a burning building, on a high floor, and you say to a robot, “Get my grandmother from the burning building,” the easiest solution is probably to push your grandma from the burning building. And it complies with your instruction, but probably that’s not what you had in mind when you gave the instruction, “Get my grandmother from the burning building.” And if you really think about how to give this instruction well, then it actually turns out to be pretty difficult to do it in such a way that the machine really understands what you want it to do.

KMOSo one way around that is to have a strong AI, and you just tell the machine in normal human language what it is you want it to do. And it, with its full understanding of human psychology and implications that you’re not aware of, will tell you whether or not that’s a good idea — or, even better from its perspective, just go and do something that isn’t what you asked for, but which gets you something that is better than what you asked for. That’s sort of the benevolent, utopian vision of what AI could do for us.

But at the same time, I don’t want my coffee maker in the morning examining, with philosophical axioms, whether or not it is appropriate for it to serve me, and whether I should be drinking decaf instead of regular coffee, or communicating with the fridge and deciding to maybe let the milk go bad because I shouldn’t have so much dairy in my coffee.

The Internet of Shit

MatthijsYeah, yeah, that’s the whole other thing: what do you want machines to do, and what don’t you want them to do? I’m not sure — do you know this Internet of Shit on Twitter? Maybe also on other social media. It’s really funny. It’s a big compilation of machines that have been made “smart,” so to say. But “smart” basically means connected and vulnerable, so it can be hacked, and it can be misused in all kinds of ways. And there are so many things that are connected to the internet for not really a good reason. There’s a connected cookie oven; there’s a connected toothbrush. And if it’s connected, well, it can be hacked.

So if you have a smart house with a door that is connected to the internet, there could also be a hacker, or there could be some ransomware that says, “Yeah, if you want to enter the house now, first you need to transfer so many dollars of some kind of cryptocurrency to this address, and otherwise we won’t open your door.”

Doorbell cameras and fear porn

KMOYou know, speaking of the front door: Amazon bought a company — oh, I forget the name of the company — but basically it’s a doorbell camera, and it monitors your front porch, so if somebody steals your packages, it can — but people have figured out — I think Amazon intentionally allowed them to do this — various people who live near each other, who all have the same service, can share their camera feeds with their neighbors.

MatthijsYeah.

KMOThese, you know, cybernetic neighborhood watch groups.

MatthijsYeah.

KMOWhich — yeah, with their neighbors and with law enforcement.

MatthijsYeah. And also, I think in the Netherlands there have been — or there are maybe some ongoing — legal fights about whether it is allowed to have a doorbell like this if it’s also filming the streets, because you can only film on the street for a good reason in the Netherlands — I think also in the rest of Europe, because of GDPR.

And one thing, I think, with these Amazon Ring doorbells is that, as a promotion of this, I think they set up a complete media channel that is full of fear porn. And of course everyone gets afraid, and then they’re basically advertising these doorbells to give people a certain false sense of safety again.

KMOThank you for introducing the term “fear porn.” So much of what I’m doing with my podcast right now is trying to make amends for having peddled collapse porn for years.

MatthijsRight.

KMOAnd yeah, I’m an anti-catastrophist at this point. This is a strange moment in which to try to hold the anti-catastrophist line, because it’s very easy to spin the current situation as the collapse. But I don’t think it is. I certainly think that it’s a moment of great import, and that how we respond to it will have lasting effects going forward. But I think it’s just really simplistic and irresponsible to say, “Oh, this is the end. This is it. We’re done.”

MatthijsYeah, yeah.

Wrapping up the conversation

KMOWell, hey, we’ve been on for a little over an hour. I very much appreciate your time. I hope we can have a follow-up conversation sometime, but thank you very much for appearing on the program.

MatthijsYeah, thanks to you for the invitation. And yeah, [wording unclear; heard “and our time”] sounds good. There are still so many things that we didn’t talk about, so I’m happy to be [wording unclear; heard “turning in”]. And in fact, it was just a scratch at the surface on each topic we did talk about.

KMOYeah, yeah. All right, well, thank you very much, and stay well.

MatthijsYeah, you too.

Program identification: “On the C-Realm Podcast: A mind that is stretched by a new experience can never go back to its old dimensions.”

Closing and sign-off

KMOAll righty. That was my introductory conversation with Matthijs Pontier. I hope you enjoyed it, and I hope that you didn’t sit through it just hoping to get to the epic rant at the end of the podcast, because there will be no rant in this episode. If you want to see and hear me rant, check out my YouTube channel. Or there is always the C-Realm Vault podcast, which features a lot more solo material than the main C-Realm Podcast, which is more interview-focused — although mine is surely not the only voice you will hear on the Vault. The only way to subscribe to the C-Realm Vault these days is through Patreon. My Patreon page can be found at patreon.com/KMO.

And the other day I put up a YouTube video, and it was up for less than an hour when suddenly I realized: you know what, there are some fairly sensitive topics that I’m discussing in this video, and it could just be a pain. It could be a headache for me. I’m going to make this an unlisted video and just post a link to it from my Patreon page, and make that link only available to people who donate $1 a month or more. So if you check out my YouTube channel and you like what you see, but you think, “KMO, you’re playing it kind of safe” — well, you’re not seeing everything there is to see. But you can, by subscribing via Patreon.

All right, that’s just about all for this episode of the C-Realm Podcast. If there are topics you would like for me to look into and talk about on a future episode, let me know. You can reach me via email, kmo@c-realm.com, or you can post a comment over on Patreon, or you can post a comment to this episode on C-Realm.com. I’m also on Twitter. To find me there, you have to spell out KMO phonetically: K-A-Y-E-M-M-O. KMO.

All right, I’m out. Stay well.

Editorial notes

1. As spoken (“six yards”); a moment later the speaker says “six feet.” The Dutch distancing rule at the time was 1.5 metres. ↩

2. As spoken. From the surrounding sentences, the speaker means that masks help limit the spread of disease. ↩

3. As spoken; the speaker is reaching for the name. The Circle (2013) is by Dave Eggers. ↩