Shoshana Zuboff on surveillance capitalism | VPRO Documentary

Harvard professor Shoshana Zubov is sometimes called the Karl Marx of our time Her monumental book the age of surveillance capitalism exposed the dubious mechanisms of our digital economy According to Zubov our personal and private experiences have been hijacked by Silicon Valley and used as the raw material for extremely profitable Digital products the term surveillance capitalism is not an arbitrary term Why? surveillance because it must be operations that are engineered as undetectable indecipherable cloaked in rhetoric that aims to misdirect Obfuscated and Just downright Bamboozled all of us all the time Connecting them don't be evil connecting them humanity. Don't be evil Humanity humanity, it's about empowerment of the individual.

The future is privately What really happens with your Facebook photos? Why are there hidden microphones in Google nest and Pokemon go exposed as child molesters in This episode of backlight Shoshanna Zubov reveals how Silicon Valley deceives us. So well How all this comes into play when I Buy a pair of shoes on the Internet well You know some people Will say to me But Professor Zubov, I like those targeted ads There's they're so useful or I enjoy personalized services sometimes people will say You know, I have nothing to hide so I don't care what they take Each one of these statements is a profound Misconception of what's really going on?

We think that the only personal information they have about us is what we've given them and we think we can exert some control over what we give them and therefore we think that our Calculation our trade-off here is is something that is is somehow under our control that we understand it what's really happening is that we provide personal information, but the information that we provide is The least important part of the information that they collect about us Thanks to their navigation and search engine Google knows where we are all the time and what we think Facebook knows our hobbies preferences and friends Because they retrieve a lot of information from the digital traces we leave behind unwittingly Spelling errors in your search terms which color buttons you prefer how fast you type how fast you drive? residual data Way back at the beginning Back in the year 2000 2001 2002 back in those days These data were considered just extra data. They were considered waste material and people called them things like digital exhauster or data exhaust Eventually, it was understood that these so-called waste materials harbored these rich predictive data The search information we retain we do for quality purposes So for example the google spell checker our did you mean feature that appears on google has been built using?

long periods of data around you know Someone issuing a query and then issuing another corrected query rate after that and us learning Those Corrections and it actually takes more than 30 days worth of data to build the world-class spell corrector that we have The companies like to say we collect data so that we can improve our service and that's true They collect data and some of it is used to improve the service to you but even more of it is Analyzed to train what they call models patterns of human behavior So once I have big training models, I can see how people with these characteristics typically behave over time and That allows me to fit your data right into that arc and to predict what you're likely to do Not only now but soon and later This is what I call behavioral surplus these data streams filled with these rich predictive data why surplus Because right from the start these were more data than was required to improve products and services Once you have the behavioral surplus the comprehensive behavioral data of hundreds of millions of people You can start predicting the Preferences of specific groups Think popular shoes for mail managers or the preferred restaurants of a group of people sharing the same zip code Maybe they do a prediction of where I have dinner tonight. Well, how can I imagine what they will predict about me? Well, you know at the simplest level They may predict The kind of food you're in the mood for right now And then sell that prediction Auction that prediction to their business customers in the in the restaurant business who will then?

Send you a very quick ad we know you're in the mood for a delicious pasta dish tonight We can invite you to our restaurant. Here's a discount coupon There are some people who are saying well It's very improbable that those targeted ads really can achieve something because people have their own will and they will not Buy some shoes just because there is a net in front of them You know, I think one of the One of the misconceptions it's really important for us to move away from is that Surveillance capitalism is something that is only Manifest in our lives when we're online Or somehow it's only restricted to online targeted advertising It's easy for us to say oh these these things don't affect me. The fact is this is being conducted at a layer that is not accessible to us We have no idea what today's algorithms can predict about us or what behavioral data they used to do it a Simple thing like buying a certain kind of shampoo can divulge essential information about us?

For example, the New York Times reported a case of a supermarket chain The new a girl was pregnant even before she did who was prepared to share the news The markets algorithms discovered that the girl switched from fragrant shampoos to more neutral smelling products Since the olfactory senses of pregnant women becomes stronger the market algorithm assumed this girl must be pregnant father didn't know until He was repeatedly sent special offers for baby products Thanks to the analysis of trillions of terabytes of behavioral data that we unwittingly leave around the digital domain Big tech sometimes knows us better than we know ourselves they can predict things like our personality our emotions our sexual orientation our political orientation a whole range of things that we never ever intended to disclose The predictive value the big tech can glean from residual data is huge the family photos We post on our Facebook pages contain residual data from which vast amounts of valuable knowledge can be distilled let's say I put my children's birthday party photo album on the web on the Facebook pages What we don't understand is that the most important thing there are not the photos per se It's the predictive signals That these companies can lift from the photos. It's not just my face but it's Allowing them to have the face so that they can analyze the hundreds of muscles in my face Uploading innocent snapshots on your Facebook page can have unforeseen consequences our faces for example are used to train algorithms to recognize facial features and we have absolutely no idea what that Facial recognition software is used to all these data streams with these rich predictive signals are fed into the new factories the computational factories analyzed for predictions of human behavior and These predictions are then sold Who are they sold to they're not sold to us. We are not the customers.

They're sold to businesses to business customers who want to maximize our Value to their business, whatever it may be They use information from our faces Which we've given billions and billions of photos to Facebook to train models for facial recognition those models are then sold to military operations some of them in China and Those Chinese operations do many things including imprisoning the Uighur, a subset of the of the Uighur Muslim population in China in what is Rightly regarded as an open air prison Where they actually don't have to have people behind bars because they track and follow them constantly through facial recognition The knowledge obtained from our residual data can be sold to anyone Facial recognition software for instance might be sold to a Chinese company that supports the oppression of the Uighurs in China or that helps track down advocates of democracy in Hong Kong that way our precious family photos might be used by Facebook to facilitate authoritarian regimes and Our privacy is guaranteed Because it's not our faces that are sold its the residual data scraped off them It's very difficult to have a concept of this for a for a very good reason It's not because we're stupid It's because these processes have been disguised They operate in stealth They have been engineered to be indecipherable to be undetectable to Create ignorance in a vast group of all of us that they call users Our ignorance is their bliss There are some things that have broken through into the public view that we do know about so let's talk about Facebook's Massive scale contagion experiments and this is where Facebook experimented with subliminal cues planted in its Facebook pages That would actually influence offline real-world Behavior and emotions to see if they could make people feel happier or sadder Using subliminal cues in with language manipulation and word manipulation and so on well when the Experimenters wrote up this work in the very prestigious scholarly journals that published the results of these experiments They emphasize two key findings number one We now know that we can manipulate subliminal cues in the online context to change real-world behavior or Real-world emotion. We know that we can be successful at doing this Number two, we can exercise this power these methods While bypassing user awareness A big group of Pokemon Go players set up a meeting in Leeuwarden to stroll through the city to catch as many Pokemon as they can. In the Netherlands there are already about 1.

3 million players. very interesting experiment occurs in the guise of an augmented reality game called Pokemon Go. In the game you are walking around in the real world.

The Pokemon are hiding in different places. You have to go to these places to collect them. What are we not seeing when Pokemon Go is introduced in our country it's important to understand that Pokemon go is an augmented reality game that was Incubated developed inside Google for many years Google being the first pioneer of surveillance capitalism the inventor of surveillance capitalism It was invented Pokemon go was invented in Google incubated there for many years developed there led by a man named John Hanke and John Hanke invented an operation called keyhole which was bested by the CIA and later purchased by Google and Called Google Earth Was a CIA startup initially?

Yes. Google Earth was something called keyhole and it was a CIA invested startup So it's important to understand that Pokemon go was not Some happy little game that just got launched into the world via a toy company or something When they Decided to bring Pokemon go to the public. They didn't want to bring it as a Google game They brought it to market as Niantic labs which no one had ever heard of Just a cool startup with this cool game So now we have this Google augmented reality game and it turns out that the big game that is on top of the little game that the children are playing is a game that precisely emulates the logic of surveillance capital capitalism So in surveillance capitalism in the original version online We predict the click-through rate and We sell the click-through rate to the advertiser Who pays to get clicks on on their on their website?

Clicking through ultimately to the Buy button. That's what they're hoping for Now in the real world Business customers paid Niantic labs the Pokemon go company paid Niantic labs Not for click-through, but for the real-world equivalent of click-through which is called footfall To actually get real bodies with their real feet into real business establishments So that their feet would go fall fall fall fall tap tap tap across this store or across the restaurant or across the bar in order to buy Something so I could order by Niantic labs Pokemon in my ice cream parlor for instance all of these establishments. They're buying what are called.

Here's the term lure modules Modules like a gym that lure people to you Not so that they will come and be happy but so they will come and spend money in your shop or your restaurant Starbucks Everybody was making money everybody was making money Niantic labs was making money and all these businesses were making money and the People playing the game had no idea so they used the rewards and Punishment of the game To heard you through the city To the places that were paying for your body this was the game of Pokemon go this was the real game the shadow game getting you Into a place where we have predicted that you will be So that our predictions are worth more if I can guarantee You're actually going to be there my prediction that you're going to be there is worth a lot more Economies of action are how I guarantee that and Pokemon go was a large-scale experiment a global scale experiment in economies of action using remote control means to automate behavior to engineer behavior to fulfill others commercial ends while You are having a great time You are intended to be in the feeling of being served You are intended to be saturated with convenience so that you will not notice and you will not complain and all of this shadow operation will remain Hidden because you will not ask questions Because you're so busy being entertained So it's no longer enough to just have what you're doing online you're browsing you're sending messages you're sending emails We want to know about your walk in the park. We want to know about what you're doing in your car We want to know about your home and what you're doing in your home What if home security was different? What if it looked different?

Well, it's a beautiful system Some bright engineering oriented person Discovered that the nest security system has a microphone built into it that Microphone does not figure in any schematic When you buy the security system and it has the piece of paper that you unfold and the schematic or you go online To learn about it and there's a schematic it does not show a microphone it Does not discuss a microphone Now, why would you have a microphone there? Well, remember what is our business? our business depends upon Extraction of behavioral surplus scale scope and action So what better device to extract?

behavioral surplus, especially new forms voices conversations What you're watching on television what you're listening what music you're listening to? Who's coming in and out of your house? Whether or not y'all you're shouting at each other over the breakfast table All of this has tremendous predictive value Voices are what everybody's after just like they're after faces So now this becomes public there's a hey Google What's up with this?

Microphone in your security system. What does Google say oops? So sorry.

Oh we didn't even know there was a microphone there. Oh Sorry, sorry why somebody put that microphone there, but we never intended to use it This is their business To obfuscate to misdirect to engineer our ignorance with mechanisms and methods that are undetectable and indecipherable and if they confront you deny, it deny it for as long as possible until they habituate if There's some element that fails to get habituated then create an adaptation All right. We'll make sure all those microphones are Are somehow You know shut off so that they can't be used and Then wait a minute and ones no one when no one is looking Redirect it.

So there will be a microphone in something else. There will be a microphone in the You know in the home device or in the music player or whatever. It might be it will be back What if the measure of it working was that you never had to think about it There's two legal scholar said there at the University of London and They analyzed the privacy policy how we give our consent to these devices, right?

So they analyzed these documents for one nest thermostat What happens is the thermostat? collects data it sends those data to third parties and those third parties send data to third parties odd infinitum an infinite regress or your data is going goodness knows where and No company takes responsibility For what the third parties that it's sending your data to? May do with your data Nest will say if you don't want us to take your data and you don't want us to send it on to third parties that's ok, but be aware that Without your data we will stop Supporting the functionality of your thermostat.

We will stop upgrading the software be aware that The smoke detector may no longer work Be aware that the the pipes in your home may freeze. So now the functionality of the device is held hostage to your agreeing to the privacy contract and They say by the way even if you agree and we maintain the functionality or sending it to these other third parties and They're going to use it the way they choose and we take no responsibility for what they do with it All right so now these two scholars do an analysis of one nest thermostat and What they conclude is that given these arrangements any self-respecting consumer anyone who's even a little bit vigilant about their consumption habit should review a minimum of 1,000 privacy contracts In order to install just one single nest thermostat in your home You have three minutes to exit What if it gave you time And what you really need from home security a sense of security So they start extracting data at scale First it's everything you do online and then pretty soon they need more data But then they also discover that We need varieties of data. It's not just volume, but it's different qualities of data The problems in mobility are going to be the same as what we saw online with the smart phones that you are paying with your privacy without you knowing.

Because the car knows exactly what you are doing, where you are going and any modern car will have about 15 cameras at a minimum and even if you have access to only 1% of all cars you know what's happening all over the world. If the data is going to be more worth than driving the car who knows, you might offer you a free car. You pay with your privacy in the same way you pay with privacy for all the free services you get from Google Well surveillance capitalism broke through in Silicon Valley 2002, three, and four it changed the bar for investment Now you had Google making money on the basis of what I call the surveillance dividend Now what in what venture capitalist wants to invest in a firm?

That's you know, just making an app When it can invest in a firm that's making an app plus a surveillance dividend. So what happened right away? Was that the investment started flowing to the people who were making more money and that?

more came from the surveillance dividend and now that is the structure that is flowing across the whole economy Why bother to put all that effort into the engineers and the factories and the science? to make a car that runs without carbon when all we need to do is sell the data from the darn vehicles and we've got the surveillance dividend and all those investors are going to come to us and give us the kind of Market capitalization that the leading surveillance capitalists have that's our new lease in life That's our path forward in the 20th century The data is worth a lot that everyone is asking for it. That's why Google want to get their products in the car not because you can make a lot of money with a car business.

You can use the car as the Trojan Horse to collect all that data. We even now see it coming full circle back to the birthplace of mass production in the Ford Motor Company Where the CEO of Ford Motor now says, you know, we want market capitalization like Facebook and Google How do we get it? Well instead of figuring out how to make a car that runs carbon-free what we're gonna do is repurpose our automobiles as surveillance vehicles, and we're going to stream data from the 100,000 people who are driving around in Ford cars we're gonna combine that with the data we have from Ford credit where he says, we already know everything about you and These data sets are gonna put us on a par with the likes of the great surveillance Capitalists and who wouldn't want to invest in Ford Motor under those circumstances?

The primary economy what we perceive as primary economy build a nice car grow some nice food build a house Means zilch in this surveillance economy. So this began with Android Once Google acquired the ability to make the Android phone There were a lot of people in Google who said oh, this is fantastic now we have a phone like Apple and we can sell a phone and we can make a great margin on that phone and That margin is gonna fund our profits and this is this gonna make us really rich but the the wiser heads in Google prevailed the people who already Understood surveillance capitalism prevail and they said just the opposite We want the phone and everything associated with the phone to be as cheap as possible In fact if we can get the price to zero, that's even better We want everybody to have a phone because the more they have the phone and the more they spend time on the phone The more data we get from them the more behavioral surplus streams into our supply chains So if we can give it away, we're gonna give it away Google's free mobile operating system Android means Google holds the key to almost 90% of the world's smartphones in Order to obtain as much data as possible from all these cell phones, Google Experimented with Network balloons in those parts of the world where mobile internet is not available Facebook not to be outdone Flew Network drones over growing markets and offered free internet in combination with the Facebook app And connecting them represents one of the greatest opportunities that available to humanity today Hi mark, why are you showing so much interest in India, answer honesty? our mission is to Give everyone in the world the power to share what's important to them and to connect every person in the world Aware of the potential dangers of American data robbery India politely declined the offer and may do without Zuckerberg's generosity In other parts of the world Facebook's true intentions are becoming clearer every day, but we need whistleblowers for that Facebook is happy to take our data but not prepared to share information on how the company works So this is a document written by Facebook executives in Australia and what they told their business customers there was that we have so much data about 6.

6 million Australian young adults and teenagers as a result of that we can predict mood shifts We can predict when they feel stressed fatigued anxious inferior frightened all of these kinds of very personal feelings and We can alert you The exact moment when they are most likely to need a confidence boost, let's say there's a young person who's contemplating a date over the weekend and it's now Thursday night and their Anxiety is peaking and they need a confidence boost if you send them an ad for a sexy black leather jacket Send it right now offer free delivery Tell them you'll have it at their door by the time they wake up in the morning. Give them a discount coupon Right, you're gonna sell that black leather jacket do it now We can tell you the exact moment when they are at peak vulnerability That is real that's happening and if you don't believe me, all you have to do is just pivot a few degrees just something that happened in 2018 And what is that? That came from yet another whistleblower.

His name was Chris Wiley and He told us about Cambridge Analytica Chris Wiley a former employee of the British company Cambridge Analytica sounded the alarm about the methods used by this political marketing business Cambridge Analytica used the Facebook data of more than eighteen million Americans to analyze the best ways of manipulating American voters One of the things that Chris Wiley said when he broke this story with the Guardian back in 2018 Is he said we knew so much about so many individuals that we could understand their inner demons and We could figure out how to target those demons how to target their fear how to target their anger how to target their paranoia and With those targets we could trigger Those emotions and by triggering those emotions we could then manipulate them into clicking on a website Joining a group Telling them what kind of things to read telling them what kind of people to hang out with even telling them who to vote for? now that is absolutely No different Than what Facebook aim to do with these young people innocent young people in Australia and using in New Zealand To target their inner demons the same mechanisms the same methods only Pivoted just a few degrees from commercial outcomes to political outcomes Cambridge Analytica was nothing but a parasite on A huge host and that host is surveillance capitalism Hi Ella, hope you're settling in okay You know how I always say everything you need is already within you The truth is there's only so much you can do by yourself We all need people to get where we're going ask yourself who you're going to do it with Everyone Welcome to f8 Today we are going to talk about building a privacy focused social platform So that's why I believe that the future is private So we all good then professor Zuboff? Well in June 2019 Which happened to be just a couple of months after that That grand announcement that the future is private there's a very interesting court case that was being tried before a California judge and This court case was a class-action suit brought by individuals who were demanding from Facebook They wanted to have the data that Facebook had made available To Cambridge Analytica To see if they had been Targeted and manipulated by Cambridge Analytica and this had come to a head in a California Court the Facebook counsel who argued this case Stood in front of the judge and said that any Facebook user gets on the platform and typically is sharing information with 100 or more people in their network once you've done that You have no expectation of privacy that would make you legitimately able to claim any kind of privacy right in a suit like this or any other kind of privacy oriented suit and So here we have Again, the public operation and the shadow operation What's happening backstage is a lawyer standing in front of a judge saying no Facebook user has any legitimate expectation of privacy This is the the next chapter for our services You'll have some Google products you use if I may ask, no, how do you survive?

Perfectly well if there is an occasion where I feel the absolute need to use Google I go through various layers of encryption And location reconfiguration for I go to Google search just on principle There is a way to escape the omnipresent eyes of Google and Facebook you can use a VPN and send all your data traffic through a secure server in another country, but Big tech won't lose any sleep over individuals resisting the surveillance capitalists. I Don't want anyone to get confused that if you forego Google search you are somehow, you know fighting the fight Because this is a collective problem and it requires collective action With us Amish we are dependent on public transportation for any distance and Local travel we do in our slow way and I think the horse and buggy slows us down and really helps the overall value that we're trying to create and preserve and not get carried away Regular computer with your safety walls, you know where you can't get on porn sites and things like that now, When our lock downs that we have that is totally locked down And you go on the internet with this computer. No, no, absolutely not it will not play a movie picture or Music or anything like that.

It's not a computer that was torn down. It was built from the ground up like this Especially for Amish. Yes.

Would you mind showing me your mobile phone? Okay. Yes The only thing you can do with that is call You cannot text do anything like that.

It's just a portable phone So this is where it stays but as far as going home, no, I don't have it Keep it right here. That's for the business The technology is progressing so fast it it's hard to deal with We have to teach What the moral impact is in being evolved in this so each individual can make better choices for themselves We use technology as long as we use it and it doesn't get to the point where it uses us And controls us. That's the bottom line of it It's kind of strange that these people who you can say are living in the past Maybe they're living in the future.

It was just a minute ago that We didn't have many of these tools and We were fine We lived rich in full lives we Had close connections and the friends and family. So having said that I want to recognize that there's a lot that the digital brings to our lives and We deserve to have all of that but we deserve to have it without paying the price of surveillance capitalism and right now We are in that kind of classic Faustian bargain 21st century citizens should not have to make a choice of either essentially, you know going analog or living in a world where our self-determination our Privacy are destroyed for the sake of this market logic that is unacceptable and let's also Not be naive you get the wrong people in charge of our Government at any moment and they look over their shoulders at the rich control Possibilities offered by these new system and there will come a time one even in the West even in our democratic Societies our government will be tempted to annex these Capabilities and use them over us and against us let's not be naive about that when we when we decide to resist surveillance capitalism right now while it lives in the market dynamic We are also Preserving our democratic future and the kinds of checks and balances that we will need going forward In an information civilization if we are to preserve freedom and democracy for another generation The European Union already has legislation with which it regularly wraps American tech businesses over the knuckles Google was fined several billion dollars because it had abused its monopoly position in theory European citizens are protected against data robbery by regulations for fair competition and By the GDPR are the general data protection regulation And do you think that the EU data protection law is a step in a good direction or is it sufficient we have privacy laws and the gdpr is the furthest frontier of our privacy laws and We have antitrust laws the thing is that as important as these laws are We still need more because surveillance capitalism is unprecedented and So we're going to need the laws and the regulatory regimes that respond directly to these new unprecedented operations GDPR talks a lot about data ownership data accessibility and data portability These are very important things if the only data that we're talking about is the data in the public operation The data that we've given to the corporations but as we have seen most of the data that they use To feed into the factories to produce the predictions these are data that we haven't given or if we've given we don't even know that we've given because it came in our exclamation points it came in our walk in the park and Picking up the cadence of our voice and and the the timbre of our voice all rich predictive signals so no matter how much data ownership we claim or accessibility or portability Most of the data is in the shadow operation and the shadow operation is never coming to us we will never get those data they claim them as Data that they own they took it from our lives They took it from our private experience without our permission They analyzed the data. They made it into products.

They sold the products and they took the profit illegitimate profit Because they took it at the beginning without Asking without our knowledge recall by passing our awareness do you have a chance against the Behemoth of surveillance capitalism, these twenty years have been a honeymoon For surveillance capitalism because they have been twenty years largely unimpeded by law Unimpeded by law why Well, the most important reason is that they're doing things that have never been done before so there are no laws against it Just like factories Employing children You know in in a mass production factory there hadn't been anything like that before and there were no laws against it and then took our Societies a while to sort of wake up. We stood up against the Extractive companies and their violence The companies that came to be known as the Gilded Age the companies where we turned around and we call their leaders robber barons That's not what they were called at the time at the time they were worshipped as these, you know wealthy gods and geniuses who who who knew how to Wield the might of machines and capital if we had been trying To stop it to curtail it to outlaw it For every one of these twenty years and we had utterly failed Then I might you know, put my chin in my hand and say gosh I'm I'm starting to feel like things might be really bad, but that's not the case. We haven't even tried yet to stop it look Surveillance capitalism is 20 years old Democracy is several centuries old I Bet on democracy.

Thank you for watching For more on this subject take a look at the playlist.