Article

The Intercept

The Intercept

The AI Doomsday Future Is Not Inevitable

2 months ago

Sam Biddle and 404 Media’s Jason Koebler discuss the government’s growing use of private contractors, from Flock to Claude, to surveil and kill abroad and at home. The post The AI Doomsday Future Is Not Inevitable appeared first on The Intercept .

Local police departments use Flock cameras to track license plates. ICE uses Palantir software to track and round up immigrants. Defense Secretary Pete Hegseth’s war department uses Claude to bomb Iran. The U.S. government spends billions employing private corporations to obtain mountains of data — and the surveillance space is only getting more privatized in Donald Trump’s second term.

“It really has become a Wild West,” 404 Media co-founder Jason Koebler tells host Akela Lacy on The Intercept Briefing, on the state of the U.S. surveillance apparatus today. While Palantir and Flock have become household names, Koebler says, “anything that can be surveilled is being surveilled, and a lot of this is happening from companies that very few people have heard of, and companies that are willing to push the envelope on what they’re willing to do, like how far they’re willing to go to surveil people.”

“A lot of the stuff that you used to associate with the NSA or CIA or another state spy agency, those kinds of capabilities you can now just buy,” Intercept senior technology reporter Sam Biddle notes of the ease and accessibility of the tools now available to the government. “You don’t need to be a top-secret government agency. You can just be a contractor in Virginia.”

“It’s a way to sidestep warrant requirements,” says Koebler. Surveillance contractors “are doing things that if the government were doing directly, they would need to get a warrant in order to do it.”

This week, for a special recording of the podcast, independent tech outlet 404 Media and The Intercept teamed up in front of a live audience in Los Angeles to discuss how the U.S. government weaponizes private data to surveil and kill abroad and at home, the resulting backlash and potential regulations on the table, what tangible actions the average person can take to “divest from the contemporary surveillance hellscape,” and refocusing AI-ending-humanity doomerism.

“When you ascribe motives and agency to AI, you’re letting Sam Altman off the hook,” says Biddle. “There are human beings building this technology. It’s not inevitable. This doomsday future is not inevitable,” adds Koebler. “We can’t bury our heads in the sand and say, ‘This is not changing society in some way.’ But it is being driven by very rich humans, and they can be regulated. They can be held to account.”

“What’s been really interesting about the Flock backlash, and heartening, is that it is completely apolitical, or it’s nonpartisan. It is completely — from the deepest red to the people in this room right now — the entire political spectrum hates these,” says Biddle. “There’s the real potential for the first time in a while to create a pro-privacy coalition that bridges a lot of the otherwise pretty giant political gaps.”

For more, listen to the full conversation of The Intercept Briefing on Apple Podcasts, Spotify, YouTube, or wherever you listen.

Transcript

Akela Lacy: Welcome to The Intercept Briefing, I’m Akela Lacy, senior politics reporter at The Intercept.

Today we’re coming to you live from Los Angeles for a special episode in collaboration with 404 Media.

Our topic? Killer tech. We’re talking lethal autonomous weapons; AI-assisted strikes in Iran; mining your digital footprint and even your financial activity to track you down; and surveillance cameras coming soon to a neighborhood near you.

This conversation is timely. Just this week, warnings from Anthropic researchers that AI could kill humanity set off alarm bells on Capitol Hill and prompted a bipartisan group of lawmakers to call for immediate action to regulate the AI industry — from Republican Sen. Ted Cruz to Independent populist Bernie Sanders, who is organizing senators for a briefing next week on the “extraordinary dangers” of AI.

The world we’ve long been told to fear is here. But how much do we actually know about these tools and how the government is using them — not just against you and your neighbors, but around the world?

To take us down the rabbit hole, I’m joined by my colleague, Intercept reporter Sam Biddle, a senior technology reporter who focuses on issues of surveillance, privacy, and corporate power.

Also joining us is Jason Koebler, co-founder of 404 Media, where he writes about surveillance and AI.

Sam and Jason, welcome to The Intercept Briefing.

Sam Biddle: Really great to be here.

Jason Koebler: Thank you for having me. Thank you all for coming. Thank you to The Intercept for having us. We’re very excited.

Akela Lacy: To start, can you all just paint us a picture of what the broad surveillance tech apparatus looks like today?

Many of our listeners are familiar with some of the biggest players, like the data mining government contractor Palantir or Flock, the operator of surveillance hardware marketing itself as a friendly neighborhood public safety tool.

But let’s zoom out a little bit. Like, how did we go from the days of the National Security Agency and Edward Snowden, to this largely privatized and, in many ways, even more lawless world of mass surveillance?

Sam, I’ll start with you. What’s stayed the same since that NSA era, and what’s new now?

SB: The NSA is still here, to say the least. I don’t think you have this conversation, or certainly not in the form that we’re going to have it tonight, without 9/11. I think that the creation of the Department of Homeland Security, the private sector economy that you described, billions and billions of dollars, is all downstream of 9/11.

Related

“Terrorist”: How ICE Weaponized 9/11’s Scarlet Letter

I looked it up: In 2006, the DHS budget was, like, $30-something billion. The presidential request for next year is $118 billion, and a lot of that is flowing to contractors like Palantir.

To answer your question, Snowden taught us correctly that we have a lot to fear from the government, but there was not as much emphasis on the companies that empower the government.

I think that has really been at the fore especially around something like ICE raids, which Jason and his colleagues have really done a good job documenting the companies that are behind that.

“Snowden taught us correctly that we have a lot to fear from the government, but there was not as much emphasis on the companies that empower the government.”

JK: The big thing is it really has become a Wild West. People have heard of Palantir, they’ve heard of Flock at this point, as you mentioned.

But we got a leaked document recently about some of the contractors that ICE works with. They are companies like Accurint, Chameleon, Locate X, Cobwebs, ISO ClaimSearch. These are all different companies in this supply chain that do different things, whether it’s cellphone location data, whether it’s browsing history, whether it is cellphone tower data, web backbone information.

“A lot of this is happening from companies that very few people have heard of, and companies that are willing to push the envelope on what they’re willing to do.”

It’s just anything that can be surveilled is being surveilled, and a lot of this is happening from companies that very few people have heard of, and companies that are willing to push the envelope on what they’re willing to do, like how far they’re willing to go to surveil people.

SB: A lot of the stuff that you used to associate with the NSA or CIA or another state spy agency — those kinds of capabilities you can now just buy, and I think that’s really changed since Snowden.

You don’t need to be a top-secret government agency. You can just be a contractor in Virginia called “Persistent Solutions Incorporated” and be selling these powers to whoever wants to buy them.

AL: Which is a great segue to talk about Flock. Jason, you and your colleagues at 404 have reported on some of the most egregious uses of Flock, police using it to stalk people, track down a woman who had an abortion, conduct surveillance on protesters, and help ICE hunt people down.

The surveillance space has become increasingly privatized, as we’re talking about, but why does that matter, and what do you think is coming next?

JK: It matters because it is a way to sidestep regulations. It’s a way to sidestep warrant requirements and things like this. We’ve done many, many, many stories about Flock, but also other surveillance contractors where, they are doing things that if the government were doing directly, they would need to get a warrant in order to do it.

Surveillance contractors “are doing things that if the government were doing directly, they would need to get a warrant in order to do it.”

But this data, sometimes harvested from cellphone apps, sometimes it’s harvested from private companies like Flock that are sticking cameras on poles all throughout America. For a long time, the government would run into these legal roadblocks, and now they’re just sidestepping it by buying that information.

We’re also seeing the localization of a lot of this stuff. Flock is a company that primarily works with local governments, small towns, big cities. And we’ve seen that kind of over and over again with drones, with ALPRs [automated license plate readers], increasingly with things like facial recognition tools as well, where these things are being piloted in small towns where they maybe have a police force that’s willing to be innovative.

I feel like I say this often, but police have not found a spy tool that they’re not willing to try. So this is how this works now is you sell it to a local government, and then that data filters up and eventually makes its way to the feds.

So I think it is important to follow these companies how their business model works, and also how they spread from local governments to state governments and up to the feds.

SB: The technology’s gotten cheaper, too. The kind of capabilities that Flock sells would have cost, 20 years ago, it would be much cruder but prohibitively expensive. This stuff gets mainstreamed because it gets more powerful and it gets more affordable.

But also, the police departments are awash in money, so they can afford, like Jason says, they’ll buy whatever they can just because they like toys.

“ This stuff gets mainstreamed because it gets more powerful and it gets more affordable.”

JK: Those cameras, for what it’s worth, run on, like, several-generations’-old Android technology. It’s not expensive, as you said.

AL: We’re going to touch on the expansion of this technology into local police departments a little bit later. But first, Sam, you’ve reported on the literal hunting of immigrants using bounty hunters and enriching private companies.

Why aren’t some of those corporations facing the same level of public scrutiny for their role in ICE’s mass deportation and arrest campaign that we’ve seen in other cases?

SB: I think that there is just such an incredible diversity of these companies providing these kinds of surveillance services.

JK: And you love to see that. [Laughter.]

SB: It’s one of the only parts of the economy that’s probably steadily growing. But yeah, I found 10 different contractors, none of which I had ever heard of before, all very obscure, with names like “Government [Support Services]” — very anodyne — that were contracted by ICE to find immigrants so that they could be deported.

The more that they found and the faster they found them they would be eligible for basic cash bounties from the government. So they were incentivized to just find as many people as possible using a combination of on-the-ground stalking and digital tools.

“Their obscurity is a big asset.”

As much scrutiny and attention on Palantir as it gets — for a reason, it’s all warranted. But I do also think that sometimes the household names soak up a lot of the anger and the blame, and it lets the minnows get by with relatively less scrutiny. Compared to Palantir, a company that sells janitorial services to Air Force bases and then gets this bounty hunting contract is tiny.

But I think that there’s just too many for people to keep track of — too many for people whose job it is to keep track of them to keep track of them. Their obscurity is a big asset.

AL: And something like Palantir is a known quantity.

SB: And they seek attention. Palantir is very attention-seeking, and they want publicity.

JK: We were talking about that a little bit before the show. Very often you’ll see a Palantir or a Flock take a lot of the heat — even like Meta take a lot of the heat. “Oh, is Facebook going to put facial recognition on their glasses,” things like this. Turns out now they have cover, and they probably will.

Related

Staten Island DA Bought Clearview Face Recognition Software With Civil Forfeiture Cash

But Clearview AI — a company that does facial recognition; you can put a photo in and search the internet and find all these instances of where that person has been. It’s used by a lot of police departments — and that is a startup that is essentially one guy. It’s bigger now, but it was started by one guy who was willing to do something that a lot of other contractors weren’t willing to do because there was so much attention on some of these big players, and the big players saying, “Oh we’ll never do facial recognition.”

But then you have Clearview come in and raise their hand and say, “We’re willing to do it.” That’s something that we’ve seen over and over again over the last decade or so, is that the backlash to some of these companies is very important, but there’s also 10 other companies that are willing to pop up and say, “Yes, we will hunt undocumented immigrants for you. We did previously do janitorial services, but I did go to a police training one time.”

Literally, that’s what these companies are in many cases. AI has allowed companies to spin up very quickly. A lot of them are super janky, but that doesn’t mean that they’re not getting government contracts.

AL: Jason, I want to talk about what this federal surveillance actually looks like in practice. 404 reported recently that the Department of Homeland Security is using a secretive predictive policing unit to pull people over. What did you all find?

JK: So it’s called the PITT [Predictive Intelligence Targeting Teams], not the TV show.

But basically it’s these Border Patrol teams that are using financial data, various federal databases of which we are not sure what they are. They are essentially finding patterns that they believe are suspicious, and they are passing information to local police and saying, “Hey, please pull over this person with this license plate.”

We wrote earlier this week about a guy, I believe in Wisconsin, who was pulled over for an obscured license plate. At least that’s what the local police told him. It comes out during his court case that this PITT team had seen some financial activity that they thought was suspicious, asked local police to pull him over, to find some pretext to pull him over, and then they charged him with basically weed offenses, which didn’t seem that was what they were actually looking for. The [Homeland Security] documents we saw were like, “Oh, this is a big narcotics trafficker.”

We’ve seen things like that kind of happen over and over again. I think with regard to ICE in particular, often Palantir brings in a lot of different federal databases, a lot of different commercial databases.

Related

Federal Agents Are Intimidating Legal Observers at Their Homes: “They Know Where You Live”

ICE has become obsessed with this tool called Mobile Fortify, which is a smartphone facial recognition app made by a company I don’t even know what they do or what they are. But they’re using basically a facial database from Customs and Border [Protection]. So any time that anyone’s interacted with the government, TSA, at the border, things like this — that’s what it’s pulling from. There’s already been a few cases of mistaken identities and things like this.

“ICE has become obsessed with … Mobile Fortify, which is a smartphone facial recognition app made by a company I don’t even know what they do or what they are.”

SB: Worth pointing out, you can opt out of doing the biometric boarding at the airport. I embarrass my wife every time we go anywhere, but you can opt out. Just say no. [Laughter]

JK: They’re always mad.

SB: You’ll still be in some database, but, you know, it feels good.

AL: When it comes to ICE tracking people down, you’ve touched on this a little bit, but I wonder if you can take a step back and really break it down and walk us through the steps and the actors involved in each phase, from locating to identifying to arresting people.

JK: So a lot of this is like patching things together from individual court cases that we’ve seen, where there’s like a deposition from an ICE agent explaining how they found a person.

Related

ICE Gave New Hires Access to Restricted Info on Palantir App Before They Passed Background Checks

A lot of it does come back to Palantir and the sort of different apps that they have. Companies like Thomson Reuters are involved, LexisNexis — these big, giant companies that collect a lot of information and feed it back into the government. We learned that ICE was doing what it was calling a “density report,” where they were trying to find densities of potential undocumented immigrants.

So a lot of the places that they were choosing to do enforcement actions were based on maybe old addresses or just like interactions with the government that they had. So they’re pulling things like IRS data, labor data, things like this, as well as anything you could possibly imagine. So like cellphone location data, browsing data, license plate information.

Customs and Border [Protection] has their own ALPR cameras, automated license plate reader cameras. But then we also reported that through the 287(g) program, which is a program that allows local police to do ICE enforcement, they were doing Flock searches on behalf of ICE, local police were.

So it’s a mix of this, where they would get a target, and then as Sam said, like a lot of this was being outsourced to random — was and is — being outsourced to fly-by-night bounty hunters and police contractors and things like this. It’s been a real mixed bag, but it does come back to surveillance and to trying to find these density of undocumented immigrants to choose for their enforcement actions.

SB: I think it’s also worth pointing out that this stuff is only possible because there’s no law prohibiting it. You can drive a truck through the holes in the Fourth Amendment. Like Jason said, in theory, you’re not supposed to be able to do a lot of this stuff without a judge giving you the OK, but there is nothing stopping the government from buying the location of your phone for the past year, and the next year, into the future.

Related

The U.S. Treasury Is Buying Private App Data to Target and Investigate People

Every single place you go from minute to minute that can just be purchased because you happen to download Candy Crush or something where that has ads in it. There’s no law against that. There’s no regulation. The sale of private data, and geolocational data in particular, market is completely unregulated. You can just do whatever.

“There is nothing stopping the government from buying the location of your phone for the past year, and the next year, into the future.”

AL: And some of these companies have come back and said “These agencies are violating their terms of service or their terms of use.” Like you guys reported on this with Shadow Dragon, Jason, right?

JK: Yeah, every once in a while, a company is like, “Oh, we didn’t know it was being used for immigration enforcement.” And they’ll either revoke access, or maybe they’ll sell a new license or something like this. It does happen every now and then that what ICE is doing goes too far even for some of these surveillance companies.

I think also worth mentioning very quickly that a lot of this security and surveillance apparatus was built under Obama and earlier than that. They had these extremely frail guardrails on separating information between agencies, things like that.

Related

Mahmoud Khalil Won His Freedom Despite the Best Efforts of ICE’s Intelligence Unit

And then suddenly, it’s, oh, HSI is not going to do workplace enforcement anymore — Homeland Security Investigations, which used to do generally really complicated investigations into drug trafficking rings, things like that. Now it’s just doing standard deportations and pulling people over and things like that. And so all of the guardrails have been knocked down, which is [pause] exciting.

SB: And the more you farm it out to private companies, the less of a public record potentially accessible paper trail there is.

If the NSA breaks the law, in theory, you can have congressional hearings about it, and you could, again, in theory, send people to jail. That wouldn’t happen, but you could.

But if a company does something, A, it’s legal, but B, probably no one’s going to find out unless, you know, Jason finds it. There’s no oversight! And there’s every incentive to maximally sell this stuff.

AL: Let’s talk about how this technology is being used to wage war around the world. We all saw the headlines about Anthropic’s Claude reportedly assisting U.S. strikes in Iran. Anthropic’s CEO said he doesn’t know if or how Claude was actually used, including if it was used in the strike that hit an elementary school and killed 120 children.

At one point in time, many of these Silicon Valley firms had sworn that they would not let their tech be used in warfare or surveillance, particularly in unmanned or automated warfare. That is no longer the case, and maybe it was always an empty promise. But partnerships between the government and tech companies are fueling wars happening right now — not just in Iran, but in Gaza and even in Ukraine.

Sam, what do we actually know about how AI is being used in those arenas, if anything?

SB: We know a pretty good amount, actually the use of American technology in the genocide in Gaza is pretty well documented at this point. Amazon, Google, and Microsoft are all very prominent contractors with the IDF [Israel Defense Forces].

Amazon and Google sell cloud computing services to Israel’s state-owned weapons companies that build the bombs that are dropped on Gaza; that’s established. And no one really tried to hide it. A lot of that stuff wasn’t secret to begin with.

The use of American technology, of AI companies’ services, in Iran is murkier. You’re absolutely right that these companies, almost all of them — maybe not Microsoft — at one point claimed they would not ever do this. OpenAI and Anthropic were both founded with these very like highfalutin missions of benefiting all of humanity.

AL: That’s, like, why it’s called Anthropic.

SB: Right. [laughter] Great point. And then they decide, “Oh, wait, there’s a lot of money to be made if we become Pentagon contractors.” So a few years ago, OpenAI just deleted the part of its terms of service that said they wouldn’t do military work. Google deleted “Don’t be evil” as its motto, which was very on the nose.

Related

Trump’s Election Is Also a Win for Tech’s Right-Wing “Warrior Class”

It’s a mistake to ever think that these policy positions of companies mean anything because they could just change them when they want to. But it is striking that, like you said, these were all things they at one point had a principled stand.

AL: And were very publicly, “We’re leading the frontier. We’re not going to let our stuff be used to kill people or surveil people.” And to Jason’s point, as soon as one person says, “Actually, that sounds kinda good,” everyone else is like, “You know what? We were wrong.”

“If you’re Google, you can only make so much money selling extra Gmail storage and ads.”

SB: If you’re Google, you can only make so much money selling extra Gmail storage and ads. At a certain point, the government is this customer that’s always going to be there and always is going to have a lot of money to give you. So yeah, I think they all looked at each other and were like, “Wait, you guys are down for this too, right?”

But as far as Iran, Anthropic is confirmed as having been involved since in the first day of the joint U.S.–Israeli attack.

JK: I saw a few minutes ago that Anthropic said the Houthis attempted to use Claude to target missiles, like typing in the chat box.

SB: I saw that!

JK: And they stopped that; they said they stopped that. I was at like this defense contract when Anthropic was nodding at the idea that it wouldn’t be helping in Iran. There were Anthropic and OpenAI executives there talking about this all, and they made very clear, at least at that point, that it’s like, “Oh, it’s not that we oppose working with the military. It’s that we want better oversight. There will be, like, an Anthropic person sitting in the Pentagon with the DoD making sure that this is being used in an ethical way. There will be an OpenAI person doing the same.”

It’s very unclear whether that is actually happening to any degree, and also what these tools are specifically being used for. We know it’s being used for targeting, we know it’s being used for planning — by targeting, I mean figuring out who to target and that sort of thing.

AL: Selecting a target.

JK: Yeah, selecting targets. But we don’t really know the level of autonomy, or at least I don’t.

AL: And they apparently don’t either, right?

SB: Well, that’s the thing: Both possibilities are really dire. If these companies are hands-on, helping wage war, that’s terrifying for a lot of reasons. But the alternative — which Anthropic’s CEO, the position he took, Bloomberg News asked him if Claude had been used to blow up that school, and he said, I don’t know.

Google has a similar position with the IDF, where they said, once we sell them this stuff, we don’t know how they’re going to use it, we don’t have visibility into how they use it, which is completely plausible.

But that’s also a pretty frightening prospect because then, they can say, “Oh we won’t let our technology be used for X, Y, Z,” but they hand it off to the customer, and they can do whatever they want with it, and the company will never find out.

AL: Sam, you have obtained hundreds of government documents on how the Pentagon is using AI. I wonder if you can tell us just how deep that relationship goes.

SB: We, through a Freedom of Information Act lawsuit got a set of contracts that OpenAI, Anthropic, Google, and xAI — Elon Musk’s outfit — signed with the Pentagon last summer to develop militarized LLMs [large language models] for the Pentagon’s use.

Related

AI Giants Work Hand-in-Hand With the Pentagon, Contracts Reveal

They were heavily redacted, but what really came through clearly from the unredacted portions was that these companies did not just sell the Pentagon a product, but they sold them a very, very intimate relationship, an ongoing collaborative relationship wherein they were advising the Pentagon essentially on how to use these products to wage war.

They were getting classified briefings. They were getting briefings on U.S. military operations. They were providing hands-on training. They were providing threat assessments. They were helping shape DOD high-level strategy. So going well beyond the typical remit of a company.

“These companies did not just sell the Pentagon a product, but they sold them a very, very intimate relationship.”

Lockheed can sell the Pentagon some missiles, and the Pentagon more or less decides how they’re going to use them. Now we have the engineers from these companies and policy figures from these companies helping the U.S. decide how it will fuck, target, and kill people. We’re supposed to have ostensibly civilian control of the military — accountable to the public. Again, all this stuff is very much in theory, but it’s supposed to work that way.

But the more these companies intertwine themselves with warfare, the more of it is completely unaccountable because Sam Altman doesn’t owe you anything.

AL: It takes all the air out of the pretense that that’s like how this works in theory, right?

SB: In terms of, like, oversight?

AL: Embedding these private people in the government. That’s not actually how it works.

SB: Yeah, these are supposed to be decisions of who to kill and

Read on The Intercept