I recently watched the Netflix documentary Unknown: Killer Robots, and came away upset and incredulous. I was astounded at the lack of basic logic in members of the AI community who couldn’t recognize that if AI can generate beneficial outcomes, it stands to reason it can also generate negative outcomes. Apply this to any use-case: pharmacology, defense, transportation, healthcare, etc. AI being, in its current state, a tool, then anywhere it’s possible for a human to act maliciously, it is ipso facto possible for a human to use AI to act maliciously. This principle should be well-understood as a fundamental axiom in technology by this point.
Apart from a logical disconnect in the AI community at large, a lack of basic human empathy is also expressed from the proponents of AI military applications in the film. Euphemisms proliferate. “Saving lives”, “trading machines for lives”.
These statements can only be made in good faith when the person delivering them holds the fundamental presupposition that some lives deserve to be lost.
Seen from another lens, it gestures toward an overly-simplistic, binary understanding of life: there are good guys and bad guys. Good guys shouldn’t die and bad guys should. This ignores the complexity of the politico-economic factors that influence warfare, that define who is “good” and who is “bad” — why are the “bad guys” fighting? Why are the “good guys” fighting? What’s at the root? Who decides who is whom? In the film, these questions seem never to have been considered by proponents of AI weapons.
Probably, they’ve been superficially considered and answered with a naive certainty that enables those who are developing technology with the potential to kill people en masse to sleep soundly at night.
I see the thematics of our particular evolution of capitalism expressed in the final leg of the documentary, where a U.S. Army Military Trade Expo is filmed. Generals strut around in full regalia, shopping for instruments of death like they’re at Target. Sweaty salesmen lure them in with promises of “providing whatever the customer needs to complete the mission”. Those who are pushing AI weapons are helping ensure “machines are doing the killing while humans supervise the kill chain.”
Kill chain? Complete the mission? Customer?
These terms drip with the same abstract heartlessness of all bureaucratic corporate speak — euphemisms, mere identifiers that feign complexity of meaning while all really pointing to one thing: mechanisms enabling a producer to provide a consumer with a product for a specified outcome.
That’s it. That’s what so much flashy corporate-bureaucratic language boils down to. It’s dehumanizing: people are understood primarily as profit-generating mechanisms, aimed at a very limited goal.
And now, with the advent of AI (which takes the rational function of the human being and expands it exponentially) human rationality itself is becoming largely obsolete from the perspective of business.
Thus, dehumanization, a key theme of our version of capitalism which finds its roots in the language of corporate-bureaucratic speak is going to move from the realm of deceitful subjugation (come work for us! we love all our employees. we’ll replace you or lay you off as soon as it’s profitable!) to blatant elimination (why use people when the AI does it better?)
—
Dehumanization has always been present in business, of course, down to the level of day-to-day language. Objective identifiers such as “customer”, “producer”, “client”, “competition”, these all constitute a linguistic structure that flattens the human being into binary functions. In more benign applications, this linguistic structure lubricates the bureaucratic decision-making mechanism. Consider that cutting hundreds to thousands of jobs is a lot easier when you’re cutting a “resource” instead of a human being with a life and family. When applied to the business of war, this ability to abstract the human being into a data point makes the decision to create, distribute, and utilize technology that kills human beings another simple calculus. The thinking works like this:
A “producer” wants to make money. They create a “product” for a “customer” using the least amount of labor possible in order to maximize profit from sales, and sell the product to the customer by out-innovating and out-marketing their “competition”, adopting persuasive language to display “responsiveness to their customer’s needs” and using the knowledge gleaned from that “responsiveness” to innovate the product in ways that “serve the customer”. These relationships can actually be diagrammed as a set of cloze statements:
[PRODUCER] creates [PRODUCT] for [CUSTOMER].
[CUSTOMER] purchases [PRODUCT].
[CUSTOMER] gives feedback to [PRODUCER].
[PRODUCER] responds to customer’s feedback.
[PRODUCER] attempts to out-innovate [COMPETITION].
[PRODUCER] recreates [PRODUCT] for [CONSUMER].
And the cycle continues. Once that abstract template of thought is constructed, anything can be slotted into it. This could, in fact, be a working construction of the essential capitalistic structure that applies broadly to every institution beneath capitalism:
Ex 1: Apple
[Apple] creates [iPhones] for [users].
[Users] purchase [iPhones].
[Users] give feedback to [Apple].
[Apple] responds to [users’] feedback.
[Apple] attempts to out-innovate [Samsung, Google, etc].
[Apple] recreates [iPhone n] for [users].
Thus we arrive at iPhone 14.
Or:
Ex.2: Public Schooling
[Public schools] create [public education] for [taxpayers].
[Taxpayers] purchase [public education].
[Taxpayers] give feedback to [public schools].
[Public schools] respond to [taxpayers’] feedback.
[Public schools] attempt to out-innovate [other schools].
[Public schools] recreate [education] for [taxpayers].
Now, public school is an interesting case where the structure is itself being used as a euphemism. Rhetoric around public schools always states they are generating benefit for kids and society-at-large, but so long as they are funded by taxpayers who themselves receive income from a capitalistic system, at the end of the day they must generate benefit for the producers, i.e. the businesses. Follow the money. Hence our public schools shape effective bureaucrats and disciplinarians far more effectively than they do critical thinkers, artists, or humanitarians. They are a control structure that teaches people how to become control structures of themselves and the world around them, so they can click neatly into the broader structure of control that is the essence of our capitalistic, bureaucratic society.
What’s useful about the school example is that it shows how the capitalistic assumptions, rooted always in the profit motive, begin to break down the closer they come to generating human well-being. Thus the structures built from these assumptions have to become increasingly convoluted and deceptive to maintain the illusion that they’re working for the benefit of society-at-large. I would go so far as to say that the profit motive is mutually exclusive from the generation of well-being.
This is because the nature of generating well-being is in giving something; the nature of generating profit is taking something away.
Not that profit is inherently destructive. It just needs to be balanced with giving. One cannot take indefinitely without inevitable collapse. Unfortunately, our system doesn’t recognize profit as only one side of a necessary duality.
What’s pernicious about our current capitalistic system is that anyone born within it is so thoroughly immersed within all of its expressions from birth (hospitals, schools, entertainment, etc…every institution reflects, to some degree, the basic profit motive and its concomitant control structures) that it is taken not as an arbitrary mental construction, but an ontological presupposition. It rests below consciousness as “just the way things work”. And when that’s the case, people are necessarily perceived primarily as objects to click in and out of the prescribed slots of producer, product, consumer, and competition.
Further, anything that furthers the profit-motive can click into those places, regardless of its ethicality. Here’s the AI Weapons model at work:
AI Weapons
[AI weapons manufacturer] creates [weapons] for [military branches].
[Military branches] purchase [weapons].
[Military branches] give feedback to [AI weapons manufacturer].
[AI weapons manufacturer] responds to [military branches’] feedback.
[AI weapons manufacturer] attempts to out-innovate [other AI weapons manufacturers].
[AI weapons manufacturer] recreates [weapons] for [military branches].
Euphemistic phrases mentioned earlier such as “saving lives, supervising the kill chain, completing the mission” become selling points, and disguise the fact that all they’re really doing is making the most lethal weapons possible which ensure “good guys live and bad guys die”, because that’s what sells.
Profit is made. Producer and consumer celebrate. Maximum profit for minimal cost. And lives are saved along the way!
…except for the lives that are lost as a result of the product.
But notice, effects of a product don’t factor into the basic thought structure — not unless it would somehow hinder the profit motive. Then they’re called “risks”, and these risks are managed to the extent it benefits the profit motive.
—
breathe
—
The union of AI weapons with the military-industrial complex may be the most perfected model of capitalism to date.
It exemplifies all of the thematics of capitalism: it’s profitable, it’s scalable, it’s effective at what it does — and incredibly dehumanizing.
It provides the most powerful mechanisms of control over human lives that can be framed within a bureaucratic business model.
Really, it embodies the logical endpoint of the dehumanizing presuppositions inherent in the capitalistic system: once dehumanization becomes normalized as the objectification of individuals within a system, it is only a matter of time until it becomes controlling. And since capitalism is not inherently balanced or sustainable, to continue to exist, it needs control. Because aggression is latent in the very act of control, it’s possible for that aggression to ferment, ultimately, into actual violence, as more and more control is demanded to maintain an unbalanced system. Then it can continue to take what it needs from the planet, from the disenfranchised, from whoever or whatever has something of perceived value, to allow the rotors to keep churning.
And what is more expressive of control over a life, than the ability and willingness to take it?