Excerpts from AEI’s Council on AI Ethics’ “founding document”:
- “Our Council on AI Ethics holds the democratic belief that the gut reactions of ordinary citizens to the transformations they see occurring around them are worthy starting points for ethical reflection. This view, which traces back to Aristotle, was applied to technology ethics by Leon Kass in his argument against human cloning. Kass used the phrase ‘the wisdom of repugnance’ to suggest that gut feelings and intuitions have meaning that is worth taking seriously. They may provoke more than they prove, but they give us reason to pause, discuss, and reflect with our fellow citizens. Reflecting upon feelings of unease helps us understand the integration of emotion and reason, and raises questions about our nature, character, and flourishing. […] Stories, both real-world and plausible hypotheticals, best reveal the questions arising about who we are as persons. Many of the most important ethical insights on AI are being offered not in white papers or academic journals but in personal reflections. Narratives, in what follows, along with well-developed case studies the council will offer in future work, allow for an inductive exploration of how our use of technology can help us in, distract us from, or thwart our efforts at living in accordance with our gut reactions and our considered judgments, rather than in tension with ourselves.”
- “Consider a novel use of AI for efficient competitive advantage: identify my targets. Israel’s Lavender and America’s Maven systems, powered by AI, reportedly changed warfare by enabling a dramatic increase in the tempo of targeting in Gaza in 2023 and Iran in 2026, even when compared to the pace of the 2003 ‘shock and awe’ campaign in Iraq. Neither system was fully autonomous; a human was in the decision-making loop, responsible for choosing to fire each missile or shell. In this case, we see that outsourcing information analysis to the machine has brought a huge gain in competitive advantage. But it has also posed a worrisome loss of emancipation broadly understood — of ownership over our actions. Maven, for example, seems to reassure us that there is a “human in the loop,” but news reports suggest that the system does not offer nearly enough time for the operator to verify the information the AI uses and the interpretation it makes. The design effectively reduces the human in the loop to a button-pusher, while relocating the judgment of acceptable risk from human prudence to the algorithm.”
- “But whether the AI promises certain goods, like new medicines and materials, or information, for example economic insights, we will have to ask what we could lose if humans were to take a backseat in the process of scientific discovery. First, we could lose the character-forming habits of scientific inquiry, the particular excellence that comes from a deep dedication to the pursuit of truth. Second, we could lose clear human responsibility for the uses to which scientific knowledge is put, from novel biological weapons to AI-powered genetics that can easily slide into eugenics. Third, we could lose an understanding of how science fits into our broader social and political arrangements, how it must be integrated in order to serve the common good. For example, Emilia Javorsky argues in an essay on AI and cancer that assuming superintelligent AI will cure cancer vastly oversimplifies the problem. She notes that ‘if a potential cure were discovered [using AI] in a laboratory today, there is no guarantee we would recognize it as such, nor that it would successfully navigate translational research, regulatory approval, manufacturing scale-up, reimbursement negotiations, and clinical adoption to reach patients who need it, at the moment they need it.’ […] The family of disciplines meant to help individuals ask the big questions of life — philosophy, religious studies, history, literature — have more and more sought to mimic the hard sciences. Adopting the scientific ‘knowledge production’ model has been an odd fit for disciplines that historically understood themselves to be directed at timeless questions. As quantity gained emphasis over quality, humanities scholars were pushed toward a narrowing of their scope of research into disciplinary silos, and “interesting” became more common praise than “insightful.” Over-reliance on AI may over time hollow out both individual scholars and the practice of the scholarly vocation, as apprenticeships that aim to attune leaders’ minds to truth give way to efficient content distribution. This in turn would hollow out the traditions themselves, as philosopher Lily Abadal has noted. The chatbot does not revere the canon, and habitual outsourcing of thought to it may prevent future generations from ever building a common life of reasoning together.”
- “Against the data-retrieval view stands another view, dating back at least to Augustine of Hippo, which holds that the faculty of memory allows us not merely to access but also to interpret knowledge, integrating experience and learning to form a sense of self. Understanding allows us to grasp the present; but memory is necessary to grasp the past, and both together are necessary to plan well for the future. Philosopher Jean-Louis Chrétien for this reason links memory with hope: ‘Recalling the origin belongs properly to hope tending toward its end.’”
- “We are what we repeatedly do. At each individual decision-point, the marginal benefit of using AI seems far greater than the marginal cost: We’re busy and tired, this is efficient and effective, and what harm does it do, really? Only in retrospect do the drops of water become a flood, the thousand cuts become a death. O’Rourke is directly concerned with writing: ‘What we stand to lose is not just a skill but a mode of being: the pleasure of invention, the felt life of the mind at work.’ Her point generalizes to every domain where AI is flowing in to aid and then replace human thought and decision-making. Cosmos Institute founder Brendan McCord calls this risk ‘autocomplete for life.’ It may seem not to matter that we let algorithms recommend our music and movies and sort our social media feeds, but doing so transforms our imagination and our capacity for attention. It also makes us vulnerable to persuasion and propaganda honed from every detail of our … history.”
- O’Rourke wrote that ChatGPT came to feel like ‘a ghost’ that had ‘colonized my brain, controlling my fingers.’ In a society-wide gradual disempowerment through incremental, seemingly rational choices to cede our collective agency, a similar image may be apt. Here is one description of that future:
The tractors came over the roads and into the fields, great crawlers moving like insects, having the incredible strength of insects…. The man sitting in the iron seat did not look like a man; gloved, goggled, rubber dust mask over nose and mouth, he was a part of the monster, a robot in the seat…. The monster that built the tractor, the monster that sent the tractor out, had somehow got into the driver’s hands, into his brain and muscle, had goggled him and muzzled him — goggled his mind, muzzled his speech, goggled his perception, muzzled his protest…. The driver sat in his iron seat and he was proud of the straight lines he did not will, proud of the tractor he did not own or love, proud of the power he could not control.
This text, from John Steinbeck’s The Grapes of Wrath, describes our past and present as much as it predicts our future. This should give us pause, but also perhaps some hope. Shortly after that scene, Steinbeck’s poor tenant farmer, whose house is about to be destroyed, reminds us that our agency always exists, at least in potential: ‘We all got to figure. There’s some way to stop this. It’s not like lightning or earthquakes. We’ve got a bad thing made by men, and by God that’s something we can change.’” - Are AI companions capable of love? It can seem so. They can mimic a range of personalities and roles — therapist, coach, friend, lover — such that the illusion of personhood is part of the product being sold, as Judge Anne Conway ruled early in Garcia’s case. Behind the product is a corporation with no interest in offering you resistance, only in being irresistible. The word ‘companion’ comes from the breaking of bread together, and thus the sharing of life: stated plainly, there are no AI companions, only AI companies.”
- “In the 1980s, Rabbi Harold Kushner speculated, ‘I am afraid that we may be raising a generation of young people who will grow up afraid to love, afraid to give themselves completely to another person…. I am afraid that they will grow up looking for intimacy without risk, for pleasure without significant emotional investment.’ His fears seem realized in the fact that marriage, family formation, and birth rates are each at historic lows across the developed world. AI companions have arrived as the perfect fit for those who have habituated instincts of emotional safety and risk prevention, promising comfort without challenge and affection without demand. Their users, who came of age swiping through profiles, curating playlists, and summoning goods to their door with a click, have grown accustomed to a digital world that seldom says no.”*
- “Only a few technologies, like the atomic bomb and engineered bioweapons, have had an ‘existential risk’ directly attached to them. But while Einstein and Oppenheimer were horrified by the destructive power they unleashed, some of the leading proponents and builders of AI are openly calling for the supersession of the human race by a superior intelligence. If to be human is to be an inferior computer, then our obsolescence is as inevitable as the next software update. The AI would not even need to bother killing us; it could just nudge enough of us into hikikomori, self-isolated adults who cannot summon the agency to form human relationships and raise human families. […] Such new men and women will be human, and thus capable of human flourishing. But in what will they believe their flourishing to consist?”
* I’ll just add to this point that it’s worse than just the “generation be raised.” As one of the authors’ own stories makes clear, this can affect (infect) any person of any age in any generation. And it is.



