Frank Kendall, former Secretary of the Air Force:
The tool Anthropic is providing to the government is enormously powerful; like other tools, it can inherently be used for good or evil. Anthropic is rightly concerned that its tool could be used in ways that are unsafe or malicious. The company doesn’t want to see its A.I. model used without human control, which could result in the killing of noncombatants or friendly troops by automated weapons, nor deployed to spy broadly on Americans in ways that could violate dearly held values like privacy and freedom from illegal search and seizure or could suppress political dissent. Most Americans would probably agree.
Aww. So very cute; so utterly stupid.
How, then, do we account for the fact that science has become so risky an enterprise that some of the world’s most distinguished scientists consider it to constitute the chief threat to the survival of humanity? This question must now be regarded as taking precedence above all others. Many philosophers reply to it by saying that Descartes’s dream—of putting man in the place of God, as the master and possessor of nature— turned into a nightmare, with the result that mastery is now itself in urgent need of being mastered. I fear that they have not the least understanding of what is really at issue. They fail to see that the technology now taking shape at the intersection of a great many fields aims precisely at non-mastery. I repeat: the engineer of tomorrow will be an apprentice sorcerer not by negligence or incompetence; he will be one deliberately. He will begin by imagining and designing complex organisms in the form of mathematical models, and will then try to determine, by systematically exploring the landscape of their functional properties, which behaviors they are capable of supporting. In adopting a “bottom-up” approach of this kind, he will be more an explorer and an experimenter than a builder; his success will be measured more by the extent to which these creatures surprise him than by their agreement with a set of preestablished criteria and specifications. Fields such as artificial life, genetic algorithms, robotics, and distributed artificial intelligence already display this character. In the years ahead the aspiration to non-mastery threatens to reach fruition with the demiurgic manipulation of matter on the atomic and molecular scale by nanotechnologies. Moreover, to the extent that the scientist is now likelier to be someone who, rather than seeking to discover a reality independent of the mind, investigates instead the properties of his own inventions (more as a researcher in artificial intelligence, one might say, than as a neurophysiologist), the roles of engineer and scientist will come to be confused and, ultimately, conflated with each other. Nature itself will become what humans have made of it, by unleashing in it processes over which, by design, there is no mastery.
(Dupuy’s book was originally published in French in 2008; English in 2013.)