Most Singularitarians have positive visions about the future with artificial general intelligence, but James Hughes of the Institute for Ethics and Emerging Technologies sees a different future. In this talk from the 2007 Singularity Summit, he describes some ways that AGI will feel different and alien from us, and the negative effects that it could have on society.
While humans are the most familiar example of intelligence, organizations provide another form of intelligence that we can look to when we imagine how AGI will behave. Companies and governments know more than one person can know and replace their components the same way biological organisms do, but they don't act like human culture. Like organizations, AGI will both magnify human intentions and develop its own, that might or might not coincide with human interests.
Even if artificial intelligence doesn't surpass human intelligence, it can still pose problems for society. Combine a botnet that defends itself from security measures with the annoyance of dumb biological organisms like cockroaches and you have a lot of potential problems. Intelligent augmentation (IA) should be considered as an alternative to AI because of the benefits of having human brains as control systems.
Dr. James "J." Hughes serves as the executive director of the Institute for Ethics and Emerging Technologies, where he produces the weekly syndicated public affairs talk show Changesurfer Radio. Dr. Hughes is a sociologist who teaches Health Policy at Trinity College in Hartford Connecticut. He is the author of Citizen Cyborg: Why Democratic Societies Must Respond to the Redesigned Human of the Future, and is working on a second book on neurotechnology, tentatively titled Cyborg Buddha: Using Neurotechnologies to Enhance Virtue. He is editor of the Journal of Evolution and Technology, a Fellow of the World Academy of Arts and Sciences, and a member of the American Society of Bioethics and Humanities and the Working Group on Ethics and Technology at Yale University. Dr. Hughes speaks on medical ethics, health care policy, and future studies worldwide, and often appears on radio and television. He lives in rural eastern Connecticut with his wife, the artist Monica Bock, and their two children.
This free podcast is from our Singularity Summit series.