t h e   t u r i n g   t e s t   p a g e

* welcome *

* what's new *

* introduction *

* tt-references online *

* background reading *

* loebner prize contest *

* talk to some programs *

* people *

* interesting links *

* about this page *

Rene Magritte
download the story of the Turing Test

* top *
Welcome to The Turing Test Page !

This page contains all the online information that
I could find concerning the Turing Test.
It is, and will always be under construction.
I would like to remark at the outset that this is not
a page about Alan Turing himself. For more
information on Alan Turing visit
The Alan Turing homepage

If you have questions or comments, know of resources
that belong here or have detected broken links on
this page please e-mail me.

I hope you find this page useful.

Hex wrote: +++ Hi Mum is Testing +++ MELON MELON MELON +++ Out of Cheese Error +++ !!!!! +++ Mr Jelly! Mr Jelly!
"Hex seems perfectly able to work out anything purely to do with numbers but when it tries anything else, it does this," said Ponder.

--- Terry Pratchett, Hogfather.

* top *

  • "Turing Test: 50 Years Later" :  An interdisciplinary review of the Turing Test. I finally put this online!

    Citation info: Saygin, A.P., Cicekli, I. and Akman, V. (2000), 'Turing Test: 50 Years Later', Minds and Machines 10(4): 463-518.

    *   Abstract
    *   Download (ps)
    *   Download (pdf)

    Note: If you have ftp problems, please let me know and I'll send you a copy via e-mail.

  • NEWS! Hugh Loebner announced that the next (and last?) Loebner Prize will be in New York in September of 2004.

  • NEW CHAT LINK! "TuringHub"

  • NEW PAPER AND LINK! Some Challenges And Grand Challenges For Computational Intelligence, Edward Feigenbaum, Journal of the ACM (JACM), 2003, 50(1).

  • NEW LINK! Oct 23 2003 article in The Guardian

  • NEW LINK! The Turing Test Bet - Post moved to 2029? Follow the link... You can also read about more tech bets in this WIRED article

  • The Future of the Turing Test: The Next 50 Years TIME IS UP!!! A conference at Dartmouth College to be held January 28-30, 2000 along with the 2000 Loebner Prize Contest Postscript: I was there! It was a good conference with many interesting discussions...

  • Mapping Great Debates: Can Computers Think? A must see! 7 large posters that aim to summarize the "Can Computers Think?" debate on nicely structured argumentation maps. Map 2 is on the Turing Test, but all are relevant. These are available online but you cannot read the arguments and links properly. I finally got a chance to see the real maps and liked them a lot. (I almost missed an appointment looking through the chart on the TT...:-))

    Coming soon: Want more on the posters? I wrote a review of them for the journal Minds and Machines...

    * top *


    The Turing Test was introduced by Alan M. Turing (1912-1954) as "the imitation game" in his 1950 article (now available online) Computing Machinery and Intelligence (Mind, Vol. 59, No. 236, pp. 433-460) which he so boldly began by the following sentence:

    I propose to consider the question "Can machines think?" This should begin with definitions of the meaning of the terms "machine" and "think."

    Turing Test is meant to determine if a computer program has intelligence. Quoting Turing, the original imitation game can be described as follows:

    The new form of the problem can be described in terms of a game which we call the "imitation game." It is played with three people, a man (A), a woman (B), and an interrogator (C) who may be of either sex. The interrogator stays in a room apart from the other two. The object of the game for the interrogator is to determine which of the other two is the man and which is the woman. He knows them by labels X and Y, and at the end of the game he says either "X is A and Y is B" or "X is B and Y is A." The interrogator is allowed to put questions to A and B.

    When talking about the Turing Test today what is generally understood is the following: The interrogator is connected to one person and one machine via a terminal, therefore can't see her counterparts. Her task is to find out which of the two candidates is the machine, and which is the human only by asking them questions. If the machine can "fool" the interrogator, it is intelligent.

    This test has been subject to different kinds of criticism and has been at the heart of many discussions in AI, philosophy and cognitive science for the past 50 years.

    * top *


    David Chalmers has prepared a comprehensive bibliography of the philosophy of Artificial Intelligence. Here is the section on the Turing test.

    Objections to the Turing test: Is the Brain a Digital Computer? Searle's Chinese Room

    Robert French's paper: Subcognition and the Limits of the Turing Test pdf (Mind 99(393) 53-65,1990; reprinted in P. Millican & A. Clark (eds.). Machines and Thought: The Legacy of Alan Turing Oxford, UK: Clarendon Press (1996).) One of the most important arguments against the Turing Test is that it only provides a test for human intelligence.

    Stevan Harnad's paper: The Turing Test Is Not A Trick: Turing Indistinguishability Is A Scientific Criterion. (SIGART Bulletin 3(4),9-10, October 1992)

    Another paper by Stevan Harnad: Other bodies, Other minds: A machine incarnation of an old philosophical problem. Minds and Machines, Vol. 1, pp. 43-54.

    One more selected reading from Harnad Minds, Machines and Turing: The Indistinguishability of Indistinguishables Journal of Logic, Language, and Information 9(4):425-445.

    Paper by Larry Hauser: Reaping the Whirlwind: Reply to Harnad's `Other Bodies, Other Minds' Minds and Machines, Vol. 3, No. 2 (May, 1993), pp. 219-238.

    Technical Report by Phillip G. Bradford and Michael Wollowski The Formalization of the Turing Test, available in postscript and pdf. 1994.

    NEW! By Bruce Edmonds:The Constructability of Artificial Intelligence (as defined by the Turing Test), Journal of Logic Language and Information, 9:419-424.

    NEW! By Judith Donath: Being Real, html format. To appear in Goldberg, K. (ed.) The Robot in the Garden: Telerobotics and Telepistemology in the Age of the Internet, MIT Press

    Paper by Jason Eisner Cognitive Science and the Search for Intelligence , pdf format. Paper presented to the Socratic Society, University of Cape Town, South Africa, 1991.

    Blay Whitby's paper: Why The Turing Test is AI's Biggest Blind Alley Whitby argues that AI should not be too distracted into direct copying of human performance and methods.

    Summary of a talk by Pat Hayes: Abandoning the Turing Test (postscript)

    A paper by Jean Lessegue: What kind of Turing Test did Turing have in Mind? (Tekhnema, 3, 37-58, 1996)

    An essay by David Barberi: The Ultimate Turing Test

    WIRED Article by Charles Platt: What's It Mean to be Human, Anyway?

    WIRED Article by Richard S. Wallace: The Lying Game

    An essay by Warren Sack: Replaying Turing's Imitation Game Paper presented at the panel "Nets and Internets" at Console-ing Passions: Television, Video and Feminism, April 25-28, 1

    Stephen J. Cowley and Karl F. MacDorman's paper, Simulating Conversations: The Communion Game (AI&Society, 9, 116-137, 1995)

    CogPrints Electronic Archive Many relevant papers on psychology, neuroscience, linguistics and computer science can be found here.

    Psycholoquy papers on the Turing Test

    You can also take a look at the Simon Laven Page for chatterbot related books and papers (as well as an excellent chatterbot resource, see below)

    * top *


    The Turing Test and Chinese Room Experiment A nice, concise page by Larry Hauser describing the Turing Test and the Chinese Room of Searle.

    Minds, Machines and Searle Stevan Harnad (Minds, Machines and Searle. Journal of Theoretical and Experimental Artificial Intelligence, 1989, 1: 5-25.

    Artificial Intelligence: A Philosophical Introduction by Jack Copeland. Oxford: Blackwell, 1993.

    Minds, Machines and Godel J.R. Lucas (Philosophy, XXXVI, 1961, pp.112-127)

    Minds, Brains and Science John Searle (Harvard University Press, Cambridge, Mass. ,1984)

    Thinking Computers and Virtual Persons Eric Dietrich (ed.)

    Android Epistemology Ken Ford, Clark Glymour, and Patrick J. Hayes (eds.) Menlo Park, CA: AAAI/The MIT Press.

    Alan Turing: The Enigma Andrew Hodges (ISBN 0-09-911641-3 Vintage, Random Century, London.)

    Paper by Larry Hauser: Why Isn't My Pocket Calculator a Thinking Thing Minds and Machines, Vol. 3, No. 1 (February, 1993), pp. 3-10.

    * top *


    In 1991 Dr. Hugh Loebner started the annual Loebner Prize competition. A $100,000 prize is offered to the author of the first computer program to pass an unrestricted Turing test.

    The Loebner Prize Homepage

    Loebner Prize 2003

    Jason Hutchens' MegaHAL page contains some fun transcripts and good information. Hutchens' programs participated in the Loebner contests in 1996, 97 and 98 and one of them won the prize in 1996.

    Robby Garner, Robitron has also participated in the contest several times and is the winner of the 1998 and 1999 Loebner prizes. His site contains a lot of cool information and links.

    Michael Mauldin's paper: Chatterbots, Tinymuds, And The Turing Test: Entering The Loebner Prize Competition This paper describes the development of a program and its performance on the first three Loebner Prize competitions.

    Stuart Shieber's paper: Lessons from a Restricted Turing Test CACM, volume 37, number 6, 1994.

    Hugh Loebner's response to Stuart Shieber: In Response

    A human confederate's perspective: Tracy Quan tells her story.

    Guardian article: Oct 6, 2001 - It's the thought that counts

    NY Times article: March 18, 1999 - Look what's talking

    * top *


    Note: Commercial bots (e.g. help services for special companies, customer service applications) are not listed here, although they can be reached from some of the links.
    I realize that I may not have been so good at keeping track of all the chatbots out there. Here are a few links to some sites that may be more frequently updated:

    Turing Hub

    Alice AI foundation


    And you MUST see The Simon Laven Page probably the most comprehensive chatterbot site on the web...


    Jabberwock, the winner of Loebner 2003

    Jabberwacky, a bot with character

    The CHAT Natural Language System (Loebner 94, 95)

    ELIZA, the oldest girl around.

    JULIA was a participant in Loebner 94.

    ALICE is another chatty girl. Beware, this one spreads gossip! Winner of 2000 and 2001 Loebner prizes.

    SPENCE'S, a virtual bar... Here, you can talk to Spence or to Erin, his niece who is also the bartender.

    MITBOLEL A customizable Java chatbot. You can even put Mitbolel on your own web page!

    THOUGHT TREASURE The approach here is a bit different. Thought Treasure has some "world knowledge" (an extensive concept hierarchy). This is an evolving system, latest version available for download.

    MeBOT You need a Java compatible browser to talk to MeBot.

    BRIAN, an 18 year old college student (3rd place in Loebner 1998)

    MIMIC Help him learn talking, he's willing to learn.

    Mr. MIND Try to convince him that you are not a computer.

    DR ABUSE An ELIZA-based program that talks in Spanish

    ELOISA You want to chat in Italian? Visit her.

    * top *

    Ned Block Online papers. Also check out the homepage of the course Minds and Machines prepared by Block.

    Selmer Bringsjord This site contains information on his books and some online papers and presentations.

    Rodney Brooks Here you can find out about Brooks' research on building intelligent robots.

    David Chalmers A comprehensive site that contains many useful resources on the philosophy of mind as well as information on Chalmers' works on consciousness

    Jack Copeland His personal page contains online papers as well as links to his Turing-related projects and conferences

    Daniel Dennett Here, you can find a lot of papers by Dennett in html format.

    Robert French Contains many papers of the author, downloadable in pdf format.

    Stevan Harnad E-Prints on Cognitive Psychology. Many relevant papers online.

    Larry Hauser Contains numerous online papers by Hauser. Also an excellent philosophy of mind resource.

    Patrick Hayes Information on his research, although not too many papers online.

    John R. Lucas Papers on various topics, online.

    John McCarthy Personal homepage with various links to online publications.

    Donald Michie Personal homepage containing full list of publications, but none are online.

    Marvin Minsky Personal homepage with some online publications.

    James Moor Personal homepage. No online publications unfortunately. Information on the Loebner Prize 2000 and The Future of the Turing Test Conference

    Aaron Sloman His personal page with some online resources.

    * top *

    NEW! AlanTuring.net

    NEW! KurzweilAI.net

    NEW! The Turing Test Science Contest !

    NEW!Turing Tournament at the California Institute of Technology.

    Preliminary Turing Test Analysis Interesting paper drawing a parallel between the Turing Test and GIS/spatial technologies research. Not really relevant to our topic, though.

    The Automatic Confession Machine, or The Catholic Turing Test

    How my program passed the Turing Test by Mark Humphrys.

    MAW'97 Turing Test page This page was prepared within the context of Math Awareness Week '97.

    Lecture slides This and the following few slides give a nice, concise summary of the Turing Test.

    Characters, improvisation and ... Lots and lots of links.

    Botspot Go there for all bot-related stuff on the net.

    Banai's Turing Test Page

    Machine Intelligence Part1: The Turing Test and Loebner's Prize

    Psycholoquy A refereed international, interdisciplinary electronic journal.

  • * top *

    Copyright 1998-2003  Ayse Pinar Saygin
    Created April 6, 1998

    Last updated: Oct 27, 2003

    get e-mail when this page is updated.
    the list is moderated and restricted so you won't be bombarded with irrelevant mail.

    Image on the top of the page is
    "Le Double Secret" by Rene Magritte
    Modified with CorelXARA 2.0
    No frames were used on this site
    Ayse Pinar Saygin

    * e-mail *

    * top *

    * home *