Winning a human-likeness contest is not passing Turing's test

The design

The Loebner Prize's own home page, archived shortly after that year's event, states that 'the 10th Annual Loebner Prize was held at Dartmouth College on 28 January 2000... Dr. Richard Wallace's program Alice was the winner.' The page explains the contest's rules: Hugh Loebner agreed in 1990 to fund an annual award of $2,000 and a bronze medal for 'the most human computer' judged that year, 'irrespective of how good it is in an absolute sense,' reserving a separate $100,000 grand prize for a program judges cannot distinguish from a human at all, a threshold the page records no entrant meeting. Wallace's program, A.L.I.C.E. (Artificial Linguistic Internet Computer Entity), is described on the A.L.I.C.E. AI Foundation's own page as running on AIML, an 'Artificial Intelligence Markup Language' pattern-matching engine, released as free software.

What the evidence says

The foundation's own history page credits originator Richard S. Wallace directly and situates AIML in a lineage running 'From Eliza to A.L.I.C.E.,' describing Wallace as having 'elaborated and expanded' Weizenbaum's earlier approach. Neither primary source claims A.L.I.C.E. passed the Turing test as Turing originally proposed it, an unbounded test with expert judges probing indefinitely. The prize's own rules describe a bounded, judged, comparative contest decided among that year's entrants, and the foundation's about page tells visitors, in its own words, to 'remember you are really chatting with a machine.' Winning the annual prize is evidence of relative conversational polish among a small field on one day, not general indistinguishability from a person.

What it asks of people

A.L.I.C.E.'s free, open architecture asked something different of people than a shipped product does: contributors donated conversational patterns to expand the bot's coverage, and the foundation's materials describe 'hundreds of contributors from around the world.' A visitor chatting with A.L.I.C.E. was asked for nothing beyond typed input; no account, payment or persistent profile is described as required.

Privacy and safeguards

Because A.L.I.C.E. was distributed as free software under the GNU General Public License rather than run as one company's hosted service, the foundation's pages describe no centralized collection of identifying user data as a later commercial companion app would require. The caution for a reader today is definitional: a contest win branded 'human-like' is a comparative judgment about that year's field, not proof of general machine understanding.

• Does a claim of 'passing' a human-likeness contest specify which contest, which year's field, and which judges?
• Is a chatbot's fluency evidence of understanding, or of a large, well-tuned library of matched patterns?
• What do the product's own materials say about whether a user is told they are talking to a machine?

The Loebner Prize's own history and the A.L.I.C.E. AI Foundation's own pages describe a real, judged, annually repeated contest and a real, influential piece of open-source pattern-matching software. Neither document supports treating a win in that contest as equivalent to passing the unbounded test Turing proposed, and a reader evaluating any later chatbot's Turing-test claim should ask the same question this entry asks of A.L.I.C.E.: which test, exactly.