The Claim: GPT-4.5 Deceives Humans

Recent news reports highlight a preprint study from the University of California San Diego, where researchers subjected four large language models (LLMs) to the Turing test. The study suggests that OpenAI’s GPT-4.5 was indistinguishable from a human more than 70% of the time.

The Turing test, popularized as the ultimate benchmark for machine intelligence, has a complex history and its validity is often debated.

The Study: Methodology and Results

Cognitive scientists Cameron Jones and Benjamin Bergen conducted the study, which is currently awaiting peer review.the LLMs tested included ELIZA, GPT-4o, LLaMa-3.1-405B,and GPT-4.5.

• Test Setup: Participants engaged in eight rounds of conversations, acting as interrogators tasked with distinguishing between a human and an LLM.
• Interface: The test interface mimicked a conventional messaging app, with participants interacting with both witnesses together across a split screen for five minutes.
• Outcome: Participants identified GPT-4.5 as human 73% of the time, and LLaMa-3.1-405B 56% of the time. ELIZA and GPT-4o only fooled participants 23% and 21% of the time, respectively.

The Turing Test: A Historical Perspective

Alan Turing, a British mathematician and computer scientist, first introduced the concept in his 1948 paper, Smart Machinery. initially, it was conceived as an experiment involving three peopel playing chess with a theoretical “paper machine.”

Turing refined the experiment in his 1950 publication, Computing Machinery and Intelligence, renaming it the “imitation game.” This version involved three participants: a woman (A), a man (B), and an interrogator (C) of either gender. The interrogator’s task was to determine whether “X is A and Y is B” or “X is B and Y is A” through a series of questions.

Turing proposed replacing the question, Can machines think? with this game.He argued that the terms machine and think were too ambiguous for a meaningful answer.

Criticisms and Contentions

Despite its popularity, the Turing test faces notable criticism. Key objections include:

• Behavior vs. Thinking: Critics argue that passing the test reflects behavioral mimicry rather than genuine intelligence. A machine could pass the test without actually “thinking.”
• Brains vs. Machines: Turing’s assertion that the brain is a machine explainable in purely mechanical terms is contested. Many academics disagree, questioning the test’s foundation.
• Internal Operations: Computers reach conclusions differently than humans. this makes direct comparisons inadequate,undermining the test’s validity.
• Scope of the Test: Assessing intelligence based on a single behavior is insufficient, according to some researchers.

The Verdict: Imitation vs. Intelligence

While the preprint study suggests GPT-4.5 passed the Turing test, the researchers themselves acknowledge its limitations:

The Turing test is a measure of substitutability: whether a system can stand-in for a real person without […] noticing the difference.

This suggests the test measures the imitation of human intelligence rather than actual intelligence. The study’s conditions, such as the short five-minute testing window and the use of specific personas for the LLMs, also raise questions.

For now, it is reasonable to conclude that GPT-4.5, while impressive, is not as intelligent as humans. However, it can convincingly mimic human conversation, potentially deceiving some individuals.