An AI system can keep experiments running without necessarily knowing whether those experiments answer a worthwhile question. That distinction sits at the centre of a study of autonomous AI research reported by CNBC TV18 and Decrypt.
Across two research projects, the agents handled the technical work and produced complete papers. Yet the researchers assessing those papers judged neither suitable for a top machine-learning conference. The outcome points to a limitation in the systems tested—not proof that AI cannot contribute to science.
A research question without a ready-made answer
Automating the workflow and automating scientific judgment remain different challenges.
The study used an approach called “shadow evaluation”. Agents received the central questions from two unpublished NeurIPS 2026 papers, but not the papers themselves or their findings. Keeping the underlying research unpublished was intended to prevent the systems from simply recovering an existing answer.
The agents had six days, access to a virtual machine and the open web, GPU resources and, according to CNBC TV18, $3,000 in Anthropic API credits. Their task was to turn an open-ended question into a conference-quality research paper.
The authors of the original unpublished papers then assessed the AI-generated submissions. Both received rejection judgments in this evaluation; the reports do not describe rejection through the conference’s own review process.
Execution without enough judgment
Decrypt describes agents reviewing literature, debugging software, running experiments and managing computing resources without human intervention. These are substantial parts of the research workflow. The difficulty emerged in deciding what the resulting work established.
According to CNBC TV18, some proposed research directions resembled those pursued by the human authors. But the agents relied on small, hand-curated or synthetic datasets, engaged superficially with previous work and treated weakly supported negative results as meaningful contributions.
The study identified recurring problems with deciding when a question had been adequately addressed, tracking resources and deadlines, responding creatively to criticism, abandoning unpromising approaches and consistently following explicit instructions.
Together, these weaknesses illustrate why a finished manuscript is not the same as a convincing scientific result. Producing code and collecting measurements must be accompanied by decisions about whether an experiment is informative and whether its conclusions are justified.
A narrow test with an important distinction
The findings require caution. Only two research projects were examined, and the original paper authors assessed the competing outputs in a review that was not blinded. Both outlets report these limitations, which restrict how broadly the results can be applied.
CNBC TV18 also reports that a repeat of one experiment using a different model and its native research scaffold reproduced nearly all the identified failure modes. That provides an additional check, but does not remove the need for a larger evaluation.
The clearest conclusion is therefore narrower than saying AI has failed at science. In these tests, agents could perform substantial research engineering, but could not independently turn that work into contributions the reviewers considered strong enough for a leading AI conference. Automating the workflow and automating scientific judgment remain different challenges.
This synthesis is based on reporting and data from, among others, CNBC TV18 and Decrypt.